Deep learning with Neural Network and Tensorflow

The idea of neural network

biological neural network:

• Dendrites
• Axon
• Axon terminal
• Synapse(递质)

Artificial network:

for each neuron

• input x1,x2,x3…
• sum them with weights
• threashold them, step function or sigmold function
• output 1 or 0

A neural network is formed by layers of neurons with hidden layers, from input to output are hidden layer 1, 2, 3… Neural network with more than 1 hidden layer is considered a deep neural network.

The input data need to be huge, 60,000 could be still pretty small. b.c. Not like SVM, which only has two parameters v and b, and convex, the optimization problem of neural network is not convex, there are a lot of independent weights that we need to determine

We still dont know what is happening in the hidden layers, why they have such good performance.

Tensorflow

Syntax are different in different frameworks, the modelling are the same.