Neural Network Architecture

Neural networks are structures made of neurons that accepts multiple units to give a single output. They mimic the behavior of brain to solve data problems. Neuron, the basic unit accepts input from other nodes. The neural network consists of an input an output layer with several layers hidden in between. The primary function of the computation of neural networks happens in the hidden layer. 

Input is the group of features that are provided to the neural model for learning. For example, for face recognition, pixel values related to the face may be fed to the model 

The value from the input layer is sent to all the hidden nodes. The values which enter the hidden nodes are multiplied by pre-determined numbers carried in the program called weights. Weights are values assigned to each node based on the importance of the nodes in predicting the final value. 

The neurons influence each other. They can intricately study and observe huge volumes of data and find out extremely complex patterns 

Node values from the yellow layer* Weight= Value for the first hidden layer 

Summation function combines the weight and the inputs together 

The blue layer has an activation function which determines whether any node will be activated and how much they would be activated. The activation function induces non-linearity in the neural model. 

Let us understand this using the simple process of making tea. 

The components for making the tea – water, tea leaves, milk and sugar represent the neurons. 

The amount of each component to be used in the tea is denoted as the weight. 

When the tea is made, all the components transform into a different state. The transformation process denotes the activation function. 

Bias– Bias shifts the activation function either left or right. 

Flow of information in neural networks 

1. Feed forward- Information flows in a single direction from input nodes to hidden nodes and to output nodes 

2. Feedback- In this the network has recurrence i..e the signal is directed towards the neuron or layer that has already processed the specific signal 

Different types of neural network based on flow of information  

Single layer feed forward networks- In this network, there are only two layers input and the output layer. Different weights are put on the input nodes to form the output layer. The neurons form the output layer for computing the output signals 

Multiple feed forward network- This network has a hidden layer which is not connected to the external layer. The presence of one or two hidden layers  provides strong computational strength to the network 

Single node feedback networks- 

Here the output is sent as input to the same layer or subsequent layer nodes. 

Single layer recurrent network- This is a single network with connection for feedback. The output of the processing element can be directed to itself or different processing elements.