Research On Non-Linearity In First Hidden Layer

Hi, last year I have been researched on the Non-Linearity in the first hidden layer. So that it can converge the faster without the need of more hidden layer. I am not going to explain more detail about this thing, but I am going to show a visualization through the playground tensorflow, So that you know the power of the Non-Linearity.

I recommend you to read my blog ‘why activation is used and also important of first hidden layer’ So that you get better intuition between linearity and Non-Linearity in the first hidden layer.

Why activation Function is used in neural network

Why First Hidden Layer Is Very Important In Building a Neural Network Model And Relation Between…

Let’s start the experiment

Now we will train a dataset with linear input with only one hidden layer and also with a single neuron in the first hidden layer.

As you see that it cannot separate the nonlinear dataset due to the single neuron in a first hidden layer can handle only linear dataset so if you want to classify non-linearity we need more neuron in the hidden layer and also we need more hidden layer if the dataset is a more complex non-linear dataset.

Now we give the non-linearity to the first hidden layer by converting the input to non-linearity and passing to the first hidden layer.

As you can see it can easily separate the dataset

As you can see that If we add non-linearity it can easily separate the dataset, But I will not say that Non-Linearity is perfect there is so much research to go on it.

note: some small green glitch is there due to some bug

Thanks for reading my blog…….