# Activating ReLU function in the hidden layers

Our neural network, which is not densely connected, has 2 hidden layers with 8 neurons each. There’s only a one-to-one correspondence between each neuron in the input, layer 1, layer 2, and the output. We will use both hidden layers to fit the line to the sine wave function.

We start by assigning baseline values–giving us this output.

Setting the hidden layers’ `weight`

values to `1.0`

results in this linear line (i.e. weights tend to affect the slope of the line).

Increasing the weights increases the slope even more.

Increasing *layer 2* ’s `bias`

by a half nudges the entire line up by a half.

Flipping *layer 1* ’s weight results in flipping the line (*explanation after the illustration).

ReLU note: `y = 0 if x <= 0 else x`

that’s why the line does not descend past `0`

### Code

NumPy:

Moving on. Let’s flip *layer 2* ’s weight to flip the line vertically.

To move that whole line upward by a half, we set the `bias`

of *layer 2* of the bottommost neuron to `0.5`

.

At this stage we have completed the **first section** of the process. The slope of the leftmost line, more or less, follows the contour of that section of the sine wave.

The aim is to get to this stage where every section follow the form of the sine wave.