Neural Networks Hands On

Open the source code for this page in your code editor.

Single layer fully connected neural network

Take a look at the code for the neural network below. The output data is incorrect: It only accounts for the biases, but not the weights. Can you fix it?

+0.0+10.0+100.0+1000.01.04.07.010.02.05.08.011.03.06.09.012.00101001000012

How many weights are there per layer?

Python

import numpy as np

W = np.array([[1, 2, 3], 
              [4, 5, 6], 
              [7, 8, 9], 
              [10, 11, 12]])
a = np.array([[0], [1], [2]])
b = np.array([[0], [10], [100], [1000]])

def sigma(x):
  return 1 / (1 + np.exp(-x))

def softmax(x):
  e = np.exp(x - np.max(x))  # stability trick
  return e / np.sum(e)

def loss(y_pred, y_true):
  return -np.sum(y_true * np.log(y_pred + 1e-9))  # cross-entropy

z = W @ a + b
h = sigma(z)
s = softmax(h)

# Suppose true class is the 4th one (one-hot encoded)
y_true = np.array([[0], [0], [0], [1]])

print("z =", z)
print("h =", h)
print("s =", s)
print("loss =", loss(s, y_true))