Feedback about Building A Handwritten Digits Classifier project

Hello,
I have done the Building A Handwritten Digits Classifier project and need your ideas.
I tried to visualize the weights of neural networks but it does not make sense. I have two main questions:

  1. Why visualizing the weights? What should I expect to see?
  2. Why the output layer has 10 neurons? (each neuron for each digit?!) I expect to be one neuron that gives us the predicted digit (0-9) but if for example, I have two hiddle layers with 256 neurons the shape of coef_ will be:
    (64 * 256)(256 * 256)(256 * 10)

This is the link to my project and I appreciate any guidance or resources about visualization of the weights.

Thanks,
Nasim

1 Like

Hi @jafarinasim,

Cool project you’ve got there! I’ve only started with Machine Learning introductions here but I’m really looking forward to building what you have built!

Regarding your questions, I came across this video from 3blue1brown on Youtube that really helped me get a better grasp on neural networks and I just revisited it, I think it answers both of your questions as to why visualizing the weights and why the output layer has 10 neurons. Besides, the visualizations are so well done it’s just fun to watch! Hope it will help!

Cheers!

3 Likes

@jafarinasim: in this case we use 10 neurons since we have 10 possible labels (0 through 9). We usually use 1 node only for binary classifications (outputs are Yes/No or 0/1, cat/not cat etc.). Having 10 neurons allow us to do a categorical (multiclass, single-label) classification so that we can output probabilities of the labels generated by the model to determine the model’s certainty in predicting each of the 10 labels.

Quoting this article:

The weights are useful to visualize because well-trained networks usually display nice and smooth filters without any noisy patterns. Noisy patterns can be an indicator of a network that hasn’t been trained for long enough, or possibly a very low regularization strength that may have led to overfitting.

Here is another interesting article:

3 Likes

Thank you @veratsien! Such a helpful video! Thanks again.

You are very welcome! I’m glad you find it helpful. :grinning:

Thank you @masterryan.prof for the answers and resources.
So the noisy patterns I have got can be a sign of overfitting. I have just added applying regularization methods to improve the model to the next step section and I hope to be able to go through it in the future.
Thanks again.

1 Like