First, I am nobody in this topic, please don’t hesitate to give better suggestion. This is actually just a “documentation” about my experience when I work on my undergraduate thesis in 2015. Once again, please don’t hesitate to discuss and give better suggestion about this topic, or.. correct my English if I made some mistakes 🙂

**Background**

Okay, before I tell you my story, I’ll tell you about my background when I started to learn neural network. I wrote this section only have one intention, to make you easy measure what you need, nothing else.

- I have familiarity with some of mathematical symbols, like how to read sigma symbol, derivative symbol and many others (but not every symbols).
- I have familiarity with basic math operation like derivative function, algebra, matrix and vector operation (but only “the basic”).
- I’ve never coded in python, but I’ve been using C++ around 2-3 years and I used to code Matlab style programming language in some project before (and thanks, this knowledge is very helpful)
- I studied the Introduction of Artificial Intelligence at the fourth semester in college, but I won’t said I did it well 🙁

*After a month googling randomly, I started to make a list what should I learn, the order of the list below it’s just the best order in my opinion. In my experience I jumped many times over the topic because at that time, I didn’t know what I didn’t know 🙁*

**If you want to learn about Deep Learning, lets begin with: What is Machine Learning?**

- Easy start to get the big picture : http://www.r2d3.us/visual-intro-to-machine-learning-part-1/
- Start to learn the basic math of machine learning : https://www.coursera.org/learn/machine-learning (I didn’t watch all videos, only the first three weeks but I did the assignment too)

**Then start to learn about Neural Network**

- Easy start to get the big picture : https://www.youtube.com/playlist?list=PLiaHhY2iBX9hdHaRr6b7XevZtgZRa1PoU (I watched all videos even I didn’t understand at that time)
- After I knew the big picture of neural net I also read the simple history of neural net, and the “basic” of neural net, I think this gave me a few intuition. I read a book “Fundamentals of Neural Network” by Fausset, only the first two chapters
- But, I think you can get the “alternative version” what I got from Fausset’s book in the first chapter from http://neuralnetworksanddeeplearning.com/ (try to read the first three chapters)
- You can also follow the first modul from http://cs231n.github.io/ and some chapters from http://deeplearning.stanford.edu/tutorial/
- Other tutorial from stanford gave me introduction to neural network for deep learning http://cs.stanford.edu/~quocle/tutorial1.pdf

**Let’s do some math, Backpropagation?**

I suggest you to fully-understand this algorithm in order to give some intuitions to help you in model tuning or error handling when you start implement the neural network

- Easy start : I watched again the neural network demistifyed, and slowly got the intuition how backpropagation work, I followed and tried the math step-by-step
- Watch the 4th week on Coursera Introduction to Machine Learning
- Try the example of how backpropagation works : https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
- Read carefully the second chapter from http://neuralnetworksanddeeplearning.com/
- And if you start thinking they have different formula / calculation / interpretation, start to prove that you are wrong. I started to compare all the calculations to understand more

**Python for machine learning?**

- Learn the basic of python. I forgot how I learn my first python syntax, I remember I use some interactive platforms like https://www.codecademy.com/learn/python then I practice and read some python code in http://codeforces.com‘s problemset
- Get some knowledge about Numpy https://docs.scipy.org/doc/numpy-dev/user/quickstart.html and because I already knew some matlab syntax I also read http://mathesaurus.sourceforge.net/matlab-numpy.html
- I started using python for machine learning by following kaggle tutorial https://www.kaggle.com/c/titanic/details/getting-started-with-python
- Then I started implement the neural network, first I followed this simple tutorial : http://iamtrask.github.io/2015/07/12/basic-python-network/ this tutorial will give you some insight about backpropagation too
- Because I want to use GPU for deep learning, I started to learn Theano from Theano’s site http://deeplearning.net/software/theano/ and this nice tutorial https://bigaidream.gitbooks.io/subsets_ml_cookbook/content/dl/theano/theano_tutorial.html

**So what is Deep Learning?**

- Easy start : read all chapter from http://neuralnetworksanddeeplearning.com/
- The second tutorial from stanford : http://cs.stanford.edu/~quocle/tutorial2.pdf
- Amazing book from Ian goodfellow and friends : http://www.deeplearningbook.org/
- start to understand the meaning of “deep learning”, you can try by read a lot of articles on the internet

**What is Convolutional Neural Network (I use CNN in my undergraduate thesis)?**

- The second module from http://cs231n.github.io/ give nice explanation
- And also try http://ufldl.stanford.edu/tutorial/supervised/ConvolutionalNeuralNetwork/
- And the great video that gives me a lot of new knowledge about CNN : https://www.youtube.com/watch?v=cDdpwAIsuD8

**Backpropagation in CNN?**

I don’t know it is necessary or not, but I learned it. I think if you understand the basic of backpropagation it’s enough. But if you are curious how backpropgation through max pooling or convolution layer, it does not hurt to learn it.

- Okay, this part is a lot of math, you can start from read carefully http://ufldl.stanford.edu/tutorial/supervised/ConvolutionalNeuralNetwork/
- Then try some cute math from http://andrew.gibiansky.com/blog/machine-learning/convolutional-neural-networks/ and http://timdettmers.com/2015/03/26/convolution-deep-learning/ and from http://cogprints.org/5869/1/cnn_tutorial.pdf

And…great, It’s done, That’s all my references that I used when I did my undergraduate thesis, I hope it will be useful for someone :/ or at least for me in the future 🙂