First, I am nobody in this topic, please don’t hesitate to give better suggestion. This is actually just a “documentation” about my experience when I work on my undergraduate thesis in 2015. Once again, please don’t hesitate to discuss and give better suggestion about this topic, or.. correct my English if I made some mistakes 🙂
Background
Okay, before I tell you my story, I’ll tell you about my background when I started to learn neural network. I wrote this section only have one intention, to make you easy measure what you need, nothing else.
- I have familiarity with some of mathematical symbols, like how to read sigma symbol, derivative symbol and many others (but not every symbols).
- I have familiarity with basic math operation like derivative function, algebra, matrix and vector operation (but only “the basic”).
- I’ve never coded in python, but I’ve been using C++ around 2-3 years and I used to code Matlab style programming language in some project before (and thanks, this knowledge is very helpful)
- I studied the Introduction of Artificial Intelligence at the fourth semester in college, but I won’t said I did it well 🙁
After a month googling randomly, I started to make a list what should I learn, the order of the list below it’s just the best order in my opinion. In my experience I jumped many times over the topic because at that time, I didn’t know what I didn’t know 🙁
If you want to learn about Deep Learning, lets begin with: What is Machine Learning?
- Easy start to get the big picture : http://www.r2d3.us/visual-intro-to-machine-learning-part-1/
- Start to learn the basic math of machine learning : https://www.coursera.org/learn/machine-learning (I didn’t watch all videos, only the first three weeks but I did the assignment too)
Then start to learn about Neural Network
- Easy start to get the big picture : https://www.youtube.com/playlist?list=PLiaHhY2iBX9hdHaRr6b7XevZtgZRa1PoU (I watched all videos even I didn’t understand at that time)
- After I knew the big picture of neural net I also read the simple history of neural net, and the “basic” of neural net, I think this gave me a few intuition. I read a book “Fundamentals of Neural Network” by Fausset, only the first two chapters
- But, I think you can get the “alternative version” what I got from Fausset’s book in the first chapter from http://neuralnetworksanddeeplearning.com/ (try to read the first three chapters)
- You can also follow the first modul from http://cs231n.github.io/ and some chapters from http://deeplearning.stanford.edu/tutorial/
- Other tutorial from stanford gave me introduction to neural network for deep learning http://cs.stanford.edu/~quocle/tutorial1.pdf
Let’s do some math, Backpropagation?
I suggest you to fully-understand this algorithm in order to give some intuitions to help you in model tuning or error handling when you start implement the neural network
- Easy start : I watched again the neural network demistifyed, and slowly got the intuition how backpropagation work, I followed and tried the math step-by-step
- Watch the 4th week on Coursera Introduction to Machine Learning
- Try the example of how backpropagation works : https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
- Read carefully the second chapter from http://neuralnetworksanddeeplearning.com/
- And if you start thinking they have different formula / calculation / interpretation, start to prove that you are wrong. I started to compare all the calculations to understand more
Python for machine learning?
- Learn the basic of python. I forgot how I learn my first python syntax, I remember I use some interactive platforms like https://www.codecademy.com/learn/python then I practice and read some python code in http://codeforces.com‘s problemset
- Get some knowledge about Numpy https://docs.scipy.org/doc/numpy-dev/user/quickstart.html and because I already knew some matlab syntax I also read http://mathesaurus.sourceforge.net/matlab-numpy.html
- I started using python for machine learning by following kaggle tutorial https://www.kaggle.com/c/titanic/details/getting-started-with-python
- Then I started implement the neural network, first I followed this simple tutorial : http://iamtrask.github.io/2015/07/12/basic-python-network/ this tutorial will give you some insight about backpropagation too
- Because I want to use GPU for deep learning, I started to learn Theano from Theano’s site http://deeplearning.net/software/theano/ and this nice tutorial https://bigaidream.gitbooks.io/subsets_ml_cookbook/content/dl/theano/theano_tutorial.html
So what is Deep Learning?
- Easy start : read all chapter from http://neuralnetworksanddeeplearning.com/
- The second tutorial from stanford : http://cs.stanford.edu/~quocle/tutorial2.pdf
- Amazing book from Ian goodfellow and friends : http://www.deeplearningbook.org/
- start to understand the meaning of “deep learning”, you can try by read a lot of articles on the internet
What is Convolutional Neural Network (I use CNN in my undergraduate thesis)?
- The second module from http://cs231n.github.io/ give nice explanation
- And also try http://ufldl.stanford.edu/tutorial/supervised/ConvolutionalNeuralNetwork/
- And the great video that gives me a lot of new knowledge about CNN : https://www.youtube.com/watch?v=cDdpwAIsuD8
Backpropagation in CNN?
I don’t know it is necessary or not, but I learned it. I think if you understand the basic of backpropagation it’s enough. But if you are curious how backpropgation through max pooling or convolution layer, it does not hurt to learn it.
- Okay, this part is a lot of math, you can start from read carefully http://ufldl.stanford.edu/tutorial/supervised/ConvolutionalNeuralNetwork/
- Then try some cute math from http://andrew.gibiansky.com/blog/machine-learning/convolutional-neural-networks/ and http://timdettmers.com/2015/03/26/convolution-deep-learning/ and from http://cogprints.org/5869/1/cnn_tutorial.pdf
And…great, It’s done, That’s all my references that I used when I did my undergraduate thesis, I hope it will be useful for someone :/ or at least for me in the future 🙂