First, I am nobody in this topic, please don’t hesitate to give better suggestion. This is actually just a “documentation” about my experience when I work on my undergraduate thesis in 2015. Once again, please don’t hesitate to discuss and give better suggestion about this topic, or.. correct my English if I made some mistakes 🙂

**Background**

Okay, before I tell you my story, I’ll tell you about my background when I started to learn neural network. I wrote this section only have one intention, to make you easy measure what you need, nothing else.

- I have familiarity with some of mathematical symbols, like how to read sigma symbol, derivative symbol and many others (but not every symbols).
- I have familiarity with basic math operation like derivative function, algebra, matrix and vector operation (but only “the basic”).
- I’ve never coded in python, but I’ve been using C++ around 2-3 years and I used to code Matlab style programming language in some project before (and thanks, this knowledge is very helpful)
- I studied the Introduction of Artificial Intelligence at the fourth semester in college, but I won’t said I did it well 🙁

*After a month googling randomly, I started to make a list what should I learn, the order of the list below it’s just the best order in my opinion. In my experience I jumped many times over the topic because at that time, I didn’t know what I didn’t know 🙁*

**If you want to learn about Deep Learning, lets begin with: What is Machine Learning?**

- Easy start to get the big picture : http://www.r2d3.us/visual-intro-to-machine-learning-part-1/
- Start to learn the basic math of machine learning : https://www.coursera.org/learn/machine-learning (I didn’t watch all videos, only the first three weeks but I did the assignment too)

**Then start to learn about Neural Network**

- Easy start to get the big picture : https://www.youtube.com/playlist?list=PLiaHhY2iBX9hdHaRr6b7XevZtgZRa1PoU (I watched all videos even I didn’t understand at that time)
- After I knew the big picture of neural net I also read the simple history of neural net, and the “basic” of neural net, I think this gave me a few intuition. I read a book “Fundamentals of Neural Network” by Fausset, only the first two chapters
- But, I think you can get the “alternative version” what I got from Fausset’s book in the first chapter from http://neuralnetworksanddeeplearning.com/ (try to read the first three chapters)
- You can also follow the first modul from http://cs231n.github.io/ and some chapters from http://deeplearning.stanford.edu/tutorial/
- Other tutorial from stanford gave me introduction to neural network for deep learning http://cs.stanford.edu/~quocle/tutorial1.pdf

**Let’s do some math, Backpropagation?**

I suggest you to fully-understand this algorithm in order to give some intuitions to help you in model tuning or error handling when you start implement the neural network

- Easy start : I watched again the neural network demistifyed, and slowly got the intuition how backpropagation work, I followed and tried the math step-by-step
- Watch the 4th week on Coursera Introduction to Machine Learning
- Try the example of how backpropagation works : https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
- Read carefully the second chapter from http://neuralnetworksanddeeplearning.com/
- And if you start thinking they have different formula / calculation / interpretation, start to prove that you are wrong. I started to compare all the calculations to understand more

**Python for machine learning?**

- Learn the basic of python. I forgot how I learn my first python syntax, I remember I use some interactive platforms like https://www.codecademy.com/learn/python then I practice and read some python code in http://codeforces.com‘s problemset
- Get some knowledge about Numpy https://docs.scipy.org/doc/numpy-dev/user/quickstart.html and because I already knew some matlab syntax I also read http://mathesaurus.sourceforge.net/matlab-numpy.html
- I started using python for machine learning by following kaggle tutorial https://www.kaggle.com/c/titanic/details/getting-started-with-python
- Then I started implement the neural network, first I followed this simple tutorial : http://iamtrask.github.io/2015/07/12/basic-python-network/ this tutorial will give you some insight about backpropagation too
- Because I want to use GPU for deep learning, I started to learn Theano from Theano’s site http://deeplearning.net/software/theano/ and this nice tutorial https://bigaidream.gitbooks.io/subsets_ml_cookbook/content/dl/theano/theano_tutorial.html

**So what is Deep Learning?**

- Easy start : read all chapter from http://neuralnetworksanddeeplearning.com/
- The second tutorial from stanford : http://cs.stanford.edu/~quocle/tutorial2.pdf
- Amazing book from Ian goodfellow and friends : http://www.deeplearningbook.org/
- start to understand the meaning of “deep learning”, you can try by read a lot of articles on the internet

**What is Convolutional Neural Network (I use CNN in my undergraduate thesis)?**

- The second module from http://cs231n.github.io/ give nice explanation
- And also try http://ufldl.stanford.edu/tutorial/supervised/ConvolutionalNeuralNetwork/
- And the great video that gives me a lot of new knowledge about CNN : https://www.youtube.com/watch?v=cDdpwAIsuD8

**Backpropagation in CNN?**

I don’t know it is necessary or not, but I learned it. I think if you understand the basic of backpropagation it’s enough. But if you are curious how backpropgation through max pooling or convolution layer, it does not hurt to learn it.

- Okay, this part is a lot of math, you can start from read carefully http://ufldl.stanford.edu/tutorial/supervised/ConvolutionalNeuralNetwork/
- Then try some cute math from http://andrew.gibiansky.com/blog/machine-learning/convolutional-neural-networks/ and http://timdettmers.com/2015/03/26/convolution-deep-learning/ and from http://cogprints.org/5869/1/cnn_tutorial.pdf

And…great, It’s done, That’s all my references that I used when I did my undergraduate thesis, I hope it will be useful for someone :/ or at least for me in the future 🙂

Halo gan, tulisannya menarik, kebetulan saya sedang ingin menyusun skripsi dengan topik CNN. Saya mau tanya nih, program akhir skripsi nya itu pake GUI ata CLI ya dgn python nya? thanks.

Kalau dulu saya pakai “semi GUI” (?) jadi bikin tampilan sederhana menggunakan Jupyter Notebook tapi ya masih kelihatan codingannya

oo begitu ya gan, boleh minta kontak social media lain gan? biar ada temen diskusi tentang machine learning wkwk. Thanks gan.

Bisa via email aja: rian.adam.r@gmail.com 🙂