simple neural net

Neural Net in 4 lines! using Scikit-Learn MLPClassifier

See the code below:

That’s right, those 4 lines code can create a Neural Net with one hidden layer! 😐

Scikit-learn just released stable version 0.18. One of the new features is MLPClassifer and you can see in the code above, it’s powerful enough to create a simple neural net program.

That code just a snippet of my Iris Classifier Program that you can see on Github. Of course, in practice, you still need to create loader, pre-process, pre-training, or other modules. But, if you see other python libraries like Keras, Lasagne, or Theano, I think this is the easiest way to create a simple neural net. I said “simple” because when you need to create more complex model that need more complex algorithm or many addons, I think it’ll become difficult to use MLPClassifier. But, for some simple project, I think I’ll choose this scikit tool 🙂

Program Explanation

Line 1: you need to load MLPClassifier

Line 2: we create an object called ‘mlp’ which is a MLPClassifier. We set hidden_layer_size to (10) which means we add one hidden layer with 10 neurons. Then we set solver as ‘sgd’ because we will use Stochastic Gradient Descent as optimizer. Then we set learning_rate_init to 0.01, this is a learning rate value (be careful, don’t confuse with alpha parameter in MLPClassifer). Then the last, we set 500 as the maximum number of training iteration.

Line 3: Train the model

Line 4: Test the model

That’s it! and my first run I got more than 90% accuration in Iris Dataset 🙂 try it!

And actually, you can create “Deeper” neural net with add more layers easily, just change hidden_layer_sizes. For example, hidden_layer_sizes=(10,10,10) will create 3 hidden layer with 10 neuron each.

source image: Flickr

Simple Tutorial Python, Boto3, and AWS S3

I’ve horrible experience to find “good” tutorial about how to use Boto3 in Amazon Web Service (AWS). Boto3, not like Boto2, has poor quality documentation. So I create this simple tutorial as reminder to myself and I hope it will help someone out there.

Boto3

The AWS SDK for Python. It simply to said, if you have a python apps and you want it to access AWS features, you need this.

Installation & Setup

Installation is very clear in python documentation and for configuration you can check in Boto3 documentation just using pip:

after install boto3 you can use awscli to make it easier setup credentials, install it using pip too:

set your configuration using command below. You can get your access key and secret key in IAM console:

Create A Simple App

Let’s create a simple app using Boto3. We will create a simple app to access stored data in AWS S3. This app will write and read a json file stored in S3.

1. Prepare Your Bucket

First, you need to create a bucket in your S3. You can create bucket by visiting your S3 service and click Create Bucket button.

Capture

or using some code below:

If you confuse what is bucket and how it works, this one have nice explanation.

2. Write JSON File

Next part is how to write a file in S3. The code below will create a json file (if it doesn’t exist, or overwrite it otherwise) named hello.json and put it in your bucket.

If you want to put it on specific path, you can change the line

to

Run it, and if you check your bucket now you will find your file in there.

3. Read JSON File

Last but not least, if you want to read your file, you can use get() function. The code below will read your hello.json file and show it on screen. File body/content will be read as string by default.

I think that’s all, a simple tutorial using boto3 to read-write file in Amazon Web Service S3. Hope it helps 🙂

A* algorithm for puzzle

A* Algorithm in Python to solve 8 puzzle problem

My team got this as A* algorithm assignment in Artificial Intelligence class few years ago taught by Mrs. Afiahayati, Ph.D. We should create an implementation of A* algorithm (read: “A” Star) to solve 8 puzzle problem. This puzzle problem is the small version of 15 sliding puzzle game.

A* Algorithm

In my opinion A* Algorithm (read more about it here) is looks like combination of Breadth First Search (BFS) and Depth First Search (DFS) algorithm (or maybe Dijkstra’s too(?)). It’s using Heuristic scoring to estimate the step from vertex to goal so make the system may running faster, and like Dijkstra’s, it’s a complete search which always finding a best solution. Read this to get better explanation.

My team agreed to use Manhattan distance to estimate distance between the current state and the goal state (h) and count the number of step as exact cost (g).

Implementation

I wrote the code using Python. And after several hours coding I found python have a “magical” function that make easy to write 🙂 My first implementation was not using any class at all and it made the code hard to read. After I saw this implementation, I realize that code is much prettier than mine, so I rewrite my code using Node class with similar class concept from that code.

See the complete source code on Github

documentation neural network

How Do I Learn Neural Network (and Deep Learning)

First, I am nobody in this topic, please don’t hesitate to give better suggestion. This is actually just a “documentation” about my experience when I work on my undergraduate thesis in 2015. Once again, please don’t hesitate to discuss and give better suggestion about this topic, or.. correct my English if I made some mistakes 🙂

Background

Okay, before I tell you my story, I’ll tell you about my background when I started to learn neural network. I wrote this section only have one intention, to make you easy measure what you need, nothing else.

  1. I have familiarity with some of mathematical symbols, like how to read sigma symbol, derivative symbol and many others (but not every symbols).
  2. I have familiarity with basic math operation like derivative function, algebra, matrix and vector operation (but only “the basic”).
  3. I’ve never coded in python, but I’ve been using C++ around 2-3 years and I used to code  Matlab style programming language in some project before (and thanks, this knowledge is very helpful)
  4. I studied the Introduction of Artificial Intelligence at the fourth semester in college, but I won’t said I did it well 🙁

After a month googling randomly, I started to make a list what should I learn, the order of the list below it’s just the best order in my opinion. In my experience I jumped many times over the topic because at that time, I didn’t know what I didn’t know 🙁

If you want to learn about Deep Learning, lets begin with: What is Machine Learning?

Then start to learn about Neural Network

Let’s do some math, Backpropagation?

I suggest you to fully-understand this algorithm in order to give some intuitions to help you in model tuning or error handling when you start implement the neural network

  • Easy start : I watched again the neural network demistifyed, and slowly got the intuition how backpropagation work, I followed and tried the math step-by-step
  • Watch the 4th week on Coursera Introduction to Machine Learning
  • Try the example of how backpropagation works : https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/
  • Read carefully the second chapter from http://neuralnetworksanddeeplearning.com/
  • And if you start thinking they have different formula / calculation / interpretation, start to prove that you are wrong. I started to compare all the calculations to understand more

Python for machine learning?

So what is Deep Learning?

What is Convolutional Neural Network (I use CNN in my undergraduate thesis)?

Backpropagation in CNN?

I don’t know it is necessary or not, but I learned it. I think if you understand the basic of backpropagation it’s enough. But if you are curious how backpropgation through max pooling or convolution layer, it does not hurt to learn it.

And…great, It’s done, That’s all my references that I used when I did my undergraduate thesis, I hope it will be useful for someone :/ or at least for me in the future 🙂