Archive

Archive for the ‘deep learning’ Category

Learning deep learning (project 2, image classification)

March 7th, 2017

In this class project, I built a network to classify images in the CIFAR-10 dataset. This dataset is freely available.

The dataset contains 60K color images (32×32 pixel) in 10 classes, with 6K images per class.

Here are the classes in the dataset, as well as 10 random images from each:

airplane
automobile
bird
cat
deer
dog
frog
horse
ship
truck

You can imagine it’s not possible to write down all rules to classify them, so we have to write a program which can learn.

The neural network I created contains 2 hidden layers. The first one is a convolutional layer with max pooling. Then drop out 70% of the connections. The second layer is a fully connected layer with 384 neurons.

def conv_net(x, keep_prob):
    """
    Create a convolutional neural network model
    : x: Placeholder tensor that holds image data.
    : keep_prob: Placeholder tensor that hold dropout keep probability.
    : return: Tensor that represents logits
    """
    # TODO: Apply 1, 2, or 3 Convolution and Max Pool layers
    #    Play around with different number of outputs, kernel size and stride
    # Function Definition from Above:
    #    conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)
    model = conv2d_maxpool(x, conv_num_outputs=18, conv_ksize=(4,4), conv_strides=(1,1), pool_ksize=(8,8), pool_strides=(1,1))
    model = tf.nn.dropout(model, keep_prob)

    # TODO: Apply a Flatten Layer
    # Function Definition from Above:
    #   flatten(x_tensor)
    model = flatten(model)

    # TODO: Apply 1, 2, or 3 Fully Connected Layers
    #    Play around with different number of outputs
    # Function Definition from Above:
    #   fully_conn(x_tensor, num_outputs)
    model = fully_conn(model,384)

    model = tf.nn.dropout(model, keep_prob)

    # TODO: Apply an Output Layer
    #    Set this to the number of classes
    # Function Definition from Above:
    #   output(x_tensor, num_outputs)
    model = output(model,10)

    # TODO: return output
    return model

Then I trained this network using Amazon AWS g2.2xlarge instance. This instance has GPU which is much faster for deep learning (than CPU). I did a simple experiment and find GPU is at least 3 times faster than CPU:

if all layers in gpu: 14 seconds to run 4 epochs,
if conv layer in cpu, other gpu, 36 seconds to run 4 epochs

This is apparently a very crude comparison but GPU is definitely much faster than CPU (at least the ones in AWS g2.2xlarge, cost: $0.65/hour)

Eventually I got ~70% accuracy on the test data, much better than random guess (10%). The time to train the model is ~30 minutes.

You can find my entire code at:
http://www.alivelearn.net/deeplearning/dlnd_image_classification_submission2.html

Author: Xu Cui Categories: brain, deep learning Tags:

Learning deep learning on Udacity

February 9th, 2017

I am taking Udacity’s deep learning class at https://www.udacity.com/course/deep-learning-nanodegree-foundation–nd101

I have done the first project, creating a neural network with 1 hidden layer (so not deep enough :)) to predict bike demands for a bike rental company. The data are real-life data; so this project is actually has real applications. In a nutshell, we can predict how many bikes will be rented in a given day based on factors such as the weather, whether the day is a holiday, etc.

The same model can also be used in other applications such as predicting number of customers of a clothes shop, or of a website.

My homework for this project can be found here:
http://www.alivelearn.net/deeplearning/dlnd-your-first-neural-network.html

Author: Xu Cui Categories: deep learning Tags:

Deep learning

January 20th, 2017

In the past months, I am shocked by the progress of artificial intelligence (mostly implemented by deep learning). In March 2016, AlphaGo won Lee Sedol (李世石) in Weiqi (go). I had mixed feelings, excited, sad, and some fear. Around new year of 2017, AlphaGo won 60 games in a row against numerous top professional Weiqi players in China, Korea and Japan, including #1 Ke Jie. There is no doubt AlphaGo is at least a level better than top human player. It’s interesting to see that the way how people call AlphaGo has changed from “dog” to “Teacher Ah”, reflecting the change of our attitude toward artificial intelligence.

Game is not the only area where AI shocked me. Below are some area AI / deep learning has done extremely well:

  1. convert text to handwriting: Try yourself at http://www.cs.toronto.edu/~graves/handwriting.html Maybe in the future you can use AI to write your greeting cards.
  2. Apply artistic style to drawings. Check out https://www.youtube.com/watch?v=Uxax5EKg0zA and https://www.youtube.com/watch?v=jMZqxfTls-0
  3. Fluid simulation
  4. Generate a text description of an image
  5. Real time facial expression transfer https://www.youtube.com/watch?v=mkI6qfpEJmI
  6. Language translation
  7. Handwriting recognition (try it here: http://cs.stanford.edu/people/karpathy/convnetjs/demo/mnist.html) This is not new progress but still worth mentioning
  8. Medical diagnosis
  9. And many more. I will update this list constantly
In the field of biology and medicine, deep learning also progresses rapidly. Below is the number of publications using keyword “deep learning” in PubMed.
deep learning publications in PubMed
deep learning publications in PubMed
“Deep Learning” is also a keyword in my Stork. I got new papers almost every day.
Some resources to learn more about deep learning and keep updated:
  1. Track “Deep Learning” publications using Stork
  2. Subscribe youtube channel Two Minute Papers (https://www.youtube.com/user/keeroyz). It contains many excellent short videos on the application of deep learning
  3. Play it here: http://playground.tensorflow.org/
  4. A few examples here: http://cs.stanford.edu/people/karpathy/convnetjs/
  5. I am going to take Udacity’s deep learning class at https://www.udacity.com/course/deep-learning-nanodegree-foundation–nd101
Author: Xu Cui Categories: deep learning, programming Tags: