Learning deep learning (project 2, image classification)

1 min read

In this class project, I built a network to classify images in the CIFAR-10 dataset. This dataset is freely available.

The dataset contains 60K color images (32×32 pixel) in 10 classes, with 6K images per class.

Here are the classes in the dataset, as well as 10 random images from each:


You can imagine it’s not possible to write down all rules to classify them, so we have to write a program which can learn.

The neural network I created contains 2 hidden layers. The first one is a convolutional layer with max pooling. Then drop out 70% of the connections. The second layer is a fully connected layer with 384 neurons.

def conv_net(x, keep_prob):
    Create a convolutional neural network model
    : x: Placeholder tensor that holds image data.
    : keep_prob: Placeholder tensor that hold dropout keep probability.
    : return: Tensor that represents logits
    # TODO: Apply 1, 2, or 3 Convolution and Max Pool layers
    #    Play around with different number of outputs, kernel size and stride
    # Function Definition from Above:
    #    conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)
    model = conv2d_maxpool(x, conv_num_outputs=18, conv_ksize=(4,4), conv_strides=(1,1), pool_ksize=(8,8), pool_strides=(1,1))
    model = tf.nn.dropout(model, keep_prob)

    # TODO: Apply a Flatten Layer
    # Function Definition from Above:
    #   flatten(x_tensor)
    model = flatten(model)

    # TODO: Apply 1, 2, or 3 Fully Connected Layers
    #    Play around with different number of outputs
    # Function Definition from Above:
    #   fully_conn(x_tensor, num_outputs)
    model = fully_conn(model,384)

    model = tf.nn.dropout(model, keep_prob)

    # TODO: Apply an Output Layer
    #    Set this to the number of classes
    # Function Definition from Above:
    #   output(x_tensor, num_outputs)
    model = output(model,10)

    # TODO: return output
    return model

Then I trained this network using Amazon AWS g2.2xlarge instance. This instance has GPU which is much faster for deep learning (than CPU). I did a simple experiment and find GPU is at least 3 times faster than CPU:

if all layers in gpu: 14 seconds to run 4 epochs,
if conv layer in cpu, other gpu, 36 seconds to run 4 epochs

This is apparently a very crude comparison but GPU is definitely much faster than CPU (at least the ones in AWS g2.2xlarge, cost: $0.65/hour)

Eventually I got ~70% accuracy on the test data, much better than random guess (10%). The time to train the model is ~30 minutes.

You can find my entire code at:

Receive email notification via email 博客有新内容通知我

Don't want to miss new papers in your field? Check out Stork we developed:

第二十五期 fNIRS Journal Club 视频

来自华东师范大学的胡银莹博士为大家介绍了具体如何计算和验证超扫描实验中脑间同步性,并分享做fNIRS实验相关方面的应用经验。视频如下: Youtube:https://youtu.be/xI_6wKd06io Youku:上传后被youku屏蔽,原因不详。
Xu Cui
3 sec read

第二十五期 fNIRS Journal Club 通知 2021/11/27,10:00am

来自华东师范大学的胡银莹博士将为大家介绍具体如何计算和验证超扫描实验中脑间同步性,并分享做fNIRS实验相关方面的应用经验。热烈欢迎大家参与讨论。 时间: 北京时间2021年11月27日上午10:00地点: https://zoom.com房间号: 846 2034 9158密码: 064380 胡银莹博士要讲解的文章如下: How to Calculate and Validate Inter-brain Synchronization in a fNIRS Hyperscanning Study. Hu, Wang, Song,...
Xu Cui
9 sec read

第二十四期 fNIRS Journal Club 视频

北京时间2021/10/23,10:00am, 来自韩国釜山国立大学Keum-Shik Hong教授团队的杨大林同学为大家介绍了如何结合深度学习和近红外扫描技术实现早期老年痴呆症的识别,并分享深度学习技术在相关方面的应用经验 。 Youtube:https://youtu.be/3OLncoL01o8 Youku:https://v.youku.com/v_show/id_XNTgxNTMxNzkxNg==.html 相关资源: 文献鸟(追踪科学文献)文献大分析(两分钟了解一个领域)
Xu Cui
5 sec read

2 Replies to “Learning deep learning (project 2, image classification)”

  1. Helpful post. Can you explain your motivation behind using standard deviation on 0.1 while initializing the weights. My network does not learn if i keep the standard deviation to 1. Only when i saw your post and fine tuned my standard deviation to 0.1, it started training. i would like to understand how did you choose the standard deviation of 0.1 🙂

  2. Can you explain how you arrived at the values below?

    model = fully_conn(model,384)
    #model = fully_conn(model,200)
    #model = fully_conn(model,20)

Leave a Reply

Your email address will not be published. Required fields are marked *