轉(zhuǎn)載自：
【面向代碼】學(xué)習(xí) Deep Learning（三）Convolution Neural Network(CNN) - DarkScope從這里開始 - 博客頻道 - CSDN.NET
最近一直在看Deep Learning，各類博客、論文看得不少
但是說實話，這樣做有些疏于實現(xiàn)，一來呢自己的電腦也不是很好，二來呢我目前也沒能力自己去寫一個toolbox
只是跟著Andrew Ng的UFLDL tutorial?寫了些已有框架的代碼(這部分的代碼見github)
后來發(fā)現(xiàn)了一個matlab的Deep Learning的toolbox ，發(fā)現(xiàn)其代碼很簡單，感覺比較適合用來學(xué)習(xí)算法
再一個就是matlab的實現(xiàn)可以省略掉很多數(shù)據(jù)結(jié)構(gòu)的代碼，使算法思路非常清晰
所以我想在解讀這個toolbox的代碼的同時來鞏固自己學(xué)到的，同時也為下一步的實踐打好基礎(chǔ)
(本文只是從代碼的角度解讀算法，具體的算法理論步驟還是需要去看paper的
我會在文中給出一些相關(guān)的paper的名字，本文旨在梳理一下算法過程，不會深究算法原理和公式)
==========================================================================================
使用的代碼：DeepLearnToolbox? ，下載地址：點(diǎn)擊打開，感謝該toolbox的作者
==========================================================================================
今天是CNN的內(nèi)容啦，CNN講起來有些糾結(jié)，你可以事先看看convolution和pooling(subsampling)，還有這篇：tornadomeet的博文
下面是那張經(jīng)典的圖：
======================================================================================================
打開\tests\test_example_CNN.m一觀
[cpp] ?view plaincopy
cnn.layers?=?{?? ????struct ( 'type' ,? 'i' )?%input?layer?? ????struct ( 'type' ,? 'c' ,? 'outputmaps' ,?6,? 'kernelsize' ,?5)?%convolution?layer?? ????struct ( 'type' ,? 's' ,? 'scale' ,?2)?%sub?sampling?layer?? ????struct ( 'type' ,? 'c' ,? 'outputmaps' ,?12,? 'kernelsize' ,?5)?%convolution?layer?? ????struct ( 'type' ,? 's' ,? 'scale' ,?2)?%subsampling?layer?? };?? cnn?=?cnnsetup(cnn,?train_x,?train_y);?????????? opts.alpha?=?1;?? opts.batchsize?=?50;?? opts.numepochs?=?1;?? cnn?=?cnntrain(cnn,?train_x,?train_y,?opts);????
似乎這次要復(fù)雜了一些啊，首先是layer，有三種，i是input，c是convolution，s是subsampling
'c'的outputmaps是convolution之后有多少張圖，比如上(最上那張經(jīng)典的))第一層convolution之后就有六個特征圖
'c'的kernelsize 其實就是用來convolution的patch是多大
's'的scale就是pooling的size為scale*scale的區(qū)域
接下來似乎就是常規(guī)思路了，cnnsetup()和cnntrain()啦,我們來看代碼
\CNN\cnnsetup.m
主要是一些參數(shù)的作用的解釋，詳細(xì)的參看代碼里的注釋啊
[cpp] ?view plaincopy
function?net?=?cnnsetup(net,?x,?y)?? ????inputmaps?=?1;?? ????mapsize?=?size(squeeze(x(:,?:,?1)));?? ?????? ?????? ????for ?l?=?1?:?numel(net.layers)??? ?? ????????if ?strcmp(net.layers{l}.type,? 's' )?? ????????????mapsize?=?mapsize?/?net.layers{l}.scale;?? ?????????????? ?????????????? ????????????assert(all(floor(mapsize)==mapsize),?['Layer?' ?num2str(l)? '?size?must?be?integer.?Actual:?' ?num2str(mapsize)]);?? ????????????for ?j?=?1?:?inputmaps? ?? ????????????????net.layers{l}.b{j}?=?0;?? ????????????end?? ????????end?? ????????if ?strcmp(net.layers{l}.type,? 'c' )?? ????????????mapsize?=?mapsize?-?net.layers{l}.kernelsize?+?1;?? ?????????????? ????????????fan_out?=?net.layers{l}.outputmaps?*?net.layers{l}.kernelsize?^?2;?? ?????????????? ????????????for ?j?=?1?:?net.layers{l}.outputmaps?? ?? ????????????????fan_in?=?inputmaps?*?net.layers{l}.kernelsize?^?2;?? ?????????????????? ????????????????for ?i?=?1?:?inputmaps?? ?? ????????????????????net.layers{l}.k{i}{j}?=?(rand(net.layers{l}.kernelsize)?-?0.5)?*?2?*?sqrt(6?/?(fan_in?+?fan_out));?? ????????????????end?? ????????????????net.layers{l}.b{j}?=?0;?? ????????????end?? ????????????inputmaps?=?net.layers{l}.outputmaps;?? ????????end?? ????end?? ?????? ?????? ?????? ?????? ????fvnum?=?prod(mapsize)?*?inputmaps;?? ????onum?=?size(y,?1);?? ?????? ????net.ffb?=?zeros(onum,?1);?? ????net.ffW?=?(rand(onum,?fvnum)?-?0.5)?*?2?*?sqrt(6?/?(onum?+?fvnum));?? end?? \CNN\cnntrain.m
?cnntrain就和nntrain是一個節(jié)奏了：
[cpp] ?view plaincopy
net?=?cnnff(net,?batch_x);?? net?=?cnnbp(net,?batch_y);?? net?=?cnnapplygrads(net,?opts);??
cnntrain是用back propagation來計算gradient的，我們一次來看這三個函數(shù)：
cnnff.m
這部分計算還比較簡單，可以說是有跡可循，大家最好看看tornadomeet的博文的步驟，說得比較清楚
[cpp] ?view plaincopy
function?net?=?cnnff(net,?x)?? ????n?=?numel(net.layers);?? ????net.layers{1}.a{1}?=?x;?? ????inputmaps?=?1;?? ?? ????for ?l?=?2?:?n??? ?? ????????if ?strcmp(net.layers{l}.type,? 'c' )?? ?????????????? ????????????for ?j?=?1?:?net.layers{l}.outputmaps??? ?? ?????????????????? ????????????????z?=?zeros(size(net.layers{l?-?1}.a{1})?-?[net.layers{l}.kernelsize?-?1?net.layers{l}.kernelsize?-?1?0]);?? ????????????????for ?i?=?1?:?inputmaps??? ?? ?????????????????????? ?????????????????????? ????????????????????z?=?z?+?convn(net.layers{l?-?1}.a{i},?net.layers{l}.k{i}{j},?'valid' );?? ????????????????end?? ?????????????????? ?????????????????? ????????????????net.layers{l}.a{j}?=?sigm(z?+?net.layers{l}.b{j});?? ????????????end?? ?????????????? ????????????inputmaps?=?net.layers{l}.outputmaps;?? ????????elseif?strcmp(net.layers{l}.type,?'s' )?? ?????????????? ????????????for ?j?=?1?:?inputmaps?? ?????????????????? ?????????????????? ????????????????z?=?convn(net.layers{l?-?1}.a{j},?ones(net.layers{l}.scale)?/?(net.layers{l}.scale?^?2),?'valid' );??? ?? ????????????????net.layers{l}.a{j}?=?z(1?:?net.layers{l}.scale?:?end,?1?:?net.layers{l}.scale?:?end,?:);?? ????????????end?? ????????end?? ????end?? ?????? ?????? ????net.fv?=?[];?? ????for ?j?=?1?:?numel(net.layers{n}.a)?? ????????sa?=?size(net.layers{n}.a{j});?? ????????net.fv?=?[net.fv;?reshape(net.layers{n}.a{j},?sa(1)?*?sa(2),?sa(3))];?? ????end?? ?????? ????net.o?=?sigm(net.ffW?*?net.fv?+?repmat(net.ffb,?1,?size(net.fv,?2)));?? ?? end??
cnnbp.m
這個就哭了，代碼有些糾結(jié)，不得已又找資料看啊，《Notes on Convolutional Neural Networks》要好一些
只是這個toolbox的代碼和《Notes on Convolutional Neural Networks》里有些不一樣的是這個toolbox在subsampling(也就是pooling層)沒有加sigmoid激活函數(shù)，只是單純地pooling了一下，所以這地方還需仔細(xì)辨別，這個toolbox里的subsampling是不用計算gradient的，而在Notes里是計算了的
還有這個toolbox沒有Combinations of Feature Maps，也就是tornadomeet的博文里這張表格：
具體就去看看上面這篇論文吧
然后就看代碼吧：
[cpp] ?view plaincopy
function?net?=?cnnbp(net,?y)?? ????n?=?numel(net.layers);?? ?????? ????net.e?=?net.o?-?y;?? ?????? ????net.L?=?1/2*?sum(net.e(:)?.^?2)?/?size(net.e,?2);?? ?????? ?????? ?????? ????net.od?=?net.e?.*?(net.o?.*?(1?-?net.o));????? ????net.fvd?=?(net.ffW'?*?net.od);???????????????? ????if ?strcmp(net.layers{n}.type,? 'c' )????????? ?? ????????net.fvd?=?net.fvd?.*?(net.fv?.*?(1?-?net.fv));?? ????end?? ?????? ?? ?????? ????sa?=?size(net.layers{n}.a{1});?? ????fvnum?=?sa(1)?*?sa(2);?? ????for ?j?=?1?:?numel(net.layers{n}.a)?? ????????net.layers{n}.d{j}?=?reshape(net.fvd(((j?-?1)?*?fvnum?+?1)?:?j?*?fvnum,?:),?sa(1),?sa(2),?sa(3));?? ????end?? ?????? ?????? ?????? ?????? ?????? ????for ?l?=?(n?-?1)?:?-1?:?1?? ????????if ?strcmp(net.layers{l}.type,? 'c' )?? ????????????for ?j?=?1?:?numel(net.layers{l}.a)?? ????????????????net.layers{l}.d{j}?=?net.layers{l}.a{j}?.*?(1?-?net.layers{l}.a{j})?.*?(expand(net.layers{l?+?1}.d{j},?[net.layers{l?+?1}.scale?net.layers{l?+?1}.scale?1])?/?net.layers{l?+?1}.scale?^?2);?? ????????????end?? ????????elseif?strcmp(net.layers{l}.type,?'s' )?? ????????????for ?i?=?1?:?numel(net.layers{l}.a)?? ????????????????z?=?zeros(size(net.layers{l}.a{1}));?? ????????????????for ?j?=?1?:?numel(net.layers{l?+?1}.a)?? ?????????????????????z?=?z?+?convn(net.layers{l?+?1}.d{j},?rot180(net.layers{l?+?1}.k{i}{j}),?'full' );?? ????????????????end?? ????????????????net.layers{l}.d{i}?=?z;?? ????????????end?? ????????end?? ????end?? ?????? ?????? ????for ?l?=?2?:?n?? ????????if ?strcmp(net.layers{l}.type,? 'c' )?? ????????????for ?j?=?1?:?numel(net.layers{l}.a)?? ????????????????for ?i?=?1?:?numel(net.layers{l?-?1}.a)?? ????????????????????net.layers{l}.dk{i}{j}?=?convn(flipall(net.layers{l?-?1}.a{i}),?net.layers{l}.d{j},?'valid' )?/?size(net.layers{l}.d{j},?3);?? ????????????????end?? ????????????????net.layers{l}.db{j}?=?sum(net.layers{l}.d{j}(:))?/?size(net.layers{l}.d{j},?3);?? ????????????end?? ????????end?? ????end?? ?????? ????net.dffW?=?net.od?*?(net.fv)'?/?size(net.od,?2);?? ????net.dffb?=?mean(net.od,?2);?? ?? ????function?X?=?rot180(X)?? ????????X?=?flipdim(flipdim(X,?1),?2);?? ????end?? end??
cnnapplygrads.m
? 這部分就輕松了，已經(jīng)有g(shù)rads了，依次進(jìn)行梯度更新就好了
[cpp] ?view plaincopy
function?net?=?cnnapplygrads(net,?opts)?? ????for ?l?=?2?:?numel(net.layers)?? ????????if ?strcmp(net.layers{l}.type,? 'c' )?? ????????????for ?j?=?1?:?numel(net.layers{l}.a)?? ????????????????for ?ii?=?1?:?numel(net.layers{l?-?1}.a)?? ????????????????????net.layers{l}.k{ii}{j}?=?net.layers{l}.k{ii}{j}?-?opts.alpha?*?net.layers{l}.dk{ii}{j};?? ????????????????end?? ????????????????net.layers{l}.b{j}?=?net.layers{l}.b{j}?-?opts.alpha?*?net.layers{l}.db{j};?? ????????????end?? ????????end?? ????end?? ?? ????net.ffW?=?net.ffW?-?opts.alpha?*?net.dffW;?? ????net.ffb?=?net.ffb?-?opts.alpha?*?net.dffb;?? end??
cnntest.m
好吧，我們得知道最后結(jié)果怎么來啊
[cpp] ?view plaincopy
function?[er,?bad]?=?cnntest(net,?x,?y)?? ?????? ????net?=?cnnff(net,?x);?? ????[~,?h]?=?max(net.o);?? ????[~,?a]?=?max(y);?? ????bad?=?find(h?~=?a);?? ?? ????er?=?numel(bad)?/?size(y,?2);?? end?? ? ? ? ? ?就是這樣~~cnnff一次后net.o就是結(jié)果
總結(jié)
? ? ? ? ? ? ? ?just code !
? ? ? ? ? ? ? ?這是一個89年的模型啊~~~，最近還和RBM結(jié)合起來了，做了一個Imagenet的最好成績(是這個吧？)：
? ? ? ? ? ? ? ?Alex Krizhevsky.ImageNet Classification with Deep??Convolutional Neural Networks. Video and Slides, 2012
? ? ? ? ? ? ? 【參考】：
? ? ? ? ? ? ? ? ? ? ? ?【Deep learning：三十八(Stacked CNN簡單介紹)】
? ? ? ? ? ? ? ? ? ? ? ?【UFLDL】
? ? ? ? ? ? ? ? ? ? ? ?【Notes on Convolutional Neural Networks】
? ? ? ? ? ? ? ? ? ? ? ?【Convolutional Neural Networks (LeNet) 】 ?這是deeplearning 的theano庫的
總結(jié)
以上是生活随笔 為你收集整理的【面向代码】学习 Deep Learning Convolution Neural Network(CNN) 的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。
如果覺得生活随笔 網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔 推薦給好友。
