深度卷積神經(jīng)網(wǎng)絡(luò)（AlexNet）

LeNet: 在大的真實(shí)數(shù)據(jù)集上的表現(xiàn)并不盡如?意。
1.神經(jīng)網(wǎng)絡(luò)計(jì)算復(fù)雜。
2.還沒有?量深?研究參數(shù)初始化和?凸優(yōu)化算法等諸多領(lǐng)域。

機(jī)器學(xué)習(xí)的特征提取:手工定義的特征提取函數(shù)
神經(jīng)網(wǎng)絡(luò)的特征提取：通過學(xué)習(xí)得到數(shù)據(jù)的多級(jí)表征，并逐級(jí)表?越來越抽象的概念或模式。

神經(jīng)網(wǎng)絡(luò)發(fā)展的限制:數(shù)據(jù)、硬件

AlexNet

首次證明了學(xué)習(xí)到的特征可以超越??設(shè)計(jì)的特征，從而?舉打破計(jì)算機(jī)視覺研究的前狀。
特征：

8層變換，其中有5層卷積和2層全連接隱藏層，以及1個(gè)全連接輸出層。

將sigmoid激活函數(shù)改成了更加簡單的ReLU激活函數(shù)。

用Dropout來控制全連接層的模型復(fù)雜度。

引入數(shù)據(jù)增強(qiáng)，如翻轉(zhuǎn)、裁剪和顏色變化，從而進(jìn)一步擴(kuò)大數(shù)據(jù)集來緩解過擬合。

#本代碼中的模型過大，CPU訓(xùn)練較慢， #可在kaggle運(yùn)行g(shù)pu：https://www.kaggle.com/boyuai/boyu-d2l-modernconvolutionalnetworkimport time import torch from torch import nn, optim import torchvision import numpy as np import sys sys.path.append("/home/kesci/input/") import d2lzh1981 as d2l import os import torch.nn.functional as Fdevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')class AlexNet(nn.Module):def __init__(self):super(AlexNet, self).__init__()self.conv = nn.Sequential(nn.Conv2d(1, 96, 11, 4), # in_channels, out_channels, kernel_size, stride, paddingnn.ReLU(),nn.MaxPool2d(3, 2), # kernel_size, stride# 減小卷積窗口，使用填充為2來使得輸入與輸出的高和寬一致，且增大輸出通道數(shù)nn.Conv2d(96, 256, 5, 1, 2),nn.ReLU(),nn.MaxPool2d(3, 2),# 連續(xù)3個(gè)卷積層，且使用更小的卷積窗口。除了最后的卷積層外，進(jìn)一步增大了輸出通道數(shù)。# 前兩個(gè)卷積層后不使用池化層來減小輸入的高和寬nn.Conv2d(256, 384, 3, 1, 1),nn.ReLU(),nn.Conv2d(384, 384, 3, 1, 1),nn.ReLU(),nn.Conv2d(384, 256, 3, 1, 1),nn.ReLU(),nn.MaxPool2d(3, 2))# 這里全連接層的輸出個(gè)數(shù)比LeNet中的大數(shù)倍。使用丟棄層來緩解過擬合self.fc = nn.Sequential(nn.Linear(256*5*5, 4096),nn.ReLU(),nn.Dropout(0.5),#由于使用CPU鏡像，精簡網(wǎng)絡(luò)，若為GPU鏡像可添加該層#nn.Linear(4096, 4096),#nn.ReLU(),#nn.Dropout(0.5),# 輸出層。由于這里使用Fashion-MNIST，所以用類別數(shù)為10，而非論文中的1000nn.Linear(4096, 10),)def forward(self, img):feature = self.conv(img)output = self.fc(feature.view(img.shape[0], -1))return output net = AlexNet() print(net)

載入數(shù)據(jù)集

# 本函數(shù)已保存在d2lzh_pytorch包中方便以后使用 def load_data_fashion_mnist(batch_size, resize=None, root='/home/kesci/input/FashionMNIST2065'):"""Download the fashion mnist dataset and then load into memory."""trans = []if resize:trans.append(torchvision.transforms.Resize(size=resize))trans.append(torchvision.transforms.ToTensor())transform = torchvision.transforms.Compose(trans)mnist_train = torchvision.datasets.FashionMNIST(root=root, train=True, download=True, transform=transform)mnist_test = torchvision.datasets.FashionMNIST(root=root, train=False, download=True, transform=transform)train_iter = torch.utils.data.DataLoader(mnist_train, batch_size=batch_size, shuffle=True, num_workers=2)test_iter = torch.utils.data.DataLoader(mnist_test, batch_size=batch_size, shuffle=False, num_workers=2)return train_iter, test_iter#batchsize=128 batch_size = 16 # 如出現(xiàn)“out of memory”的報(bào)錯(cuò)信息，可減小batch_size或resize train_iter, test_iter = load_data_fashion_mnist(batch_size,224) for X, Y in train_iter:print('X =', X.shape,'\nY =', Y.type(torch.int32))break

訓(xùn)練

lr, num_epochs = 0.001, 3 optimizer = torch.optim.Adam(net.parameters(), lr=lr) d2l.train_ch5(net, train_iter, test_iter, batch_size, optimizer, device, num_epochs)

使用重復(fù)元素的網(wǎng)絡(luò)（VGG）

VGG：通過重復(fù)使?簡單的基礎(chǔ)塊來構(gòu)建深度模型。
Block:數(shù)個(gè)相同的填充為1、窗口形狀為$3\times 3$的卷積層,接上一個(gè)步幅為2、窗口形狀為$2\times 2$的最大池化層。
卷積層保持輸入的高和寬不變，而池化層則對(duì)其減半。

VGG11的簡單實(shí)現(xiàn)

def vgg_block(num_convs, in_channels, out_channels): #卷積層個(gè)數(shù)，輸入通道數(shù)，輸出通道數(shù)blk = []for i in range(num_convs):if i == 0:blk.append(nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1))else:blk.append(nn.Conv2d(out_channels, out_channels, kernel_size=3, padding=1))blk.append(nn.ReLU())blk.append(nn.MaxPool2d(kernel_size=2, stride=2)) # 這里會(huì)使寬高減半return nn.Sequential(*blk) conv_arch = ((1, 1, 64), (1, 64, 128), (2, 128, 256), (2, 256, 512), (2, 512, 512)) # 經(jīng)過5個(gè)vgg_block, 寬高會(huì)減半5次, 變成 224/32 = 7 fc_features = 512 * 7 * 7 # c * w * h fc_hidden_units = 4096 # 任意 def vgg(conv_arch, fc_features, fc_hidden_units=4096):net = nn.Sequential()# 卷積層部分for i, (num_convs, in_channels, out_channels) in enumerate(conv_arch):# 每經(jīng)過一個(gè)vgg_block都會(huì)使寬高減半net.add_module("vgg_block_" + str(i+1), vgg_block(num_convs, in_channels, out_channels))# 全連接層部分net.add_module("fc", nn.Sequential(d2l.FlattenLayer(),nn.Linear(fc_features, fc_hidden_units),nn.ReLU(),nn.Dropout(0.5),nn.Linear(fc_hidden_units, fc_hidden_units),nn.ReLU(),nn.Dropout(0.5),nn.Linear(fc_hidden_units, 10)))return net net = vgg(conv_arch, fc_features, fc_hidden_units) X = torch.rand(1, 1, 224, 224)# named_children獲取一級(jí)子模塊及其名字(named_modules會(huì)返回所有子模塊,包括子模塊的子模塊) for name, blk in net.named_children(): X = blk(X)print(name, 'output shape: ', X.shape) ratio = 8 small_conv_arch = [(1, 1, 64//ratio), (1, 64//ratio, 128//ratio), (2, 128//ratio, 256//ratio), (2, 256//ratio, 512//ratio), (2, 512//ratio, 512//ratio)] net = vgg(small_conv_arch, fc_features // ratio, fc_hidden_units // ratio) print(net) batchsize=16 #batch_size = 64 # 如出現(xiàn)“out of memory”的報(bào)錯(cuò)信息，可減小batch_size或resize # train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size, resize=224)lr, num_epochs = 0.001, 5 optimizer = torch.optim.Adam(net.parameters(), lr=lr) d2l.train_ch5(net, train_iter, test_iter, batch_size, optimizer, device, num_epochs)

?絡(luò)中的?絡(luò)（NiN）

LeNet、AlexNet和VGG：先以由卷積層構(gòu)成的模塊充分抽取 空間特征，再以由全連接層構(gòu)成的模塊來輸出分類結(jié)果。
NiN：串聯(lián)多個(gè)由卷積層和“全連接”層構(gòu)成的小?絡(luò)來構(gòu)建?個(gè)深層?絡(luò)。
?了輸出通道數(shù)等于標(biāo)簽類別數(shù)的NiN塊，然后使?全局平均池化層對(duì)每個(gè)通道中所有元素求平均并直接?于分類。

1×1卷積核作用
1.放縮通道數(shù)：通過控制卷積核的數(shù)量達(dá)到通道數(shù)的放縮。
2.增加非線性。1×1卷積核的卷積過程相當(dāng)于全連接層的計(jì)算過程，并且還加入了非線性激活函數(shù)，從而可以增加網(wǎng)絡(luò)的非線性。
3.計(jì)算參數(shù)少

def nin_block(in_channels, out_channels, kernel_size, stride, padding):blk = nn.Sequential(nn.Conv2d(in_channels, out_channels, kernel_size, stride, padding),nn.ReLU(),nn.Conv2d(out_channels, out_channels, kernel_size=1),nn.ReLU(),nn.Conv2d(out_channels, out_channels, kernel_size=1),nn.ReLU())return blk # 已保存在d2lzh_pytorch class GlobalAvgPool2d(nn.Module):# 全局平均池化層可通過將池化窗口形狀設(shè)置成輸入的高和寬實(shí)現(xiàn)def __init__(self):super(GlobalAvgPool2d, self).__init__()def forward(self, x):return F.avg_pool2d(x, kernel_size=x.size()[2:])net = nn.Sequential(nin_block(1, 96, kernel_size=11, stride=4, padding=0),nn.MaxPool2d(kernel_size=3, stride=2),nin_block(96, 256, kernel_size=5, stride=1, padding=2),nn.MaxPool2d(kernel_size=3, stride=2),nin_block(256, 384, kernel_size=3, stride=1, padding=1),nn.MaxPool2d(kernel_size=3, stride=2), nn.Dropout(0.5),# 標(biāo)簽類別數(shù)是10nin_block(384, 10, kernel_size=3, stride=1, padding=1),GlobalAvgPool2d(), # 將四維的輸出轉(zhuǎn)成二維的輸出，其形狀為(批量大小, 10)d2l.FlattenLayer()) X = torch.rand(1, 1, 224, 224) for name, blk in net.named_children(): X = blk(X)print(name, 'output shape: ', X.shape) batch_size = 128 # 如出現(xiàn)“out of memory”的報(bào)錯(cuò)信息，可減小batch_size或resize #train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size, resize=224)lr, num_epochs = 0.002, 5 optimizer = torch.optim.Adam(net.parameters(), lr=lr) d2l.train_ch5(net, train_iter, test_iter, batch_size, optimizer, device, num_epochs)

NiN重復(fù)使?由卷積層和代替全連接層的1×1卷積層構(gòu)成的NiN塊來構(gòu)建深層?絡(luò)。
NiN去除了容易造成過擬合的全連接輸出層，而是將其替換成輸出通道數(shù)等于標(biāo)簽類別數(shù) 的NiN塊和全局平均池化層。
NiN的以上設(shè)計(jì)思想影響了后??系列卷積神經(jīng)?絡(luò)的設(shè)計(jì)。

GoogLeNet

由Inception基礎(chǔ)塊組成。

Inception塊相當(dāng)于?個(gè)有4條線路的??絡(luò)。它通過不同窗口形狀的卷積層和最?池化層來并?抽取信息，并使?1×1卷積層減少通道數(shù)從而降低模型復(fù)雜度。

可以?定義的超參數(shù)是每個(gè)層的輸出通道數(shù)，我們以此來控制模型復(fù)雜度。

class Inception(nn.Module):# c1 - c4為每條線路里的層的輸出通道數(shù)def __init__(self, in_c, c1, c2, c3, c4):super(Inception, self).__init__()# 線路1，單1 x 1卷積層self.p1_1 = nn.Conv2d(in_c, c1, kernel_size=1)# 線路2，1 x 1卷積層后接3 x 3卷積層self.p2_1 = nn.Conv2d(in_c, c2[0], kernel_size=1)self.p2_2 = nn.Conv2d(c2[0], c2[1], kernel_size=3, padding=1)# 線路3，1 x 1卷積層后接5 x 5卷積層self.p3_1 = nn.Conv2d(in_c, c3[0], kernel_size=1)self.p3_2 = nn.Conv2d(c3[0], c3[1], kernel_size=5, padding=2)# 線路4，3 x 3最大池化層后接1 x 1卷積層self.p4_1 = nn.MaxPool2d(kernel_size=3, stride=1, padding=1)self.p4_2 = nn.Conv2d(in_c, c4, kernel_size=1)def forward(self, x):p1 = F.relu(self.p1_1(x))p2 = F.relu(self.p2_2(F.relu(self.p2_1(x))))p3 = F.relu(self.p3_2(F.relu(self.p3_1(x))))p4 = F.relu(self.p4_2(self.p4_1(x)))return torch.cat((p1, p2, p3, p4), dim=1) # 在通道維上連結(jié)輸出

GoogLeNet模型

完整模型結(jié)構(gòu)

b1 = nn.Sequential(nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3),nn.ReLU(),nn.MaxPool2d(kernel_size=3, stride=2, padding=1))b2 = nn.Sequential(nn.Conv2d(64, 64, kernel_size=1),nn.Conv2d(64, 192, kernel_size=3, padding=1),nn.MaxPool2d(kernel_size=3, stride=2, padding=1))b3 = nn.Sequential(Inception(192, 64, (96, 128), (16, 32), 32),Inception(256, 128, (128, 192), (32, 96), 64),nn.MaxPool2d(kernel_size=3, stride=2, padding=1))b4 = nn.Sequential(Inception(480, 192, (96, 208), (16, 48), 64),Inception(512, 160, (112, 224), (24, 64), 64),Inception(512, 128, (128, 256), (24, 64), 64),Inception(512, 112, (144, 288), (32, 64), 64),Inception(528, 256, (160, 320), (32, 128), 128),nn.MaxPool2d(kernel_size=3, stride=2, padding=1))b5 = nn.Sequential(Inception(832, 256, (160, 320), (32, 128), 128),Inception(832, 384, (192, 384), (48, 128), 128),d2l.GlobalAvgPool2d())net = nn.Sequential(b1, b2, b3, b4, b5, d2l.FlattenLayer(), nn.Linear(1024, 10))net = nn.Sequential(b1, b2, b3, b4, b5, d2l.FlattenLayer(), nn.Linear(1024, 10))X = torch.rand(1, 1, 96, 96)for blk in net.children(): X = blk(X)print('output shape: ', X.shape)#batchsize=128 batch_size = 16 # 如出現(xiàn)“out of memory”的報(bào)錯(cuò)信息，可減小batch_size或resize #train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size, resize=96)lr, num_epochs = 0.001, 5 optimizer = torch.optim.Adam(net.parameters(), lr=lr) d2l.train_ch5(net, train_iter, test_iter, batch_size, optimizer, device, num_epochs)

總結(jié)

以上是生活随笔為你收集整理的《动手学深度学习》组队学习打卡Task5——卷积神经网络进阶的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。