FCN:
Fully Convolutional Networks for Semantic Segmentation (CVPR)
/pdf/1411.4038.pdf
PyTorch: /shanglianlm0525/SemanticSegmentation-network
第一篇将CNN引入image segmentation 的文章，提出了一种end-to-end的做semantic segmentation的方法。
特点：
a) 将之前分类网络的全连接层都换成卷积层；
b) 使用上采样操作,上采样的参数是可学习的，这里采用的方法是反卷积；
c) 将这些特征图进行上采样之后，将特征图对应元素相加；
因为经过多次卷积和pooling之后，得到的图像越来越小，分辨率越来越低，FCN为了得到信息，使用上采样（使用deconvolution）实现尺寸还原。不仅对pool5之后的特征图进行了还原，也对pool4和pool3之后的特征图进行了还原，结果表明，从这些特征图能很好的获得关于图片的语义信息，而且随着特征图越来越大，效果越来越好。
PyTorch代码:
import torchimport torch.nn as nnimport torchvisionclass FCN8s(nn.Module):def __init__(self, num_classes):super(FCN8s, self).__init__()vgg = torchvision.models.vgg16()features = list(vgg.features.children())self.padd = nn.ZeroPad2d([100,100,100,100])self.pool3 = nn.Sequential(*features[:17])self.pool4 = nn.Sequential(*features[17:24])self.pool5 = nn.Sequential(*features[24:])self.pool3_conv1x1 = nn.Conv2d(256, num_classes, kernel_size=1)self.pool4_conv1x1 = nn.Conv2d(512, num_classes, kernel_size=1)self.output5 = nn.Sequential(nn.Conv2d(512, 4096, kernel_size=7),nn.ReLU(inplace=True),nn.Dropout(),nn.Conv2d(4096, 4096, kernel_size=1),nn.ReLU(inplace=True),nn.Dropout(),nn.Conv2d(4096, num_classes, kernel_size=1),)self.up_pool3_out = nn.ConvTranspose2d(num_classes, num_classes, kernel_size=16, stride=8)self.up_pool4_out = nn.ConvTranspose2d(num_classes, num_classes, kernel_size=4, stride=2)self.up_pool5_out = nn.ConvTranspose2d(num_classes, num_classes, kernel_size=4, stride=2)def forward(self, x):_,_, w, h = x.size()x = self.padd(x)pool3 = self.pool3(x)pool4 = self.pool4(pool3)pool5 = self.pool5(pool4)output5 = self.up_pool5_out(self.output5(pool5))pool4_out = self.pool4_conv1x1(0.01 * pool4)output4 = self.up_pool4_out(pool4_out[:,:,5:(5 + output5.size()[2]) ,5:(5 + output5.size()[3])]+output5)pool3_out = self.pool3_conv1x1(0.0001 * pool3)output3 = self.up_pool3_out(pool3_out[:, :, 9:(9 + output4.size()[2]), 9:(9 + output4.size()[3])] + output4)out = self.up_pool3_out(output3)out = out[:, :, 31: (31 + h), 31: (31 + w)].contiguous()return outif __name__ == '__main__':model = FCN8s(num_classes=20)print(model)input = torch.randn(1,3,224,224)output = model(input)print(output.shape)
