Pytorch實現全連接層的操作
全連接神經網絡(FC)
全連接神經網絡是一種最基本的神經網絡結構,英文為Full Connection,所以一般簡稱FC。
FC的準則很簡單:神經網絡中除輸入層之外的每個節點都和上一層的所有節點有連接。
以上一次的MNIST為例
import torch import torch.utils.data from torch import optim from torchvision import datasets from torchvision.transforms import transforms import torch.nn.functional as F batch_size = 200 learning_rate = 0.001 epochs = 20 train_loader = torch.utils.data.DataLoader( datasets.MNIST('mnistdata', train=True, download=False, transform=transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ])), batch_size=batch_size, shuffle=True) test_loader = torch.utils.data.DataLoader( datasets.MNIST('mnistdata', train=False, download=False, transform=transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ])), batch_size=batch_size, shuffle=True) w1, b1 = torch.randn(200, 784, requires_grad=True), torch.zeros(200, requires_grad=True) w2, b2 = torch.randn(200, 200, requires_grad=True), torch.zeros(200, requires_grad=True) w3, b3 = torch.randn(10, 200, requires_grad=True), torch.zeros(10, requires_grad=True) torch.nn.init.kaiming_normal_(w1) torch.nn.init.kaiming_normal_(w2) torch.nn.init.kaiming_normal_(w3) def forward(x): x = [email protected]() + b1 x = F.relu(x) x = [email protected]() + b2 x = F.relu(x) x = [email protected]() + b3 x = F.relu(x) return x optimizer = optim.Adam([w1, b1, w2, b2, w3, b3], lr=learning_rate) criteon = torch.nn.CrossEntropyLoss() for epoch in range(epochs): for batch_idx, (data, target) in enumerate(train_loader): data = data.view(-1, 28*28) logits = forward(data) loss = criteon(logits, target) optimizer.zero_grad() loss.backward() optimizer.step() if batch_idx % 100 == 0: print('Train Epoch : {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format( epoch, batch_idx*len(data), len(train_loader.dataset), 100.*batch_idx/len(train_loader), loss.item() )) test_loss = 0 correct = 0 for data, target in test_loader: data = data.view(-1, 28*28) logits = forward(data) test_loss += criteon(logits, target).item() pred = logits.data.max(1)[1] correct += pred.eq(target.data).sum() test_loss /= len(test_loader.dataset) print('\nTest set : Averge loss: {:.4f}, Accurancy: {}/{}({:.3f}%)'.format( test_loss, correct, len(test_loader.dataset), 100.*correct/len(test_loader.dataset) ))
我們將每個w和b都進行瞭定義,並且自己寫瞭一個forward函數。如果我們采用瞭全連接層,那麼整個代碼也會更加簡介明瞭。
首先,我們定義自己的網絡結構的類:
class MLP(nn.Module): def __init__(self): super(MLP, self).__init__() self.model = nn.Sequential( nn.Linear(784, 200), nn.LeakyReLU(inplace=True), nn.Linear(200, 200), nn.LeakyReLU(inplace=True), nn.Linear(200, 10), nn.LeakyReLU(inplace=True) ) def forward(self, x): x = self.model(x) return x
它繼承於nn.Moudle,並且自己定義裡整個網絡結構。
其中inplace的作用是直接復用存儲空間,減少新開辟存儲空間。
除此之外,它可以直接進行運算,不需要手動定義參數和寫出運算語句,更加簡便。
同時我們還可以發現,它自動完成瞭初試化,不需要像之前一樣再手動寫一個初始化瞭。
區分nn.Relu和F.relu()
前者是一個類的接口,後者是一個函數式接口。
前者都是大寫的,並且調用的的時候需要先實例化才能使用,而後者是小寫的可以直接使用。
最重要的是後者的自由度更高,更適合做一些自己定義的操作。
完整代碼
import torch import torch.utils.data from torch import optim, nn from torchvision import datasets from torchvision.transforms import transforms import torch.nn.functional as F batch_size = 200 learning_rate = 0.001 epochs = 20 train_loader = torch.utils.data.DataLoader( datasets.MNIST('mnistdata', train=True, download=False, transform=transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ])), batch_size=batch_size, shuffle=True) test_loader = torch.utils.data.DataLoader( datasets.MNIST('mnistdata', train=False, download=False, transform=transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ])), batch_size=batch_size, shuffle=True) class MLP(nn.Module): def __init__(self): super(MLP, self).__init__() self.model = nn.Sequential( nn.Linear(784, 200), nn.LeakyReLU(inplace=True), nn.Linear(200, 200), nn.LeakyReLU(inplace=True), nn.Linear(200, 10), nn.LeakyReLU(inplace=True) ) def forward(self, x): x = self.model(x) return x device = torch.device('cuda:0') net = MLP().to(device) optimizer = optim.Adam(net.parameters(), lr=learning_rate) criteon = nn.CrossEntropyLoss().to(device) for epoch in range(epochs): for batch_idx, (data, target) in enumerate(train_loader): data = data.view(-1, 28*28) data, target = data.to(device), target.to(device) logits = net(data) loss = criteon(logits, target) optimizer.zero_grad() loss.backward() optimizer.step() if batch_idx % 100 == 0: print('Train Epoch : {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format( epoch, batch_idx*len(data), len(train_loader.dataset), 100.*batch_idx/len(train_loader), loss.item() )) test_loss = 0 correct = 0 for data, target in test_loader: data = data.view(-1, 28*28) data, target = data.to(device), target.to(device) logits = net(data) test_loss += criteon(logits, target).item() pred = logits.data.max(1)[1] correct += pred.eq(target.data).sum() test_loss /= len(test_loader.dataset) print('\nTest set : Averge loss: {:.4f}, Accurancy: {}/{}({:.3f}%)'.format( test_loss, correct, len(test_loader.dataset), 100.*correct/len(test_loader.dataset) ))
補充:pytorch 實現一個隱層的全連接神經網絡
torch.nn 實現 模型的定義,網絡層的定義,損失函數的定義。
import torch # N is batch size; D_in is input dimension; # H is hidden dimension; D_out is output dimension. N, D_in, H, D_out = 64, 1000, 100, 10 # Create random Tensors to hold inputs and outputs x = torch.randn(N, D_in) y = torch.randn(N, D_out) # Use the nn package to define our model as a sequence of layers. nn.Sequential # is a Module which contains other Modules, and applies them in sequence to # produce its output. Each Linear Module computes output from input using a # linear function, and holds internal Tensors for its weight and bias. model = torch.nn.Sequential( torch.nn.Linear(D_in, H), torch.nn.ReLU(), torch.nn.Linear(H, D_out), ) # The nn package also contains definitions of popular loss functions; in this # case we will use Mean Squared Error (MSE) as our loss function. loss_fn = torch.nn.MSELoss(reduction='sum') learning_rate = 1e-4 for t in range(500): # Forward pass: compute predicted y by passing x to the model. Module objects # override the __call__ operator so you can call them like functions. When # doing so you pass a Tensor of input data to the Module and it produces # a Tensor of output data. y_pred = model(x) # Compute and print loss. We pass Tensors containing the predicted and true # values of y, and the loss function returns a Tensor containing the # loss. loss = loss_fn(y_pred, y) print(t, loss.item()) # Zero the gradients before running the backward pass. model.zero_grad() # Backward pass: compute gradient of the loss with respect to all the learnable # parameters of the model. Internally, the parameters of each Module are stored # in Tensors with requires_grad=True, so this call will compute gradients for # all learnable parameters in the model. loss.backward() # Update the weights using gradient descent. Each parameter is a Tensor, so # we can access its gradients like we did before. with torch.no_grad(): for param in model.parameters(): param -= learning_rate * param.grad
上面,我們使用parem= -= learning_rate* param.grad 手動更新參數。
使用torch.optim 自動優化參數。optim這個package提供瞭各種不同的模型優化方法,包括SGD+momentum, RMSProp, Adam等等。
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate) for t in range(500): y_pred = model(x) loss = loss_fn(y_pred, y) optimizer.zero_grad() loss.backward() optimizer.step()
以上為個人經驗,希望能給大傢一個參考,也希望大傢多多支持WalkonNet。如有錯誤或未考慮完全的地方,望不吝賜教。
推薦閱讀:
- 手把手教你實現PyTorch的MNIST數據集
- Python機器學習之基於Pytorch實現貓狗分類
- Pytorch中求模型準確率的兩種方法小結
- pytorch實現加載保存查看checkpoint文件
- PyTorch一小時掌握之神經網絡分類篇