FocalLoss解析

1 FocalLoss

a. 关于Focal loss具体的解析可以参考https://zhuanlan.zhihu.com/p/49981234
对于二分类FocalLoss ，代码参考ptorch官方https://pytorch.org/vision/stable/generated/torchvision.ops.sigmoid_focal_loss.html?highlight=focal#torchvision.ops.sigmoid_focal_loss，这里主要从数值解析上去验证：

from torchvision.ops import sigmoid_focal_lossinput = torch.tensor([0.1,0.2])
target = torch.tensor([0,1])
weight = 0.25
gamma = 2
loss = sigmoid_focal_loss(input.float(), target.float(), weight=0.25, gamma=2, reduction='none')
print(loss)
'''
loss值 tensor([0.1539, 0.0303])
对于第一个是负样本，计算过程
pt = 1-torch.sigmoid(input[0])
loss_1 = -(1-weight)*(1-pt)**gamma*torch.log(pt) #值为0.1539
对于第二个是正样本，计算过程
pt = torch.sigmoid(input[1])
loss_2 = -weight*(1-pt)**gamma*torch.log(pt) #值为0.0303
'''

b. 对于多分类focal loss (multi-class focal loss)，暂时还未找到靠谱代码，基本就是说weight对所有类别都是一致的，后续在补充
b1. 这里加一个对于multi-class 的二分类 focal loss，就是将多分类转化成二分类，然后计算focal loss（retina net计算方式）

'''
假设输入 经过sigmoid之后，一共4类(包括背景类，最后一类是背景类)，二个框分类,所以输入大小是2*3，target是[3,2]，进过onehot之后[[0,0,0],[0,0,1]]
prob = torch.tensor([[0.0247,0.0248,0.0249],[0.0247,0.0248,0.0249],[0.0247,0.0248,0.0249]])
targets = torch.tensor([[0,0,0],[0,0,1]])
gamma = 2
alpha = 0.25
ce_loss = F.binary_cross_entropy(inputs, targets, reduction="none")
p_t = prob * targets + (1 - prob) * (1 - targets)
loss = ce_loss * ((1 - p_t) ** gamma)if alpha >= 0:alpha_t = alpha * targets + (1 - alpha) * (1 - targets)loss = alpha_t * loss对于第一个框第一类（算是负样本）的计算过程就是
pt = 1-prob[0][0]
loss_0 = (1-alpha)*(1-pt)**gamma*(-torch.log(pt)) # 值是1.1416e-5
整体计算过程参照上面的a计算过程

c. 基于分割的focal loss，代码参考https://docs.monai.io/en/stable/_modules/monai/losses/focal_loss.html#FocalLoss，这里计算是按所有类别weight都是一样，这里主要从数值解析上去验证：

import torch
from monai.losses import FocalLossinput = torch.tensor([[[[0.1,0.2],[0.3,0.4]],       [[0.5,0.6],[0.7,0.8]],[[0.9,0.1],[0.2,0.4]],]])target = torch.tensor([[[[1,0],[0,1]]]])
weight = 0.25
gamma = 2
pt = torch.exp(input[0,0,0,0])/(1+torch.exp(input[0,0,0,0]))
pt1 = torch.exp(input[0,0,0,1])/(1+torch.exp(input[0,0,0,1]))
self = FocalLoss(reduction='none', gamma=gamma, weight=weight, to_onehot_y=True) #对于这个weight可以设置成个list长度和类别长度一致，表示每一类权重大小，参考源码解释
loss = self(input, target)
'''
对应loss值是tensor([[[[0.0513, 0.0303],[0.0251, 0.0818]],[[0.0169, 0.1081],[0.1231, 0.0089]],[[0.1568, 0.0513],[0.0603, 0.0818]]]])
这里是3个类别标签，大小是2*2
计算第一个类别第一个位置的loss，此为负样本
pt = 1 - torch.exp(input[0,0,0,0])/(1+torch.exp(input[0,0,0,0]))
loss1 = -weight*(1-pt)**gamma*torch.log(pt) # 值是0.0513
计算第一个类别第二个位置的loss，此为正样本
pt =  torch.exp(input[0,0,0,1])/(1+torch.exp(input[0,0,0,1]))
loss2 = -weight*(1-pt)**gamma*torch.log(pt) # 值是0.0303
'''

2 Dice Loss

关于dice loss具体的解析可以参考https://zhuanlan.zhihu.com/p/269592183，具体代码解析参考https://docs.monai.io/en/stable/_modules/monai/losses/dice.html#DiceLoss.forward

from monai.losses.dice import *  # NOQA
import torch
from monai.losses.dice import DiceLossinput = torch.tensor([[[[0.1,0.2],[0.3,0.4]],       [[0.5,0.6],[0.7,0.8]],[[0.9,0.1],[0.2,0.4]],]]) # input的shape 1*3*2*2（对应batch*num_class*h*w）
target_idx = torch.tensor([[[1,0],[0,1]]]) #label的shape 1*1*2*2
target = one_hot(target_idx[:, None, ...], num_classes=C)  #这里是转化成one-hot形式
'''
target 的值
target = torch.tensor([[[0,1],[1,0]],[[1,0],[0,1]],[[0,0],[0,0]]])
'''self = DiceLoss(reduction='none')
loss = self(input, target)'''
对应的loss 结果
loss = tensor([[[[0.6667]],[[0.4348]],[[1.0000]]]])
如何计算
首先是有3个类别的输入，对于每个类别loss计算
整体公式就是 loss = 1-2*tp/(预测的概率和+标签的和)    （tp是指label为真对应的概率值）
loss_1 = 1-2*(0.2+0.3)/(0.1+0.2+0.3+0.4+2) = 0.6667
loss_2 = 1-2*(0.5+0.8)/(0.5+0.6+0.7+0.8+2) = 0.4348
loss_3 = 1-2*0/(0.9+0.1+0.2+0.4+0)