Self-supervised Image Enhancement Network: Training with Low Light Images Only

Yu Zhang, Xiaoguang Di, Bin Zhang, and Chunhui Wang

本文用信息熵理论和Retinex 模型构建了一个自监督图像增强网络，实现了弱光图像中，亮度与反射系数的分解。其中，最为重要的约束是：反射系数的最大通道与弱光图像的最大通道是conform的，并且前者的熵在本文的模型中应该是最大的。

问题：什么是最大通道？为什么 conform？为什么熵最大？

最大通道是指对每个位置，r、g、b最大的值取出来，构成的灰度图像。
对于弱光图像，最大通道对其视觉效果的影响最大；所以，两幅图像的最大通道应该相符合；
第三点，没大明白，希望懂的读者大神指教。

Abstract

This paper proposes a self-supervised low light image enhancement method based on deep learning.

Inspired by information entropy theory and Retinex model, we proposed a maximum entropy based Retinex model. With this model, a very simple network can separate the illumination and reflectance, and the network can be trained with low light images only. We introduce a constraint that the maximum channel of the reflectance conforms to the maximum channel of the low light image and its entropy should be largest in our model to achieve self-supervised learning.

Our model is very simple and does not rely on any well-designed data set (even one low light image can complete the training). The network only needs minute-level training to achieve image enhancement. It can be proved through experiments that the proposed method has reached the state-of-the-art in terms of processing speed and effect.

Problem

As for low light image enhancement task, in the previous work, some solutions such as synthesizing low light images [7], using different exposure time images to obtain data [8], and so on, have achieved good visual effects. However, there are still two problems with those methods. One is how to ensure that the pre-trained network can be used for images collected from different devices, different scenes, and different lighting conditions rather than building new training data set. The other is how to determine whether the normal light image used for supervision is the best, there can be lots of normal light images for a low light image. Usually, the builder of the data set gets the normal light images by experience or artificial adjustment, which will cost lots of time and energy and we cannot make sure that the enhanced image can show the information contained in the low light image to the greatest extent with those normal light images.

传统基于数据集网络模型的两个问题：

传统的数据集都采用一种摄像机采集不同曝光下的图像对；采用这种数据集训练的pre-trained work不能保证对不同摄像机拍摄的、或者不同曝光时间下的、或者其他没有在数据集中出现的场景的图像也都有较好的效果；
给出的真实图像，未必是最好的图像。

Strategy

Retinex model : decompose low light image into reflectance and illumination;
Self-supervised;
Information entropy theory : histogram distribution of the maximum channel of the enhanced image should conform to the histogram distribution of the maximum channel of the low light image histogram equalization.

Method

Maximum Entropy Based Retinex model (损失函数的构建）

采用了三个损失函数，分别如下：

Retinex model : S = R ◦ I

reconstruction loss : lrcon = || S − R ◦ I ||1

reflectance loss : lR =|| max c∈R,G,B Rc − F( max c∈R,G,B Sc ) ||1 + λ||△R||1 ,

where, F() means the histogram equalization operator to input image.

这个损失函数是本文的核心，下面解释了为什么提出这样一个损失函数：

This loss function means that maximum channel of the reflectance should conform to the maximum channel of the low light image and has the maximum entropy.

There are three main reasons why we choose the maximum channel to constrain.

Firstly, for a low light image, the maximum channel has the greatest impact on its visual effect.

Secondly, if other channels are selected, there is no doubt that saturation will occur according to the prior that the maximum channel must be greater than the other two channels.

Thirdly, if we choose one of the color channel, such as R, G or B channel, it is obviously not in line with the natural image.

本文的一个重要先验：反射图像的损失。

该损失函数意味着反射系数的最大通道应符合弱光图像的最大通道，且具有最大熵。这个先验的原因在于：

对于弱光图像，最大通道对其视觉效果的影响最大；
如果选择其他通道，那么最大通道将过饱和；
如果我们选择其中一个颜色通道，如R、G或B通道，它显然是不符合自然图像的。

illumination loss : adopt structure-aware smoothness loss lI = ||△I ◦ exp (−λ3 △R)||1

最后的损失函数就是三者相加。这个公式的解很难，且费时。因此，这个混合损失函数，采用深度学习的方式求其解。

Self-supervised Network Based Solution

only need a very simple CNN structure to achieve the decomposition of the illumination and reflectance.

网络结构如下图所示。

论文速读：自监督弱光图像增强：Self-supervised Image Enhancement Network: Training with Low Light Images Only相关推荐

低光图像增强论文：Self-supervised Image Enhancement Network: Training with Low Light Images Only阅读笔记
论文地址:https://arxiv.org/abs/2002.11300 项目地址:https://github.com/hitzhangyu/Self-supervised-Image-Enhan ...
[论文速读] 图像增强系列：2020 TIP 最新去雾算法（摘要、网络结构图及论文链接）
[论文速读] 图像增强系列:2020 TIP 最新去雾算法(摘要.网络结构图及论文链接) 本博客先介绍 2020 TIP 最新去雾算法的摘要.网络结构图及论文链接,后续将陆续补充较为详细的内容. 目录 ...
论文速读 -- BEVerse
论文速读 – BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Dr ...
【论文速读】城市自动驾驶应用的概率语义地图
点云PCL免费知识星球,点云论文速读. 标题:Probabilistic Semantic Mapping for Urban Autonomous Driving Applications 作者:D ...
【论文速读】RandLA-Net大规模点云的高效语义分割
点云PCL免费知识星球,点云论文速读. 文章:RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds 作者:Qi ...
【论文速读】基于投影方法的激光雷达点云处理比较
点云PCL免费知识星球,点云论文速读. 文章:LiDAR point-cloud processing based on projection methods: a comparison 作者:Gui ...
【论文速读】基于图像的伪激光雷达三维目标检测
点云PCL免费知识星球,点云论文速读. 标题:End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection 作者:Rui Qian, Divy ...
【点云论文速读】最佳点云分割分析
点云PCL免费知识星球,点云论文速读. 标题:Learning to Optimally Segment Point Clouds 作者:Peiyun Hu, David Held 星球ID:part ...
【点云论文速读】点云高质量3D表面重建
点云PCL免费知识星球,点云论文速读. 标题:Local Implicit Grid Representations for 3D Scenes 作者:Chiyu "Max" Ji ...

论文速读：自监督弱光图像增强：Self-supervised Image Enhancement Network: Training with Low Light Images Only