一种结构和纹理感知 Retinex 模型 (2020 TIP) (1 of 2)

STAR: A Structure and Texture Aware Retinex Model

[PAPER] [GitHub]

Fig. 1. An example to illustrate the applications of the proposed STAR model based on Retinex theory. (a) The input low-light and color-distorted image; (b) the estimated illumination component of (a); (c) the estimated reflectance component of (a); (d) the extracted structure and texture maps (half each) of (a); (e) the illumination enhanced low-light image of (a); (f) the color corrected image of (a).

Method: Structure and Texture Aware Retinex Model

Proposed Model

In this section, we propose a Structure and Texture Aware Retinex (STAR) model to simultaneously estimate the illumination I and the reflectance R of an observed image O. To make our STAR model as simple as possible, we adopt the TV $\mathfrac{l}_2$ -norm to regularize the illumination and reflectance components. The proposed STAR model is formulated as

where $S_0$ and $T_0$ are the two matrices defined in (9)【式（9）相关内容查看一种结构和纹理感知 Retinex 模型 (2020 TIP) (1 of 2)】, indicating the structure map of the illumination and the texture map of the reflectance, respectively. The structure should be small enough to preserve the edges of objects in the scene, while large enough to suppress the details (as the inverse of Figure 2 (d,e)). On the other hand, the texture map should be small enough to reveal the details (as the inverse of Figure 2 (b,c)).

给出总的正则化方程：

STAR 模型采用 TV $\mathfrac{l}_2$ -norm 来规则化光照和反射率成分。所提出的星模型表述为公式（10）其中 $S_0$ and $T_0$ 为 (9) 中定义的两个矩阵，分别表示光照的结构图和反射率的纹理图。结构应该足够小，以保留场景中对象的边缘，同时足够大，以抑制细节。另一方面，纹理映射应该足够小，以展示细节。

Optimization Algorithm

Since the objective function (10) is separable w.r.t. the two variables I and R, it can be solved via an alternative optimization algorithm. The two separated sub-problems are convex and alternatively solved. We initialize the matrix variables . Denote as the illumination and reflectance components at the k-th (k = 0, 1, 2, ..., K) iteration, respectively, and K is the maximum iteration number. By optimizing one variable at a time while fixing the other, we can alternatively update the two variables as follows：

由于目标函数 (10) 是可分离的 w.r.t. 两个变量 I 和 R，它可以通过另一种优化算法求解。两个分离的子问题是凸的，且可交替解决。我们初始化矩阵变量。表示第 k 次迭代的光照分量和反射率分量 (k = 0,1,2，…， K) 迭代，K 为最大迭代次数。通过一次优化一个变量，同时修复另一个变量，本文选择如下方式更新这两个变量：

a) Update I while fifixing R.

With in the k-th iteration, the optimization problem with respect to I becomes:

To solve the problem (11), we reformulate it into a vectorized format. To this end, with the vectorization operator vec(·), we denote vectors , , which are of length nm. Denote by G the Toeplitz matrix from the discrete gradient operator with forward difference, then we have . Denote by the matrices with lying on the main diagonals, respectively. Then, the problem (11) is transformed into a standard least squares regression problem

By differentiating problem (12) with respect to i, and setting the derivative to 0, we have the following solution

We then reformulate the obtained into matrix format via the inverse vectorization .

可分离迭代优化之：固定 R 的同时更新 I：

为了解决问题 (11)，我们将其重新表述为向量化的格式。为此，我们用矢量化算子 vec(·) 表示向量，长度为 nm。用正差分离散梯度算子的 Toeplitz 矩阵表示，则。用分别表示位于主对角线上的矩阵。然后将问题 (11) 转化为标准最小二乘回归问题，即公式（12）。通过对问题 (12) 对 i 求导，并使导数为 0，得解（13）。

b) Update R while fifixing I.

After acquiring from the solution (11), the optimization problem (10) with respect to R is similar to that of I:

Similarly, we reformulate the problem (14) into a vectorized format. Additionally, we denote . which are of length nm. We also have . Denote by the matrices with lying on the main diagonals, respectively. Then, the problem (14) is also transformed into a standard least squares problem

By differentiating the problem (15) with respect to r, and setting the derivative to 0, we have the following solution

We then reformulate the obtained into the matrix format via inverse vectorization .

可分离迭代优化之：固定 I 的同时更新 R：

该过程同 固定 R 的同时更新 I 相同，此处略。

The Iteration

The above alternative algorithm are repeated until the convergence condition is satisfied or the number of iterations exceeds a preset threshold. The convergence condition of the alternative optimization algorithm is: kIk+1 1IkkF /kIkkF ε or kRk+1 1 RkkF /kRkkF k ε is satisfied, or the maximum iteration number K is achieved. We set ε = 1002 and K = 20 in our experiments. Our STAR model (10) can be efficiently solved since there are only two variables in problem (10) and each sub-problem has closed-form solution.

迭代过程：

重复上述替代算法，直到满足收敛条件或迭代次数超过预先设定的阈值。该替代优化算法的收敛条件为: 或满足，或达到最大迭代次数 k。本文实验中设置 ε = 1002； K = 20。由于问题 (10) 中只有两个变量，而且每个子问题都有封闭形式的解，所以STAR 模型 (10) 可以有效地求解。

Convergence Analysis.

The convergence of Algorithm 1 can be guaranteed since the overall objective function (10) is convex with a global optimal solution. In Figure 4, we plot the {average convergence curves of the errors of on the 35 low-light images collected from [9], [17], [19], [24]. One can see that either of them is reduced to less than ε = 0.01 in 10 iterations.

收敛性分析。由于总体目标函数(10)是凸的，具有全局最优解，因此算法1的收敛性可以得到保证。在图4中，本文对 [9]、[17]、[19]、[24] 采集的 35 张微光图像绘制了误差的{平均收敛曲线。可以看到，在 10 次迭代中，它们中的任何一个都减小到 ε = 0.01 以下。

Fig. 4: The average convergence curves of (blue line) and (orange line) of our STAR on the 35 images collected from [9], [17], [19], [24].

[9] A joint intrinsicextrinsic prior model for Retinex. (ICCV 2017).

[17] A probabilistic method for image enhancement with simultaneous illumination and reflectance estimation. (TIP 2015).

[19] A weighted variational model for simultaneous reflectance and illumination estimation.(CVPR 2016).

[24] LIME: Low-light image enhancement via illumination map estimation. (TIP 2017).

Updating Structure and Texture Awareness

Until now, we have obtained the decomposition of O = I $\odot$ R. To achieve better estimation on illumination and reflflectance, we update the structure and texture aware maps S and T , and then solve the renewed problem (10). The alternative updating of (S, T) and (I, R) are repeated for L iterations. We set L = 4 to balance the speed-accuracy trade-off of the proposed STAR model in our experiments. We summarize the updating procedures in Algorithm 2.

为了更好的估计光照和反射率，本文还更新了结构和纹理感知映射 S 和 T，然后解决更新的问题 (10)。(S, T) 和 (I, R) 的交替更新重复进行 L 次迭代。在实验中，设 L = 4 来平衡所提出的 STAR 模型的速度-精度平衡。总结了算法 2 中的更新过程。

Complexity Analysis.

Now we discuss the complexity analysis of the proposed Algorithms 1 and 2. Assume that the input image is of size n×m. In Algorithm 1, the costs for updating I and R are both O(nmK) due to the diagonalization operations, where K is the number of iterations in Algorithm 1. In Algorithm 2, the costs for updating S and T are also O(nmKL), where L is the number of updating in Algorithm 2. As such, the overall complexity of our STAR for Retinex decomposition is O(nmKL).

假设输入图像的大小为 n×m。在算法 1 中，由于对角化操作，更新 I 和 R 的代价均为 O(nmK)，其中 K 为算法 1 的迭代次数。在算法 2 中，更新 S 和 T 的代价也是 O(nmKL)，其中 L 是算法 2 中更新的次数。因此，STAR 的 Retinex分解的总体复杂性是 O(nmKL)。

一些实验结果展示：

Fig. 5: Comparisons of illumination and reflflectance components by different Retinex decomposition methods on the image “2” from the 35 low-light images collected from [9], [17], [19], [24].

Fig. 6: Comparisons of illumination and reflflectance components by different Retinex decomposition methods on the image “15” from the 35 low-light images collected from [9], [17], [19], [24].

Fig. 7: Comparison of illumination and reflflectance components by the proposed STAR model with ETV or EMLV weighting schemes. For better comparisons, we illustrate the components in the RGB channels instead of V channel.

Fig. 8: Ablation study of structure or texture map on the illumination and reflflectance decomposition performance of the proposed STAR model. For better comparisons, we illustrate the components in the RGB channels instead of V channel.

Fig. 9: Ablation study of the parameters (γs, γt) on the illumination and reflflectance decomposition performance of the proposed STAR model. For better comparisons, we illustrate the components in the RGB channels instead of V channel.

Fig. 10: Ablation study of updating iterations L on the illumination and reflflectance decomposition performance of the proposed STAR model. For better comparisons, we illustrate the components in the RGB channels instead of V channel.

一种结构和纹理感知 Retinex 模型 (2020 TIP) (2 of 2)相关推荐

CM-GAN：图像大面积缺失修复，兼顾全局结构和纹理细节
本文机器之心编辑部来自罗彻斯特大学和 Adobe Research 的研究者提出了一种新的生成网络 CM-GAN,很好地合成了整体结构和局部细节,在定量和定性评估方面都显著优于现有 SOTA 方 ...
如何利用 C# 实现神经网络的感知器模型？
前几天我们介绍了如何利用 C# 对神经网络模型进行抽象,在这篇图文中,我们抽象了单个神经元 Neuro,网络层 Layer,网络结构 Network,激活函数 IActivationFunction ...
CNN结构基元：纹理结构和纹理基元方程化GLOH、Gabor...（Code）
模式识别专注于寻找相同模式的共性和不同模式的分离.CNN把特征提取全局化,其中重要的一个是纹理特征,利用卷积核来表示纹理基元,用以重现模式,应如何显示表示. 第一次使用纹理特征,2012年,使用了灰度 ...
语音识别 AI 挑战赛上线：用深度学习三种结构，对 50 种环境声音分类！
雷锋网 AI 源创评论按:此前,AI 研习社(https://god.yanxishe.com )陆续推出了医疗.美食.安全等多个领域的图像识别挑战赛以及 NLP 方向的挑战赛 30 余场.在这过程中 ...
计算智能——感知器模型
主要内容 1.感知器总述 2.感知器模型 3.感知器策略(建立损失函数) 4.感知器算法(梯度下降和随机梯度下降) 4.1梯度下降 4.2随机梯度下降 5.感知器MATLAB简单实现 5.1newp函 ...
多层神经元感知器模型_使用多层感知器模型对星系进行分类
多层神经元感知器模型 Cosmos is an intriguing space to observe and analyse, it is the stronghold for any scienc ...
ECCV / TNNLS 20 - 如何在异常检测中利用“结构structure - 纹理texture”一致性【P-Net，MemSTC-Net】
文章目录前言 P-Net MEMSTC-Net 总结前言不管是医学图像还是工业图像领域,异常检测都是非常重要的一个研究方向(在训练时只有正常样本,测试时需要模型判断输入样本是否存在异常).存在一 ...
让机器搞懂100万种隐含语义腾讯Peacock模型全揭秘
让机器搞懂100万种隐含语义腾讯Peacock模型全揭秘腾讯模型 36大数据 · 2015-02-28 17:21 Peacock通过并行计算对10亿x1亿级别的大规模矩阵进行分解,从海量样本数据 ...
WebGL three.js学习笔记 6种类型的纹理介绍及应用
WebGL three.js学习笔记 6种类型的纹理介绍及应用本文所使用到的demo演示: 高光贴图Demo演示反光效果Demo演示(因为是加载的模型,所以速度会慢) (一)普通纹理计算机图形学 ...

一种结构和纹理感知 Retinex 模型 (2020 TIP) (2 of 2)