声学——声源定位阅读笔记

博客资料

图像融合常用数据集整理

基于深度学习的多聚焦图像融合(Multi-Focus Image Fusion)论文及代码整理

文献

1.Multi-focus image fusion with a deep convolutional neural network [CNN(IF 2017)][paper][code]

2.Ensemble of CNN for multi-focus image fusion [ECNN(IF 2019)][paper][code]

3.Multilevel features convolutional neural network for multifocus image fusion [MLFCNN(TCI 2019)] [paper]

4.DRPL: Deep Regression Pair Learning for Multi-Focus Image Fusion [DRPL(TIP 2020)] [paper][code]

5.An α-Matte Boundary Defocus Model-Based Cascaded Network for Multi-Focus Image Fusion [MMF-Net(TIP 2020)] [paper][code]

6.Towards Reducing Severe Defocus Spread Effects for Multi-Focus Image Fusion via an Optimization Based Strategy [MFF-SSIM(TCI 2020)][paper][code]

7.Structural Similarity Loss for Learning to Fuse Multi-Focus Images [MFNet(Sensors 2020)][paper]

8.Global-Feature Encoding U-Net (GEU-Net) for Multi-Focus Image Fusion [GEU-Net(TIP 2021)][paper][code]

9.DTMNet: A Discrete Tchebichef Moments-Based Deep Neural Network for Multi-Focus Image Fusion [DTMNet(ICCV 2021)][paper]

10.SMFuse: Multi-Focus Image Fusion Via Self-Supervised Mask-Optimization [SMFuse(TCI 2021)][paper][code]

11.Depth-Distilled Multi-focus Image Fusion [D2MFIF(TMM 2021)][paper]

12.SESF-Fuse: an unsupervised deep model for multi-focus image fusion [SESF-Fuse(NCAA 2021)][paper][code]

单点定位

笔记

A Deep Learning Localization Method for Measuring Abdominal Muscle Dimensions in Ultrasound Images

We propose a novel localization framework that usesFCN8 [14] with a CoordConv layer [16] to detect the points for muscle dimension measurement while leveraging Localization-based Counting Fully Convolutional Network (LCFCN) loss to maximize the performance of the network with point-level supervision;

We demonstrate that our novel approach of using LCFCN along with CoordConv yields significant improvement over conventional landmark detection and almost matches the performance of ultrasound technicians.

多模态声源定位（SSL）

Sound source localization based on multi-task learning and image translation network [paper]

ym=s∗hm+nmy_m=s*h_m+n_mym=s∗hm+nm

其中，ym∈RLy_m\in \mathbb R^Lym∈RL是第mmm个麦克风接收到的信号(M∈{1,2,⋯,M}M\in\{1,2,\cdots ,M\}M∈{1,2,⋯,M}),sss是源信号，nmn_mnm是麦克风第nnn个噪声信号，它表示房间的混响。y=[y1,y2,⋯,ym]T∈RM×L\mathbf y=[y_1,y_2,\cdots,y_m]^T\in\mathbb R^{M\times L}y=[y1,y2,⋯,ym]T∈RM×L表示从音频长度为LLL的所有传感器接收的信号的集合。

对于每个阵列有KKK个麦克风的NNN个阵列，即M=NKM=NKM=NK，则y\mathbf yy为K×N×LK\times N\times LK×N×L的张量，即研究的y∈RK×N×L\mathbf y\in\mathbb R^{K\times N\times L}y∈RK×N×L

特征提取

快照的数量可以用音频长度LLL和快速傅里叶变换（FFT）和窗口长度FFF表示为S=L/2FS=L/2FS=L/2F(仅考虑频率段的正一半)，可得到Y∈CS×F×K×NY\in\mathbb C^{S\times F\times K\times N}Y∈CS×F×K×N作为y\mathbf{y}y的快速傅里叶变换（STFT）的输出，这里yyy是针对单个测量情况计算的。如果数据集中有TTT个独立的测量值，那么Y\mathbf YY的帧数为C=TSC=TSC=TS(Y∈CS×F×K×NY\in\mathbb C^{S\times F\times K\times N}Y∈CS×F×K×N)

DOA特征提取

假设均匀线阵(ULA)和宽带信号再任意角度θ\thetaθ上传统波束形成输出的幅度为

P(θ)=∣wθHYfns∣=∣∑k=1KYfns[k]ej(2πkusin⁡θf0/c)∣P(\theta)=\left|w_\theta^H Y_f^{ns}\right|=\left|\sum^K_{k=1}Y_f^{ns}[k]e^{j(2\pi ku\sin{\theta}f_0/c)}\right|P(θ)=∣∣wθHYfns∣∣=∣∣k=1∑KYfns[k]ej(2πkusinθf0/c)∣∣

这里，wθ=[ej(2πkusin⁡θf01/c),ej(2πkusin⁡θf02/c),⋯,ej(2πkusin⁡θf0K/c)]Tw_{\theta}=\left[e^{j(2\pi ku\sin{\theta}f_01/c)},e^{j(2\pi ku\sin{\theta}f_02/c)},\cdots,e^{j(2\pi ku\sin{\theta}f_0K/c)}\right]^Twθ=[ej(2πkusinθf01/c),ej(2πkusinθf02/c),⋯,ej(2πkusinθf0K/c)]T,uuu,f0f_0f0和ccc代表麦克风间距，中频和声速，P(θ)P(\theta)P(θ)中的每个点表示信号与基于自由空间传播的预期波束形成期响应的相似性；Yfns∈CK×1Y_f^{ns}\in\mathbb C^{K\times 1}Yfns∈CK×1表示YYY中第nnn个数组，第sss个快照和第fff个频率的数据

范围特征提取

范围特征是基于距离的特征，首先假设只考虑距离引起的相位差，与角度特征提取类似，距离ddd处的常规波束形成输出为

P(d)=∣wdHYkns∣=∣∑l=1FYkns[l]ej(2πlfld/c)∣P(d)=\left|w_d^H Y_k^{ns}\right|=\left|\sum_{l=1}^F Y_k^{ns}[l]e^{j(2\pi lf_ld/c)}\right|P(d)=∣∣wdHYkns∣∣=∣∣l=1∑FYkns[l]ej(2πlfld/c)∣∣

其中，wd=[ej(2πlfld1/c),ej(2πlfld2/c),⋯,ej(2πlfldF/c)]w_d=\left[e^{j(2\pi lf_ld1/c)},e^{j(2\pi lf_ld2/c)},\cdots,e^{j(2\pi lf_ldF/c)}\right]wd=[ej(2πlfld1/c),ej(2πlfld2/c),⋯,ej(2πlfldF/c)],Ykns∈CF×1Y_k^{ns}\in\mathbb{C}^{F\times 1}Ykns∈CF×1表示第nnn个阵列中第kkk个麦克风和第sss个快照的数据，flf_lfl对应的是第lll频段的频率

DOA范围特征提取

对于角度为θ\thetaθ距离为ddd的数组

Pns(θ,d)=∣wθHYnswd∗∣=∣∑k=1K∑l=1FYklej(2πkusin⁡θf0/c)ej(2πlfld/c)∣P_{ns}(\theta,d)=\left|w_\theta^HY^{ns}w_d^*\right|=\left|\sum_{k=1}^K\sum_{l=1}^FY_{kl}e^{j(2\pi ku\sin\theta f_0/c)}e^{j(2\pi lf_ld/c)}\right|Pns(θ,d)=∣∣wθHYnswd∗∣∣=∣∣k=1∑Kl=1∑FYklej(2πkusinθf0/c)ej(2πlfld/c)∣∣

其中，n∈{1,2,⋯,N}n\in \{1,2,\cdots,N\}n∈{1,2,⋯,N},s∈{1,2,⋯,S}s\in \{1,2,\cdots,S\}s∈{1,2,⋯,S},Yns[k,l]=Ykl⋅Pns(θ,d)Y_{ns}[k,l]=Y_{kl}\cdot P_{ns}(\theta,d)Yns[k,l]=Ykl⋅Pns(θ,d)是常规波束形成器的幅度输出，YklY_{kl}Ykl表示第kkk个麦克风第lll个频段的STFT输出；(⋅)H(\cdot)^H(⋅)H,(⋅)T(\cdot)^T(⋅)T,(⋅)∗(\cdot)^*(⋅)∗分别表示埃米尔特复共轭转置、转置和复共轭。

定位目标

预测的位置(x′,y′)(x',y')(x′,y′)与真实位置(x,y)(x,y)(x,y)的距离为

d(x′,y′)=(x′−x)2+(y′−y)2d(x',y')=\sqrt{(x'-x)^2+(y'-y)^2}d(x′,y′)=(x′−x)2+(y′−y)2

它在目标图像中的值

l(x′,y′)=e−d(x′,y′)2/σl(x',y')=e^{-d(x',y')^2/\sigma}l(x′,y′)=e−d(x′,y′)2/σ

其中σ\sigmaσ是控制衰减速率的超参数,在d(x′,y′)=0d(x',y')=0d(x′,y′)=0时，l(x′,y′)=1l(x',y')=1l(x′,y′)=1为其最大值

k-Wave工具箱

示例