前言

显著性检测，就是使用图像处理技术和计算机视觉算法来定位图片中最“显著”的区域。显著区域就是指图片中引人注目的区域或比较重要的区域，例如人眼在观看一幅图片时会首先关注的区域。例如下图，我们人眼一眼看过去首先注意到的不是草坪，而是躺在草坪上的内马尔，内马尔所在的区域就是显著性区域。这种自动定位图像或场景重要区域的过程称为显着性检测。显著性检测在目标检测、机器人领域有很多应用。

三种显著性检测算法

在OpenCV的saliency模块中有三种显著性检测算法：Static saliency：此类显着性检测算法依赖于图像特征和统计信息来定位图像中显著性区域。

Motion saliency:此类显着性检测算法输入为视频或一系列连续帧。运动显着性算法处理这些连续的帧，并跟踪帧中“移动”的对象。这些移动的对象被认为是显着性区域。

Objectness：这类显著性检测算法计算出一个个的建议区域(proposals)，这些建议区域被认为是目标可能存在的区域。

需要注意的是显著性检测不是目标检测，显著性检测算法并不能判断图片中是否有“目标”，它只能检测出图片中它“认为”有目标的区域，这些区域可能真的包含目标，也可能不包含目标。通常显著性检测算法是许多计算机视觉任务的第一步，检测出显著性区域后再对这些显著性区域进一步判断和预测。

显着性检测器通常是非常快速的算法，能够实时运行。显着性检测器的结果随后被传递到计算量更大的算法中。这样后续计算量大的算法不必在图像的每个区域运行，而只需在显著性区域上运行。

OpenCV中的显著性检测算法

OpenCV的显著性检测算法在contrib包中，所以要使用显著性检测算法需要安装OpenCV_contrib包。OpenCV中显著性检测模块Saliency的类关系如下：

OpenCV提供类4种显著性检测算法的实现：cv2.saliency.ObjectnessBING_create()

cv2.saliency.StaticSaliencySpectralResidual_create()

cv2.saliency.StaticSaliencyFineGrained_create()

cv2.saliency.MotionSaliencyBinWangApr2014_create()

下面我将使用这四种显著性检测算法来检测图片中的显著性区域。

代码结构

├── images

│ ├── barcelona.jpg

│ ├── boat.jpg

│ ├── neymar.jpg

│ └── players.jpg

├── objectness_trained_model [9 entries]

│ ├── ObjNessB2W8HSV.idx.yml.gz

│ ├── ...

├── static_saliency.py

├── objectness_saliency.py

└── motion_saliency.py

images文件夹中是测试图片。

objectness_trained_model是Objectness Saliency算法需要的模型文件。

Static saliency

static_saliency.py使用两种Static saliency算法：cv2.saliency.StaticSaliencySpectralResidual_create()和cv2.saliency.StaticSaliencyFineGrained_create()。使用computeSaliency()计算图片的显著性区域，返回结果是和输入图片一样大小的矩阵，每个像素位置的取值[0,1]，值越大表示该像素位置越显著。最后我将返回显著性矩阵可视化出来。

这两种算法的论文：

代码如下：

# USAGE

# python static_saliency.py --image images/neymar.jpg

# import the necessary packages

import argparse

import cv2

# construct the argument parser and parse the arguments

ap = argparse.ArgumentParser()

ap.add_argument("-i", "--image", required=True,

help="path to input image")

args = vars(ap.parse_args())

# args={"image":"images/boat.jpg"}

# load the input image

image = cv2.imread(args["image"])

# initialize OpenCV's static saliency spectral residual detector and

# compute the saliency map

saliency = cv2.saliency.StaticSaliencySpectralResidual_create()

(success, saliencyMap) = saliency.computeSaliency(image)

saliencyMap = (saliencyMap * 255).astype("uint8")

cv2.imshow("Image", image)

cv2.imshow("Output", saliencyMap)

cv2.waitKey(0)

# initialize OpenCV's static fine grained saliency detector and

# compute the saliency map

saliency = cv2.saliency.StaticSaliencyFineGrained_create()

(success, saliencyMap) = saliency.computeSaliency(image)

saliencyMap = (saliencyMap * 255).astype("uint8")

# if we would like a *binary* map that we could process for contours,

# compute convex hull's, extract bounding boxes, etc., we can

# additionally threshold the saliency map

threshMap = cv2.threshold(saliencyMap, 0, 255,

cv2.THRESH_BINARY | cv2.THRESH_OTSU)[1]

# show the images

cv2.imshow("Image", image)

cv2.imshow("Output", saliencyMap)

cv2.imshow("Thresh", threshMap)

cv2.waitKey(0)

命令行运行：

python static_saliency.py --image images/neymar.jpg

cv2.saliency.StaticSaliencySpectralResidual_create()结果如下：

cv2.saliency.StaticSaliencyFineGrained_create()结果如下：

Objectness

OpenCV使用的算法是：

objectness_saliency.py代码如下：

# USAGE

# python objectness_saliency.py --model objectness_trained_model --image images/barcelona.jpg

# import the necessary packages

import numpy as np

import argparse

import cv2

# construct the argument parser and parse the arguments

ap = argparse.ArgumentParser()

ap.add_argument("-m", "--model", required=True,

help="path to BING objectness saliency model")

ap.add_argument("-i", "--image", required=True,

help="path to input image")

ap.add_argument("-n", "--max-detections", type=int, default=10,

help="maximum # of detections to examine")

args = vars(ap.parse_args())

# load the input image

image = cv2.imread(args["image"])

# initialize OpenCV's objectness saliency detector and set the path

# to the input model files

saliency = cv2.saliency.ObjectnessBING_create()

saliency.setTrainingPath(args["model"])

# compute the bounding box predictions used to indicate saliency

(success, saliencyMap) = saliency.computeSaliency(image)

numDetections = saliencyMap.shape[0]

# loop over the detections

for i in range(0, min(numDetections, args["max_detections"])):

# extract the bounding box coordinates

(startX, startY, endX, endY) = saliencyMap[i].flatten()

# randomly generate a color for the object and draw it on the image

output = image.copy()

color = np.random.randint(0, 255, size=(3,))

color = [int(c) for c in color]

cv2.rectangle(output, (startX, startY), (endX, endY), color, 2)

# show the output image

cv2.imshow("Image", output)

cv2.waitKey(0)

命令行运行：

python objectness_saliency.py --model objectness_trained_model --image images/boat.jpg

结果如下：

Motion saliency

OpenCV使用的算法是：

motion_saliency.py代码如下：

# USAGE

# python motion_saliency.py

# import the necessary packages

from imutils.video import VideoStream

import imutils

import time

import cv2

# initialize the motion saliency object and start the video stream

saliency = None

vs = VideoStream(src=0).start()

time.sleep(2.0)

# loop over frames from the video file stream

while True:

# grab the frame from the threaded video stream and resize it

# to 500px (to speedup processing)

frame = vs.read()

frame = imutils.resize(frame, width=500)

# if our saliency object is None, we need to instantiate it

if saliency is None:

saliency = cv2.saliency.MotionSaliencyBinWangApr2014_create()

saliency.setImagesize(frame.shape[1], frame.shape[0])

saliency.init()

# convert the input frame to grayscale and compute the saliency

# map based on the motion model

gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

(success, saliencyMap) = saliency.computeSaliency(gray)

saliencyMap = (saliencyMap * 255).astype("uint8")

# display the image to our screen

cv2.imshow("Frame", frame)

cv2.imshow("Map", saliencyMap)

key = cv2.waitKey(1) & 0xFF

# if the `q` key was pressed, break from the loop

if key == ord("q"):

break

# do a bit of cleanup

cv2.destroyAllWindows()

vs.stop()

命令行运行：

python motion_saliency.py

结果如下：视频显著性检测https://www.zhihu.com/video/1224748542654451712