Note from author :

作者注：

This tutorial is the foundation of computer vision delivered as “Lesson 3” of the series, there are more Lessons upcoming which would talk to the extend of building your own deep learning based computer vision projects. You can find the complete syllabus and table of content here

本教程是本系列“第3课”中提供的计算机视觉的基础，接下来将有更多课程与构建基于深度学习的计算机视觉项目相关。 您可以在此处找到完整的课程提纲和目录

Target Audience : Final year College Students, New to Data Science Career, IT employees who wants to switch to data science Career .

目标受众 ：最后一年的大学生，数据科学职业新手，想要转用数据科学职业的IT员工。

Takeaway : Main takeaway from this article :

外卖 ：本文的主要外卖：

1. Morphological operations形态运算
2. Exercise to extract the tabular structure in an invoice using Morphological operations练习使用形态学运算来提取发票中的表格结构

形态运算 (Morphological operations)

Morphological operations are simple transformations applied to binary images. More specifically, we apply morphological operations to shapes and structures inside of images.

形态学运算是应用于二进制图像的简单转换。 更具体地说，我们将形态学运算应用于图像内部的形状和结构 。

We can use morphological operations to increase the size of objects in images as well as decrease them.

我们可以使用形态学运算来增加和减少图像中对象的大小。

We can also utilize morphological operations to close gaps between objects as well as open them.

我们也可以利用接近空白形态学操作对象之间以及打开它们。

Some of the morphological operations :

一些形态学操作：

• Erosion

  侵蚀

• Dilation

  扩张

• Opening

  开场

• Closing

  闭幕

• Morphological gradient

  形态梯度

• Black hat

  黑帽

• Top hat (or “White hat”)

  高顶礼帽(或“白帽”)

Sometime, we don’t have to use fancy algorithms to solve computer vision problems. Say few months back i was working on extracting the contents of an invoice The contents inside the structuring element of the invoice

有时，我们不必使用幻想算法来解决计算机视觉问题。 说几个月前，我正在提取发票的内容发票的结构元素中的内容

You are more likely to find solution using morphological operations.

您更有可能使用形态学运算来找到解决方案。

侵蚀： (Erosion:)

In erosion, the foreground object is eroded to make it smaller as shown in the Fig

在腐蚀中，前景物体被腐蚀以使其变小，如图所示。

Erosion works by defining a structuring element and then sliding this structuring element from left-to-right and top-to-bottom across the input image.

侵蚀的工作方式是定义一个结构元素，然后在输入图像上从左到右和从上到下滑动此结构元素。

A foreground pixel in the input image will be kept only if ALL pixels inside the structuring element are > 0. Otherwise, the pixels are set to 0 (i.e. background).

仅当结构化元素内的所有 像素均大于0时，才会保留输入图像中的前景像素。 否则，像素设置为0 (即背景)。

Erosion is useful for removing small blobs in an image or disconnecting two connected objects.

侵蚀对于去除图像中的小斑点或断开两个连接的对象很有用。

#Erosion of text with 2 different iteration

＃2次不同迭代的文字侵蚀

import cv2import argparse

导入cv2导入argparse

apr = argparse.ArgumentParser()apr.add_argument(“-i”, “ — image”, required=True, help=”Path to the image”)args = vars(apr.parse_args())

apr = argparse.ArgumentParser()apr.add_argument(“-i”，“ —图片”，required = True，help =“图片路径”)args = vars(apr.parse_args())

image = cv2.imread(args[“image”])

图片= cv2.imread(args [“ image”])

print(f’(Height,Width,Depth) of the image is: {image.shape}’)

print(图像的f'(高度，宽度，深度)为：{image.shape}')

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)cv2.imshow(“Original”, image)

灰色= cv2.cvtColor(图片，cv2.COLOR_BGR2GRAY)cv2.imshow(“原始图片”，图片)

eroded2 = cv2.erode(gray.copy(), None, iterations=2)cv2.imshow(“Eroded Image with 2 Iteration”, eroded2) eroded4 = cv2.erode(gray.copy(), None, iterations=4)cv2.imshow(“Eroded Image with 4 Iteration”, eroded4)

eroded2 = cv2.erode(gray.copy()，无，迭代次数= 2)cv2.imshow(“腐蚀2次迭代的图像”，eroded2)eroded4 = cv2.erode(gray.copy()，无，迭代次数= 4) cv2.imshow(“带有4次迭代的腐蚀图像”，腐蚀4)

cv2.waitKey(0)

cv2.waitKey(0)

As morphological operations can only be done on grey scale image(black and white) ,however there are exceptions to it .We convert the colored image to a grey scale image by calling cv2.cvtColor function.

由于形态学操作只能在灰度图像(黑白)上进行，因此有例外。我们通过调用cv2.cvtColor函数将彩色图像转换为灰度图像。

We perform the actual erosion on the next line by making a call to the cv2.erode function. This function takes two required arguments and a third optional one. The first argument is the image that we want to erode — in this case, it’s our binary image. The second argument to cv2.erode is the structuring element. If this value is None , then a structuring element, identical to the 8-neighborhood structuring element we saw above will be used. Of course, you could supply your own custom structuring element here instead of None as well.

我们通过调用cv2.erode函数在下一行执行实际腐蚀。 此函数接受两个必需的参数和第三个可选的参数。 第一个参数是我们要侵蚀的图像-在这种情况下，它是我们的二进制图像。 cv2.erode的第二个参数是结构元素。 如果此值为None，那么将使用与我们在上面看到的8邻域结构化元素相同的结构化元素。 当然，您可以在此处提供您自己的自定义结构元素，而不是无。

The last argument is the number of iterations the erosion is going to be performed. As the number of iterations increases, we’ll see more and more of the original image characters eaten away.

最后一个参数是腐蚀将要执行的迭代次数。 随着迭代次数的增加，我们将看到越来越多的原始图像字符被吞噬。

扩张： (Dilation:)

The opposite of an erosion is a dilation. Just like an erosion will eat away at the foreground pixels, a dilation will grow the foreground pixels.

侵蚀的反面是膨胀 。 就像侵蚀会侵蚀 掉前景像素一样， 膨胀也会增加前景像素。

Dilations increase the size of foreground object and are especially useful for joining broken parts of an image together.

膨胀会增加前景对象的大小，对于将图像的损坏部分连接在一起尤其有用。

Dilations, just as an erosion, also utilize structuring elements — a center pixel p of the structuring element is set to white if ANY pixel in the structuring element is > 0.

像侵蚀一样，膨胀也使用结构元素-如果结构元素中的任何像素> 0 ，则结构元素的中心像素p设置为白色 。

#Dilation of text with 2 different iteration

＃使用2个不同的迭代对文本进行膨胀

import cv2import argparse

导入cv2导入argparse

apr = argparse.ArgumentParser()apr.add_argument(“-i”, “ — image”, required=True, help=”Path to the image”)args = vars(apr.parse_args())

apr = argparse.ArgumentParser()apr.add_argument(“-i”，“ —图片”，required = True，help =“图片路径”)args = vars(apr.parse_args())

image = cv2.imread(args[“image”])

图片= cv2.imread(args [“ image”])

print(f’(Height,Width,Depth) of the image is: {image.shape}’)

print(f'(图像的高度，宽度，深度)为：{image.shape}')

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)cv2.imshow(“Original”, image)

灰色= cv2.cvtColor(图片，cv2.COLOR_BGR2GRAY)cv2.imshow(“原始图片”，图片)

dilated2 = cv2.dilate(gray.copy(), None, iterations=2)cv2.imshow(“Dilated Image with 2 Iteration”, dilated2) dilated4 = cv2.dilate(gray.copy(), None, iterations=4)cv2.imshow(“Dilated Image with 4 Iteration”, dilated4)

dilated2 = cv2.dilate(gray.copy()，无，迭代次数= 2)cv2.imshow(“具有2个迭代的扩张图像”，dilated2)dilated4 = cv2.dilate(gray.copy()，无，迭代次数= 4) cv2.imshow(“具有4个迭代的放大图像”，dilated4)

cv2.waitKey(0)

cv2.waitKey(0)

The actual dilation is performed by making a call to the cv2.dilate function, where the actual function signature is identical to that of cv2.erode .

实际的扩展是通过调用cv2.dilate函数执行的，其中实际的函数签名与cv2.erode的签名相同。

开场时间： (Opening:)

An opening is an erosion followed by a dilation.

开口是侵蚀，然后是膨胀 。

Performing an opening operation allows us to remove small blobs from an image: first an erosion is applied to remove the small blobs, then a dilation is applied to regrow the size of the original object.

执行打开操作使我们可以从图像中去除小斑点：首先应用腐蚀以去除小斑点，然后进行膨胀以重新生成原始对象的大小。

#Opening with kernel size of 10,10

＃以10,10的内核大小打开

import cv2import argparse

导入cv2导入argparse

apr = argparse.ArgumentParser()apr.add_argument(“-i”, “ — image”, required=True, help=”Path to the image”)args = vars(apr.parse_args())

apr = argparse.ArgumentParser()apr.add_argument(“-i”，“ —图片”，required = True，help =“指向图片的路径”)args = vars(apr.parse_args())

image = cv2.imread(args[“image”])

图片= cv2.imread(args [“ image”])

print(f’(Height,Width,Depth) of the image is: {image.shape}’)

print(f'(图像的高度，宽度，深度)为：{image.shape}')

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)cv2.imshow(“Original”, image)

灰色= cv2.cvtColor(图片，cv2.COLOR_BGR2GRAY)cv2.imshow(“原始图片”，图片)

kernelSize =(10,10)kernel = cv2.getStructuringElement(cv2.MORPH_RECT, kernelSize)opening = cv2.morphologyEx(gray, cv2.MORPH_OPEN, kernel)cv2.imshow(“Opening Image for kernel size 10,10:”, opening)

kernelSize =(10,10)内核= cv2.getStructuringElement(cv2.MORPH_RECT，kernelSize)打开= cv2.morphologyEx(灰色，cv2.MORPH_OPEN，内核)cv2.imshow(“正在为内核尺寸10,10打开图像：”，打开)

cv2.waitKey(0)

cv2.waitKey(0)

Here we make a call to cv2.getStructuringElement to build our structuring element. The cv2.getStructuringElement function requires two arguments: the first is the type of structuring element we want, and the second is the size of the structuring element (we have defined our kernel size to be (10,10)).

在这里，我们调用cv2.getStructuringElement来构建结构元素。 cv2.getStructuringElement函数需要两个参数：第一个是我们想要的结构元素的类型 ，第二个是结构元素的大小 (我们将内核大小定义为(10,10))。

We pass in a value of cv2.MORPH_RECT to indicate that we want a rectangular structuring element. But you could also pass in a value of cv2.MORPH_CROSS to get a cross shape structuring element (a cross is like a 4-neighborhood structuring element, but can be of any size), or cv2.MORPH_ELLIPSE to get a circular structuring element. Exactly which structuring element you use is dependent upon your application — and I’ll leave it as an exercise to the reader to play with each of these structuring elements.

我们传入一个cv2.MORPH_RECT值来表示我们想要一个矩形结构元素。 但是，您也可以传入值cv2.MORPH_CROSS以获取十字形结构元素(十字形类似于4邻域结构元素，但可以是任意大小)，也可以传入cv2.MORPH_ELLIPSE以获得圆形结构元素。 确切地说，您使用哪种结构化元素取决于您的应用程序-我将把它留给读者作为练习使用这些结构化元素的练习。

The actual opening operation is performed on next line by making a call to the cv2.morphologyEx function. This function is abstract in a sense — it allows us to pass in whichever morphological operation we want, followed by our kernel/structuring element. Please try passing other operations and see the difference for yourself.

实际的打开操作是通过调用cv2.morphologyEx函数在下一行执行的。 从某种意义上说，该函数是抽象的-它允许我们传递想要的任何形态运算，然后传递内核/结构元素。 请尝试通过其他操作，自己看看有什么不同。

The first required argument of cv2.morphologyEx is the image we want to apply the morphological operation to. The second argument is the actual type of morphological operation — in this case, it’s an opening operation. The last required argument is the kernel/structuring element that we are using.

cv2.morphologyEx的第一个必需参数是我们要对其应用形态学运算的图像。 第二个参数是形态运算的实际类型 -在这种情况下，它是一个打开运算。 最后一个必需的参数是我们正在使用的内核/结构元素。

Opening allows us to remove small blobs in an image. However, we can also perform other tasks like extracting the Horizontal and vertical lines in an image. We will see more about this in the below final exercise, where we will extract the tabular structure from an invoice using morphological operations.

开口使我们能够去除图像中的小斑点 。 但是，我们还可以执行其他任务，例如提取图像中的水平线和垂直线。 在下面的最终练习中，我们将看到更多相关信息，在该练习中，我们将使用形态学运算从发票中提取表格结构。

闭幕： (Closing:)

The exact opposite to an opening would be a closing. A closing is a dilation followed by an erosion.

与开口完全相反的是闭合 。 结束是扩张之后是侵蚀 。

As the name suggests, a closing is used to close holes inside of objects or for connecting components together.

顾名思义， 闭合用于闭合对象内部的Kong或将组件连接在一起。

#Closing with kernel size of 13,13

＃以13,13的内核大小关闭

import cv2import argparseimport numpy as np

导入cv2import argparseimport numpy as np

apr = argparse.ArgumentParser()apr.add_argument(“-i”, “ — image”, required=True, help=”Path to the image”)args = vars(apr.parse_args())

apr = argparse.ArgumentParser()apr.add_argument(“-i”，“ —图片”，required = True，help =“图片路径”)args = vars(apr.parse_args())

image = cv2.imread(args[“image”])

图片= cv2.imread(args [“ image”])

print(f’(Height,Width,Depth) of the image is: {image.shape}’)

print(f'(图像的高度，宽度，深度)为：{image.shape}')

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)cv2.imshow(“Original”, image)

灰色= cv2.cvtColor(图片，cv2.COLOR_BGR2GRAY)cv2.imshow(“原始图片”，图片)

kernelSize =(13,13)kernel = cv2.getStructuringElement(cv2.MORPH_RECT, kernelSize)closing = cv2.morphologyEx(gray, cv2.MORPH_CLOSE, kernel)cv2.imshow(“closing performed with kernel size of 13,13 “, closing)

kernelSize =(13,13)kernel = cv2.getStructuringElement(cv2.MORPH_RECT，kernelSize)关闭= cv2.morphologyEx(灰色，cv2.MORPH_CLOSE，内核)cv2.imshow(“以13,13的内核大小执行关闭，关闭)

cv2.waitKey(0)

cv2.waitKey(0)

形态梯度 (Morphological gradient)

A morphological gradient is the difference between the dilation and erosion. It is useful for determining the outline of a particular object of an image

形态梯度是膨胀和侵蚀之间的差异 。 这对于确定图像特定对象的轮廓很有用

#Morphological gradient applied to extract the boundary of an Image

＃应用形态学梯度来提取图像的边界

import cv2import argparseimport numpy as np

导入cv2import argparseimport numpy as np

apr = argparse.ArgumentParser()apr.add_argument(“-i”, “ — image”, required=True, help=”Path to the image”)args = vars(apr.parse_args())

apr = argparse.ArgumentParser()apr.add_argument(“-i”，“ —图片”，required = True，help =“指向图片的路径”)args = vars(apr.parse_args())

image = cv2.imread(args[“image”])

图片= cv2.imread(args [“ image”])

print(f’(Height,Width,Depth) of the image is: {image.shape}’)

print(f'(图像的高度，宽度，深度)为：{image.shape}')

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)cv2.imshow(“Original”, image)

灰色= cv2.cvtColor(图片，cv2.COLOR_BGR2GRAY)cv2.imshow(“原始图片”，图片)

kernelSize =(7,7)

kernelSize =(7,7)

kernel = cv2.getStructuringElement(cv2.MORPH_RECT, kernelSize)gradient = cv2.morphologyEx(gray, cv2.MORPH_GRADIENT, kernel)cv2.imshow(“Gradient “, gradient)

内核= cv2.getStructuringElement(cv2.MORPH_RECT，kernelSize)渐变= cv2.morphologyEx(灰色，cv2.MORPH_GRADIENT，内核)cv2.imshow(“渐变”，渐变)

cv2.waitKey(0)

cv2.waitKey(0)

高顶礼帽和黑帽礼帽： (Top Hat & Black Hat:)

A top hat (also known as a white hat) morphological operation is the difference between the original (grayscale/single channel) input image and the opening.

高顶礼帽 (也称为白帽 )的形态操作是原始(灰度/单通道)输入图像与开口 之间的差异 。

A top hat operation is used to reveal bright regions of an image on dark backgrounds. As seen in the Fig the results of Top Hat aren’t very existing. Notice how only regions that are light against a dark background are clearly displayed — in this case, we can clearly see that the license plate region of the car has been revealed.

使用礼帽操作可显示深色背景上图像的明亮区域 。 如图所示，Top Hat的结果并不十分完善。 请注意，只有清晰地显示了深色背景下的 浅色区域-在这种情况下，我们可以清楚地看到汽车的车牌区域已经露出。

But also note that the license plate characters themselves have not been included. This is because the license plate characters are dark against a light background. And to reveal our license plate characters we’ll need the black hat operator.

但也请注意，车牌字符本身并未包括在内。 这是因为牌照字符在浅色背景下是深色的 。 为了揭示我们的车牌字符，我们需要黑帽操作员。

The black hat operator is simply the opposite of the white hat operator! The results of black hat is a lot more promising as the license plate text itself being darker than the license plate background.

黑帽操作员与白帽操作员完全相反 ！ 由于车牌文本本身比车牌背景更暗，黑帽的结果更有希望。

Inference: APPLYING A TOP HAT OPERATION REVEALS LIGHT REGIONS ON A DARK BACKGROUND.

推论 ：在黑暗的背景下应用顶级帽子操作可显示轻区域。

APPLYING THE BLACK HAT OPERATOR REVEALS THE DARK LICENSE PLATE TEXT AGAINST THE LIGHT LICENSE PLATE BACKGROUND.

应用黑帽操作员可将黑暗许可板文本显示为反对轻许可板背景。

#Top Hat and Black Hat performed on Car number plate

#Top Hat和Black Hat在车号牌上执行

import cv2import argparseimport numpy as np

导入cv2import argparseimport numpy as np

apr = argparse.ArgumentParser()apr.add_argument(“-i”, “ — image”, required=True, help=”Path to the image”)args = vars(apr.parse_args())

apr = argparse.ArgumentParser()apr.add_argument(“-i”，“ —图片”，required = True，help =“图片路径”)args = vars(apr.parse_args())

image = cv2.imread(args[“image”])

图片= cv2.imread(args [“ image”])

print(f’(Height,Width,Depth) of the image is: {image.shape}’)

print(图像的f'(高度，宽度，深度)为：{image.shape}')

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)cv2.imshow(“Original”, image)

灰色= cv2.cvtColor(图片，cv2.COLOR_BGR2GRAY)cv2.imshow(“原始图片”，图片)

# tophat (also called a “whitehat”) operation will enable s to find light regions on a dark backgroundtophat = cv2.morphologyEx(gray, cv2.MORPH_TOPHAT, rectKernel)

＃顶帽(也称为“白帽”)操作将使s到找到一个黑暗的背景上突= cv2.morphologyEx光区域(灰色，cv2.MORPH_TOPHAT，rectKernel)

rectKernel = cv2.getStructuringElement(cv2.MORPH_RECT, (13, 5))blackhat = cv2.morphologyEx(gray, cv2.MORPH_BLACKHAT, rectKernel) # show the output imagescv2.imshow(“Original”, image)cv2.imshow(“Blackhat”, blackhat)cv2.imshow(“Tophat”, tophat)

rectKernel = cv2.getStructuringElement(cv2.MORPH_RECT，(13，5) ) Blackhat”，blackhat)cv2.imshow(“ Tophat”，tophat)

cv2.waitKey(0)

cv2.waitKey(0)

练习： (Exercise :)

Objective: The objective of this exercise is to extract the tabular region of an invoice document using Morphological operations.

目的 ：本练习的目的是使用形态学操作提取发票文件的表格区域。

The invoice document looks like this:

发票文件如下所示：

# Solution

＃解决方案

# import the necessary packagesimport argparseimport cv2import numpy as np # construct the argument parser and parse the argumentsap = argparse.ArgumentParser()ap.add_argument(“-i”, “ — image”, required=True, help=”Path to the image”)args = vars(ap.parse_args()) # load the image and convert it to grayscaleimage = cv2.imread(args[“image”])gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)cv2.imshow(“Original”, image)

＃导入必要的软件包import argparseimport cv2import numpy as np＃构造参数解析器并解析参数ap = argparse.ArgumentParser()ap.add_argument(“-i”，“ — image”，required = True，help =”路径到图像”)args = vars(ap.parse_args())＃加载图像并将其转换为灰度图像= cv2.imread(args [“ image”])gray = cv2.cvtColor(image，cv2.COLOR_BGR2GRAY)cv2。 imshow(“原始图片”)

kernel = np.ones((3,3),np.uint8)dilation = cv2.dilate(~gray.copy(),kernel,iterations = 1)h_kernel = np.ones((100,1),np.uint8)h_lines = cv2.morphologyEx(dilation, cv2.MORPH_OPEN, h_kernel)v_kernel = np.ones((1,100),np.uint8)v_lines = cv2.morphologyEx(dilation, cv2.MORPH_OPEN, v_kernel)line_img = h_lines + v_linescv2.imshow(“component’, line_img)cv2.waitKey(0)

内核= np.ones((3,3)，np.uint8)膨胀= cv2.dilate(〜gray.copy()，内核，迭代= 1)h_kernel = np.ones((100,1)，np.uint8 )h_lines = cv2.morphologyEx(膨胀，cv2.MORPH_OPEN，h_kernel)v_kernel = np.ones(((1,100)，np.uint8)v_lines = cv2.morphologyEx(膨胀，cv2.MORPH_OPEN，v_cnelv)line_img = h_lines + v_lines。 (“ component”，line_img)cv2.waitKey(0)

Result:

结果：

Here in the above code, the first 11 lines of code are familiar to us. We are loading and converting the image to a gray scale image.

在上面的代码中，我们很熟悉前11行代码。 我们正在加载图像并将其转换为灰度图像。

The next two lines are codes are used to dilate the objects in the image. This is done to ensure that there are no broken parts in the objects across the image.

接下来的两行代码用于扩展图像中的对象。 这样做是为了确保整个图像的对象中没有损坏的部分。

Next 4 lines of codes are used to extract all the horizontal lines in the image and all the vertical lines in the image using opening operation with horizontal and vertical kernels respectively. After extracting the horizontal and vertical lines in the image, we connect the lines at the point of intersections and extract the connected components using line_img = h_lines + v_lines resulting in the tabular structure.

接下来的4行代码分别使用水平和垂直核的打开操作来提取图像中的所有水平线和图像中的所有垂直线。 提取图像中的水平和垂直线后，我们在相交点连接这些线，并使用line_img = h_lines + v_lines提取连接的分量，从而形成表格结构。

However, the line kernels — kernel = np.ones((100,1),np.uint8) that we pass on to the horizontal line detector and vertical line detector is what that makes difference in detecting the lines.

但是，传递给水平线检测器和垂直线检测器的线核— kernel = np.ones((100,1)，np.uint8)才是检测线的不同之处。

kernel = np.ones((100,1),np.uint8) — Horizontal line kernel

kernel = np.ones(((100,1)，np.uint8)—水平线内核

kernel = np.ones((1,100),np.uint8) — Vertical line kernel

kernel = np.ones(((1,100)，np.uint8)—垂直线内核

To read the other Lessons from this course, Jump to this article to find the complete syllabus and table of content

要阅读本课程的其他课程，请跳转至本文以找到完整的课程提纲和目录

— — — — — — — — — — -> Click Here

— — — — — — — — — — — —> 单击此处