facebook最近开源的python图像增强库

序言

最近在在github上看到一个很火的仓库：Augly，截止目前已经获得了3600+的标星。

Augly是由facebook ai开源的一款用于数据增强的python库，它支持音频、图像、视频、文本4种形式的数据，提供了超过100种数据增强功能，可以直接对输入的内容进行多种处理。对于图像而言，提供了裁剪、选择图像、图像加文字、增加饱和度、亮度调整等100多种数据增强的方式，pip安装，开箱即用，很方便我们用来对图像进行处理。可以说功能非常的强大了，不仅可以用来离线数据增强，还可以在torch的训练中进行在线的数据增强，具体怎么使用，可以参考官方给出的例子。

与常用的图像增强库相比，增加了许多更实用的功能，所以看到介绍后我就已经跃跃欲试，并且写一篇文章记录一下一些常用的API使用的过程，方便以后查阅。

一、Augly安装

直接使用pip命令安装即可：

pip install augly

如果运行以上指令还是不能够正常使用的话，介绍中有Installation说明，可能还需要安装：

sudo apt-get install python3-magic

然后找来一张用于测试的图片：

augly使用起来也比较简单，可以看一下官方给出的例子：

import augly.image as imaugsimage_path = "your_img_path.png"
output_path = "your_output_path.png"# Augmentation functions can accept image paths as input and
# always return the resulting augmented PIL Image
aug_image = imaugs.pixelization(image_path,ratio=0.8)# Augmentation functions can also accept PIL Images as input
aug_image = imaugs.pad_square(aug_image)# If an output path is specified, the image will also be saved to a file
aug_image = imaugs.overlay_onto_screenshot(aug_image, output_path=output_path)

三个例子翻译过来的意思大概是：augly可以接受PIL格式的图片对象输入，也可以接受文件路径形式的输入，在调用时还可以指定输出路径进行保存。

好了，调用方式就是这么简单，那我们开始编写第一个例子：

import augly.image as imaugs
import PIL.Image as Image
import timeimage_path = "/home/cai/0.jpg"                             # 输入图片路径
output_path = "/home/cai/0_augly.jpg"             # 输出图片路径time1 = time.time()aug_image = Image.open(image_path)                                # 这里以PIL对象形式传入augly函数中，也可以直接将输入图片路径直接传入
aug_image = imaugs.blur(aug_image，radius=2.0)                                 # 模糊增强time2 = time.time()
print(time2-time1)                      # 计时模块，计算一张图片处理多长时间
aug_image.show()

运行后效果对比，左边为模糊后的图片，可以通过修改参数radius的值进行调整：

具体的参数列表，可以查看源码文件AugLy/augly/image/functional.py的函数具体实现，以及该函数的相关介绍、使用，做的还是很人性化的，写的非常的详细。

具体提供了多少种数据增强功能，可以在源码的AugLy/augly/image/init.py 中查看：

__all__ = ["ApplyLambda","Blur","Brightness","ChangeAspectRatio","ColorJitter","Compose","Contrast","ConvertColor","Crop","EncodingQuality","Grayscale","HFlip","MaskedComposite","MemeFormat","OneOf","Opacity","OverlayEmoji","OverlayImage","OverlayOntoScreenshot","OverlayStripes","OverlayText","Pad","PadSquare","PerspectiveTransform","Pixelization","RandomAspectRatio","RandomBlur","RandomBrightness","RandomEmojiOverlay","RandomNoise","RandomPixelization","RandomRotation","Resize","Rotate","Saturation","Scale","Sharpen","ShufflePixels","VFlip","apply_lambda","aug_np_wrapper","blur","brightness","change_aspect_ratio","color_jitter","contrast","convert_color","crop","encoding_quality","grayscale","hflip","masked_composite","meme_format","opacity","overlay_emoji","overlay_image","overlay_onto_screenshot","overlay_stripes","overlay_text","pad","pad_square","perspective_transform","pixelization","random_noise","resize","rotate","saturation","scale","sharpen","shuffle_pixels","vflip","apply_lambda_intensity","blur_intensity","brightness_intensity","change_aspect_ratio_intensity","color_jitter_intensity","contrast_intensity","convert_color_intensity","crop_intensity","encoding_quality_intensity","grayscale_intensity","hflip_intensity","masked_composite_intensity","meme_format_intensity","opacity_intensity","overlay_emoji_intensity","overlay_image_intensity","overlay_onto_screenshot_intensity","overlay_stripes_intensity","overlay_text_intensity","pad_intensity","pad_square_intensity","perspective_transform_intensity","pixelization_intensity","random_noise_intensity","resize_intensity","rotate_intensity","saturation_intensity","scale_intensity","sharpen_intensity","shuffle_pixels_intensity","vflip_intensity",
]

然后我们主要来测试在functional.py源码中常用到的那几种。

二、常用的Augly图像增强接口

functional文件中一共有32种数据增强的功能，接下来我打算一个个的测试然后展示效果，有需要的同学可以直接通过增强后效果图对比是不是自己想要的图片，然后进行调用。统一的，左边是增强后的效果，右边是原图。

2.1 图像模糊

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.blur(aug_image,output_path=output_path)aug_image.show()

2.2 改变图像亮度

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.brightness(aug_image,factor=1.5,output_path=output_path)     # 修改factor参数调整亮度aug_image.show()

2.3 改变图像宽高比

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.change_aspect_ratio(aug_image,ratio=2,output_path=output_path)aug_image.show()

2.4 颜色晃动

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.color_jitter(aug_image,brightness_factor=1.0,contrast_factor=1.0,saturation_factor=1.5,output_path=output_path)aug_image.show()

2.5 对比度增强

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.contrast(aug_image,factor=1.5,output_path=output_path)aug_image.show()

2.6 按比例裁剪

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.crop(aug_image,x1=0.1,y1=0.1,x2=0.9,y2=0.9,output_path=output_path)aug_image.show()

2.7 改变图像质量

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.encoding_quality(aug_image,output_path=output_path)aug_image.show()

2.8 转灰度

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.grayscale(aug_image,output_path=output_path)aug_image.show()

2.9 水平翻转

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.hflip(aug_image,output_path=output_path)aug_image.show()

2.10 在图片上方创建一个区域写上文字

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.meme_format(aug_image,text="hello world",caption_height=50,output_path=output_path)aug_image.show()

2.11 改变图像的不透明度

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.opacity(aug_image,level=0.5,output_path=output_path)aug_image.show()

2.12 在图片中添加表情

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.overlay_emoji(aug_image,output_path=output_path)aug_image.show()

2.13 两幅图像重叠

import augly.image as imaugs
import PIL.Image as Imageimage_path1 = "/home/cai/0.jpg"
image_path2 = "/home/cai/1.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image1 = Image.open(image_path1)
aug_image2 = Image.open(image_path2)aug_image = imaugs.overlay_image(aug_image1,image_path2,output_path=output_path)aug_image.show()

2.14 在Instagram的主页上覆盖（看起来跟真的一样）

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.overlay_onto_screenshot(aug_image,output_path=output_path)aug_image.show()

2.15 在图像中间加入横条

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.overlay_stripes(aug_image,line_width=0.1,output_path=output_path)aug_image.show()

2.16 图片中添加文字

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.overlay_text(aug_image,output_path=output_path)aug_image.show()

2.17 图像边界填充

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.pad(aug_image,w_factor=0.25,h_factor=0.25,output_path=output_path)aug_image.show()

2.18 填充较短的边，使其变为正方形

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.pad_square(aug_image,color=(255,255,255),output_path=output_path)aug_image.show()

2.19 透视变换

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.perspective_transform(aug_image,output_path=output_path)aug_image.show()

2.20 马赛克

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.pixelization(aug_image,ratio=0.5,output_path=output_path)aug_image.show()

2.21 随机噪声

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.random_noise(aug_image,output_path=output_path)aug_image.show()

2.22 旋转图像

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.rotate(aug_image,degrees=10,output_path=output_path)aug_image.show()

2.23 改变图像饱和度

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.saturation(aug_image,factor=1.5,output_path=output_path)aug_image.show()

2.24 改变图像分辨率

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.scale(aug_image,factor=0.8,output_path=output_path)aug_image.show()

2.25 锐化

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.sharpen(aug_image,factor=1.5,output_path=output_path)aug_image.show()

2.26 随机像素比任意变化

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.shuffle_pixels(aug_image,factor=0.1,output_path=output_path)aug_image.show()

2.27 垂直翻转

import augly.image as imaugs
import PIL.Image as Imageimage_path = "/home/cai/0.jpg"
output_path = "/home/cai/0_augly.jpg"aug_image = Image.open(image_path)
aug_image = imaugs.vflip(aug_image,output_path=output_path)aug_image.show()

三、其他扩展

上面的例子可以看出，都是非常实用的增强转换，图片增强后很真实，相比于一些其他的库，没有太严重的失真。除此之外，还可以将任何功能扩展调用为Transform类，将它们组合在一起，并以给定的概率应用它们。如果有需要，你可以轻松地与PyTorch转换集成，加入到模型训练中，能够提高模型的鲁棒性：

import torchvision.transforms as transforms
import augly.image as imaugsCOLOR_JITTER_PARAMS = {"brightness_factor": 1.2,"contrast_factor": 1.2,"saturation_factor": 1.4,
}AUGMENTATIONS = [imaugs.Blur(),imaugs.ColorJitter(**COLOR_JITTER_PARAMS),imaugs.OneOf([imaugs.OverlayOntoScreenshot(), imaugs.OverlayEmoji(), imaugs.OverlayText()]),
]TRANSFORMS = imaugs.Compose(AUGMENTATIONS)
TENSOR_TRANSFORMS = transforms.Compose(AUGMENTATIONS + [transforms.ToTensor()])# aug_image is a PIL image with your augs applied!
# aug_tensor_image is a Tensor with your au如果你的图像目前是NumPy数组的形式，你不想在使用增强函数之前将图像保存为文件，你可以使用我们的NumPy包装器gs applied!
image = Image.open("your_img_path.png")
aug_image = TRANSFORMS(image)
aug_tensor_image = TENSOR_TRANSFORMS(image)

如果你的图像当前以NumPy数组的形式存在，并且你不想在使用增强函数之前将图像保存为文件，你可以使用NumPy包装器:

from augly.image import aug_np_wrapper, overlay_emojinp_image = np.zeros((300, 300))
# pass in function arguments as kwargs
np_aug_img = aug_np_wrapper(np_image, overlay_emoji, **{'opacity': 0.5, 'y_pos': 0.45})