创建外部快照

by Arjun Krishna Babu

通过Arjun Krishna Babu

快照事件：现在如何仅通过拍照即可创建日历事件 (Snap Event: How you can now create calendar events just by taking a picture)

Google just published my first Android app, Snap Event, in their Play Store. Snap Event creates calendar events from photographs of the event’s poster.

Google刚刚在他们的Play商店中发布了我的第一个Android应用Snap Event 。 Snap Event根据事件海报的照片创建日历事件。

The app is by no means perfect, but it’s functional and is a proof of concept that the idea works.

该应用程序绝不是完美的，但是它是功能性的，并且是该想法行之有效的概念证明。

This article is a write-up of how I built and published this app.

本文是我如何构建和发布此应用程序的文章。

动机 (Motivation)

People say the best way to learn a new programming language or framework is to do a project with it. Sure, you need some familiarity with the basics of whatever it is that you’re trying to learn. But past that stage, projects are the way to go.

人们说学习一种新的编程语言或框架的最好方法是用它来做一个项目。 当然，您需要熟悉所尝试学习的基础知识。 但是经过那个阶段，项目才是路要走。

There were three motivating factors behind the development of this app:

此应用程序的开发背后有三个激励因素：

1. My previous attempt at learning Android two years ago had failed. I wanted to give it another shot.我两年前以前尝试学习Android的尝试失败了。 我想再试一试。
2. I already had a project idea in mind.我已经有了一个项目构想。
3. One of my professors asked us each to build an Android app. We’d get credit for it.我的一位教授要求我们每个人都构建一个Android应用程序。 我们会为此而获得荣誉。

Whenever I saw a poster for some event, such as a concert or conference, I took its picture to keep track of what exactly was on the poster. Most people I knew did this. I searched, but I couldn’t find an app that converts images of posters into calendar events. This was surprising. Or maybe I did not search well enough.

每当我看到某事件的海报时，例如音乐会或会议，我都会拍张照片来跟踪海报上的确切内容。 我认识的大多数人都这样做。 我进行了搜索，但找不到将海报图像转换为日历事件的应用程序。 这真是令人惊讶。 也许我的搜索不够好。

So my friend Alexander “Alex” Kaberlein and I decided to build this app.

因此，我和我的朋友Alexander“ Alex” Kaberlein决定开发此应用。

应用程式开发 (App Development)

Creating calendar events from their posters comes down to three things:

从海报创建日历事件可以归结为三件事：

1. Detect text from the image.从图像中检测文本。
2. Make sense of the detected text.使检测到的文本有意义。
3. Create calendar event.创建日历事件。

Our focus was to ship a working app as early as possible, and then iron out its shortcomings. As a consequence, we made some compromises during the design phase. This was supposed to be a learning experience, after all. We have zero intentions of making money from this app.

我们的重点是尽早发布可运行的应用程序，然后消除其缺点。 结果，我们在设计阶段做出了一些妥协。 毕竟，这应该是一次学习经历。 我们从这个应用程序赚钱的意图零。

从图像中检测文本 (Detecting Text From The Image)

Though in retrospect step two was harder, our project could’ve been shelved right away if we had failed to detect text from the image. I had no knowledge whatsoever about computer vision. Alex had some familiarity with OpenCV, but we did not have enough time to come up with our own image recognition models.

尽管回想起来第二步要难一些，但是如果我们未能从图像中检测到文本，我们的项目可能会立即被搁置。 我对计算机视觉一无所知。 Alex对OpenCV有一定的了解，但是我们没有足够的时间来提出自己的图像识别模型。

Based on what I read on the internet, I was aware of certain off-the-shelf libraries and services for image recognition. In particular, the three services we considered were:

根据我在互联网上阅读的内容，我了解到某些现成的图像识别库和服务。 特别是，我们考虑的三项服务是：

1. Amazon Rekognition

   亚马逊认可

2. Google Cloud Vision

   Google Cloud Vision

3. Google Mobile Vision

   Google移动视觉

It didn’t take long for us to zero in on Google Mobile Vision:

我们花了很长时间将Google Mobile Vision归零：

• Amazon Rekognition was supposed to be pretty good at detecting objects within the image. But we did not know how well it worked with text.亚马逊Rekognition应该能够很好地检测图像中的物体。 但是我们不知道它与文本的配合情况。
• Cloud Vision was capable of detecting text, but it’s not free (despite being inexpensive).Cloud Vision能够检测文本，但是它不是免费的(尽管价格便宜)。
• Mobile Vision was capable of detecting text, free, and likely worked well with Android.Mobile Vision能够免费检测文本，并且可能与Android兼容。

So we chose Mobile Vision.

因此，我们选择了Mobile Vision。

Next up, we wanted to be sure that Mobile Vision detected text to our satisfaction. We didn’t want to discover at the last moment if Mobile Vision wasn’t up to the task. For this, we built a small toy-app to detect text from images we hard-coded into the app. You can find it here.

接下来，我们希望确保Mobile Vision能够检测到令人满意的文本。 我们不想在最后一刻发现Mobile Vision是否无法完成任务。 为此，我们构建了一个小型玩具应用程序，以从我们硬编码到应用程序的图像中检测文本。 你可以在这里找到它。

The prototype worked well, and we decided to proceed with the main app.

原型运行良好，我们决定继续使用主应用程序。

Capturing the image and saving it on the phone was easier than I thought. However, I wanted to display all the images taken by the app in vertically scrolling cards. That’s where I ran into my first major set of problems:

捕获图像并将其保存在手机上比我想象的要容易。 但是，我想在垂直滚动卡中显示该应用程序拍摄的所有图像。 那是我遇到的第一个主要问题集的地方：

• The high-resolution images take up a lot of memory, causing the scrolls to jitter. This was despite using RecyclerView.

  高分辨率图像占用大量内存，从而导致滚动抖动。 尽管使用了RecyclerView 。

• Centering images within the cards was harder than I thought. You have to fiddle around with a Bitmap object and make non-trivial mathematical calculations.

  在卡中居中放置图像比我想象的要难。 您必须摆弄Bitmap对象并进行非平凡的数学计算。

The Android documentation (precisely this page) pointed me to solutions for both these problems. There is a library called Glide which handles the complexities associated with pulling multiple images into your app. It also handles annoyances like centering your images properly into your ImageView.

Android文档(恰好是此页面)为我指出了这两个问题的解决方案。 有一个名为Glide的库，用于处理与将多个图像拖入您的应用程序相关的复杂性。 它还可以处理将图像正确居中到ImageView烦恼。

Documentation for Glide could have been better. I had to resort to 3rd party websites to figure out certain use-cases. Speaking of which, you might want to read this excellent introduction to Glide.

Glide的文档本来可以更好。 我不得不求助于第3方网站来确定某些用例。 说到这，您可能想要阅读有关Glide的出色介绍。

使用Mobile Vision进行文本识别 (Text Recognition using Mobile Vision)

Mobile Vision supports:

Mobile Vision支持：

1. Face detection

   人脸检测

2. Barcode detection

   条码检测

3. Text detection, which is what we wanted.

   文本检测 ，这是我们想要的。

The library is capable of providing us text in whichever format we desire. It can be entire blocks of text, lines, words, and so on. Our plan of attack was to extract every single line of text in the poster, and then figure out what each of those lines meant.

该库能够为我们提供所需格式的文本。 它可以是文本，行，单词等的整个块。 我们的攻击计划是提取海报中每行文本，然后弄清楚每行的含义。

There were a couple of things I noticed about text detection using Mobile Vision:

我注意到有关使用Mobile Vision进行文本检测的几件事：

• It does not work well with handwritten text.它不适用于手写文本。
• Extracted lines of text are not always in the top-to-bottom order.提取的文本行并不总是从上到下的顺序。
• Detected text sometimes differs for the same image on multiple runs — an unexpected non-deterministic behavior.对于同一图像，多次运行时检测到的文本有时会有所不同-意外的不确定性行为。

• It occasionally crashes when using the .jpg format for the images. I resorted to using .png formats.

  使用.jpg格式的图像时，有时会崩溃。 我求助于使用.png格式。

• Use the ARGV_8888 methodology for storing pixels in memory. It took me hours to figure out why my app kept crashing. Part of my app was using RGB_565 by default, and another part was expecting images in ARGV_8888. See this page to know more about what those configurations mean.

  使用ARGV_8888方法将像素存储在内存中。 我花了几个小时才弄清楚为什么我的应用程序持续崩溃。 我的应用程序的一部分默认情况下使用RGB_565 ，而另一部分则期待ARGV_8888图像。 请参阅此页面以了解有关那些配置的含义的更多信息。

理解文本 (Making Sense of the Text)

Alex and I concluded that for any calendar event, the three crucial pieces of information are:

我和亚历克斯得出的结论是，对于任何日历事件，三个关键信息是：

• Schedule时间表
• Title标题
• Location位置

The problem with dates is that they are written in different ways in different parts of the world. For instance, “05–07–2017” is May 7th in the United States and July 5th in almost every other country. Weird ideas involving tricks with the geographic location of the user crossed my mind, but it didn’t feel right. Besides, in countries like the United States where people from all over the world are present, this idea is not foolproof.

日期的问题在于在世界的不同地方以不同的方式编写日期。 例如，“ 05-07-2017”在美国是5月7日，在几乎所有其他国家是7月5日。 涉及到与用户地理位置有关的技巧的怪异想法在我脑海中浮现，但感觉并不正确。 此外，在像美国这样的世界各地都有人们聚集的国家，这种想法并非万无一失。

Due to these complications, we decided to infer the month only when it is written out in full, such as “January” (or its abbreviated form “Jan.”). Though not always true, it is somewhat safe to assume that the date and the year would be on the same line as the month.

由于这些复杂性，我们决定仅在完整写出月份(例如“ January”(或缩写为“ Jan.”))时推断月份。 尽管并非总是如此，但可以肯定地说日期和年份与月份在同一行。

Our methodology for detecting the event title and location is so terrible that I’d rather not talk about it. For a given a line of text, I have no reliable strategy to conclusively determine if it is an event title, location, or some other information.

我们检测事件标题和位置的方法如此糟糕，以至于我不想谈论它。 对于给定的一行文本，我没有可靠的策略来最终确定它是事件标题，位置还是其他信息。

To compensate for this, we dump every single line of text we detect into the description field of the calendar event.

为了弥补这一点，我们将检测到的每一行文本都转储到日历事件的描述字段中。

创建日历事件 (Creating the Calendar Event)

A trivial task in the grand scheme of things.

宏伟的事物中的琐碎任务。

Android stores all calendar events to a central repository called the Calendar Provider. This is why calendar events you create using any calendar app show up on any other calendar app you install. Think of it this way — all media files in your phone would show up on whatever media player you installed.

Android将所有日历事件存储到名为Calendar Provider的中央存储库中。 这就是您使用任何日历应用程序创建的日历事件会显示在您安装的任何其他日历应用程序上的原因。 以这种方式思考-手机中的所有媒体文件都会显示在您安装的任何媒体播放器上。

Once all the information needed to create a calendar event is ready, it’s just a matter of starting an activity to create the calendar event.

准备好创建日历事件所需的所有信息后，只需启动一个活动来创建日历事件即可。

We chose to open the calendar app populated with the event information to give users an opportunity to review the event before it is saved. Otherwise, it would silently create the calendar event in the background.

我们选择打开填充有事件信息的日历应用程序，以便用户有机会在保存事件之前对其进行查看。 否则，它将在后台静默创建日历事件。

在Google Play商店上发布 (Publishing on Google Play Store)

Like most other things by Google, the steps for publishing an Android app are well-documented. Once I felt ready to launch the app, I followed the official launch checklist.

与Google的大多数其他事情一样，发布Android应用程序的步骤也有据可查。 一旦我准备好启动该应用程序，就遵循官方的启动清单。

It wasn’t a quick process, though. It took them over 24 hours to verify my one-time registration fee of $25, and 6 more hours for my app to show up on the Google Play store after I clicked the button to rollout and publish the app.

但是，这不是一个快速的过程。 他们花了超过24小时的时间才能确认我的一次性注册费25美元，并且在我单击按钮进行展示和发布该应用后，我的应用又在Google Play商店中展示了6个多小时。

我从开发此应用程序中学到的东西 (What I Learned From Developing This App)

• Basic Android programming. I’m now confident about reading through the documentation (as well as other resources) to figure out how to get things done. Now I can create more complex Android apps in the future.基本的Android编程。 我现在有信心阅读文档(以及其他资源)以弄清楚如何完成任务。 现在，我可以在将来创建更复杂的Android应用程序。
• How to actually publish the Android app.如何实际发布Android应用程序。
• How to use the Glide library to display multiple images efficiently.如何使用Glide库有效显示多个图像。
• How to set up and use Google’s Mobile Vision library.如何设置和使用Google的Mobile Vision库。

缺点 (Shortcomings)

I’ve already covered many of the shortcomings of the app in the App Development section above. But I’ll summarize the issues here:

我已经在上面的“应用程序开发”部分中介绍了该应用程序的许多缺点。 但我将在这里总结这些问题：

• Date detection happens only if the date is written out in full.

  仅当日期被完整写出时，才会进行日期检测。

  In the real world, dates are not always written out in full. This severely limits the usability of the app.

  在现实世界中，日期不一定总是完整写出。 这严重限制了该应用程序的可用性。

• Date detection fails if it contains ordinal suffixes like “st,” “nd,” “rd,” or “th.” However, this can be quickly fixed.如果日期检测包含序数后缀(如“ st”，“ nd”，“ rd”或“ th”)，则日期检测将失败。 但是，可以快速解决此问题。
• Event title and location detection are not perfect. I’m contemplating using machine learning models to accomplish this task, but it’s still quite far away.事件标题和位置检测并不完美。 我正在考虑使用机器学习模型来完成此任务，但是它仍然很遥远。
• The images are saved onto the phone using a terrible strategy that I’m ashamed to mention here. Even though it works perfectly. This would be evident only if you read through the source-code.这些图片会以一种可怕的策略保存到手机中，对此我感到here愧。 即使效果完美。 仅当您阅读源代码时，这才显而易见。
• User experience can be improved.可以改善用户体验。

接下来是什么 (What Next)

I’ll fix the relatively easy things I mentioned in the shortcomings section. But I likely won’t pursue this project too much further due to reasons I mention next.

我将修复我在缺点部分中提到的相对简单的问题。 但是由于接下来要提及的原因，我可能不会对这个项目进行过多的研究。

谷歌镜头 (Google Lens)

Among the host of exciting things Google announced during Google I/O 2017 is Google Lens. Google Lens does exactly what our app does (along with a bunch of other cool things). And Google Lens does it rather well, much better than what I have created through Snap Event.

在Google I / O 2017期间Google宣布的令人兴奋的事情之一是Google Lens。 Google Lens确实完成了我们的应用程序的工作(以及许多其他很酷的事情)。 而且Google Lens的效果相当好，比我通过Snap Event创建的要好得多。

And I’m happy and excited about the fact that Google did it. At least I expect it to be a lot more reliable and useful.

我为Google做到这一点感到高兴和兴奋。 至少我希望它更加可靠和有用。

获取快照事件 (Get Snap Event)

• As mentioned, Snap Event is now available on Google Play.

  如前所述， Snap Event现在可在Google Play上使用。

• The source-code of the app is available on GitHub.

  该应用程序的源代码可在GitHub上找到。

You can read about more of my projects on my blog.

您可以在我的博客上阅读有关我的更多项目的信息 。

翻译自: https://www.freecodecamp.org/news/snap-events-how-you-can-now-create-calendar-events-just-by-taking-a-picture-af21f3bfaeef/

创建外部快照