增强现实采用什么相机

by Avi Ashkenazi

通过Avi Ashkenazi

从增强现实到情感检测：相机如何成为解读世界的最佳工具 (From Augmented Reality to emotion detection: how cameras became the best tool to decipher the world)

The camera is finally on stage to help solve user experience (UX) design, technology, and communication issues.

摄像机终于在舞台上，以帮助解决用户体验(UX)设计，技术和通信问题。

Years after the Kinect was trashed and Google Glass failed, there is now new hope. The impressive technological array that Apple minimized from a PrimeSense to the iPhone X is the beginning of emotion-dependent interactions.

在Kinect被丢弃和Google Glass失败数年之后，现在有了新的希望。 苹果从PrimeSense到iPhone X最小化的令人印象深刻的技术阵列是依赖于情感的交互的开始。

It’s not new. It’s commercialized and gives developers access to indispensable information.

这不是新的。 它已商业化，使开发人员可以访问必不可少的信息。

Recently, Mark Zuckerberg mentioned that much of Facebook’s focus will be on the camera and its surrounding environment. Snapchat has defined itself as a camera company. Apple and Google are also heavily investing in cameras. The camera has tremendous power that we have not yet tapped into. It has the power to detect emotions.

最近，马克·扎克伯格(Mark Zuckerberg)提到Facebook的大部分重点将放在相机及其周围环境上。 Snapchat将自己定义为一家相机公司。 苹果和谷歌也在大力投资相机。 相机具有我们尚未开发的强大功能。 它具有检测情绪的能力。

输入内容必须轻松，自然且轻松 (Inputs need to be easy, natural, and effortless)

When Facebook first introduced emojis as an enhanced reaction to Like, I realized that they were onto something. Facebook recently added five emotions which helped Facebook to better understand its users’ emotional reactions to its content. I argue that the emojis are glorified form of the same thing, but one that works better than anything else.

当Facebook首次引入表情符号作为对Like的增强React时，我意识到它们已经成为现实。 Facebook最近添加了五种情绪，这些情绪有助于Facebook更好地了解用户对其内容的情绪React。 我认为表情符号是同一事物的美化形式，但效果比其他事物更好。

In the past, Facebook only had the Like button while YouTube had the Like and Dislike buttons. But these are not enough to track emotions, and do not provide much value to researchers and advertisers. Most people express their emotions in comments, and yet there are more Likes than comments.

过去，Facebook只有“ 赞” 当YouTube拥有“ 赞”按钮时 和“ 不喜欢”按钮。 但是这些还不足以追踪情绪，也无法为研究人员和广告商提供太多价值。 大多数人都在评论中表达自己的情感，但Like的数量多于评论。

The comments are text based or even presented with an image, which is harder to analyze. That is because there are many contextual connections the algorithm needs to guess. For example, how familiar is the person who reacts to a post with the person who posted it, and vice versa? How is the person connected to the specified subject?

评论基于文本，甚至带有图像，很难分析。 那是因为算法需要猜测许多上下文关联。 例如，对帖子做出React的人与对帖子做出React的人之间有多熟悉，反之亦然？ 该人如何与指定主题联系？

Is there subtext, slang, or anything related to the person’s experience? Is it a continued conversation from the past? Facebook did a wonderful job of keeping the conversation positive. Facebook prevented the Dislike button from pulling focus, which could have discouraged people from creating and sharing content. Facebook kept it positively pleasant.

是否有潜台词，语或与人的经历有关的任何东西？ 是过去的持续对话吗？ Facebook在保持对话积极性方面做得非常出色。 Facebook阻止了“ 不喜欢”按钮吸引焦点，这可能会阻止人们创建和共享内容。 Facebook保持它积极愉快。

Now I would compare Facebook to a glorified forum. Users can reply to comments or to the emojis. Users can Like a Like ?. Yet it is still very hard to know what people are feeling. Most people who read don’t leave a comment. What do readers feel when they read a post then?

现在，我将Facebook与一个荣耀的论坛进行比较。 用户可以回复评论或表情符号。 用户可以像一个什么样的 ？ 然而，仍然很难知道人们的感受。 大多数阅读的人不会发表评论。 那么，读者在阅读帖子时会有什么感觉？

相机的旧用户体验 (The old user experience for cameras)

What do you do with a camera? Take pictures and videos, and that’s about it. There has been huge development in camera apps. There are many features that are related to the surroundings of the main use case, for example high dynamic range (HDR), slow motion, portrait mode, etc.

你用相机做什么？ 拍摄照片和视频，就是这样。 相机应用程序已经有了巨大的发展。 有许多与主要用例环境相关的功能，例如高动态范围(HDR)，慢动作，人像模式等。

Based on the enormous number of pictures users generate, a new wave of smart galleries, photo processing, and metadata apps has been created.

基于用户生成的大量图片，新一波的智能画廊，照片处理和元数​​据应用程序已创建。

However, recently the focus has changed. It is now on the life-integrated camera, which is a combination of the strongest treats and best use cases for mobile phones. The next generation of cameras will be fully integrated into our lives and could replace all the input icons in a messaging app (microphone, camera, and location).

但是，最近的重点已经改变。 现在，它已集成在生活中的相机上，该相机结合了最坚固的款式和最佳的手机用例。 下一代相机将完全融入我们的生活中，并可以替换消息传递应用程序(麦克风，相机和位置)中的所有输入图标。

Cameras are one of the three items that has been consistently developed at a dizzying pace. The screen and the processor are the other two. Every phone that has come out has pushed the limits, and has done so year after year. The improvements made to the cameras are to their megapixels, movement stabilization, aperture, speed and, as mentioned above, the apps.

相机是一直以令人眼花consistently乱的速度不断发展的三项之一。 屏幕和处理器是另外两个。 每个问世的手机都突破了极限，并且年复一年地这样做。 相机的改进之处在于它们的百万像素，运动稳定性，光圈，速度以及如上所述的应用程序。

Let’s evaluate the evolution of a few products created by companies.

让我们评估一些公司创造的产品的发展。

Much of the development was focused on the camera at the back of the phone because, at least at first, the front camera was thought to be useful for video calls only. However, selfie culture and Snapchat changed that. Snapchat’s masks, which were later copied by everyone else, are a huge success. Face masks are not new. Google introduced them a while ago, but Snapchat was effective at growing the use of masks.

大多数开发工作都集中在手机背面的摄像头上，因为至少在一开始，人们认为前置摄像头仅对视频通话有用。 但是，自拍文化和Snapchat改变了这一点。 Snapchat的口罩，后来被其他所有人复制，取得了巨大的成功。 口罩不是新的。 谷歌不久前推出了它们，但是Snapchat在增加口罩的使用方面很有效。

内存通道的亮点 (Highlights from memory lane)

In December 2009, Google introduced Google Goggles. It was the first time that users could use their phone to get information about the environment around them. The information was mainly about landmarks initially.

Google在2009年12月推出了Google Goggles 。 这是用户第一次使用手机来获取有关周围环境的信息。 这些信息最初主要是关于地标的。

In November 2011, Samsung Nexus introduced facial recognition as a way to unlock phones. Like many things done for the first time, it wasn’t very good and was later scrapped.

2011年11月， 三星Nexus推出了面部识别技术来解锁手机。 就像第一次完成很多事情一样，它不是很好，后来被废弃了。

In February 2013, Google released Google Glass. It had more use cases because it was able to receive input not just from the camera but also from other items like the voice. The Google Glass was always there, but it failed to gain traction because it was too expensive, looked unfashionable, and triggered a backlash from the public. It was just not ready for prime time.

2013年2月，Google发布了Google Glass 。 它具有更多的用例，因为它不仅能够从摄像机接收输入，而且还可以从其他项目(如语音)接收输入。 Google Glass一直都在那儿，但由于价格太贵，看起来不时尚且引起公众的强烈反对而未能获得关注。 只是还没有准备好黄金时间。

So far, devices have only limited information at their disposal. The information they have are audio visual with GPS and historical data. But it limited Google Glass from displaying the information on the small screen that is positioned near the users’ eye. The screen blocked users from looking at anything else. Putting this technology on a phone for external use is not just a technological limitation but also a physical one.

到目前为止，设备只能使用有限的信息。 他们拥有的信息是带有GPS和历史数据的视听图像。 但是，这限制了Google Glass无法在用户眼睛附近的小屏幕上显示信息。 屏幕阻止了用户查看其他内容。 将这项技术放在手机上供外部使用不仅是一项技术限制，而且是一项物理限制。

When you focus on your phone, you cannot see anything else. Your field of view is limited. This is similar to the field of view in user experience principles for virtual reality (VR). That’s why there are cities that have created routes for people who use their phones while walking and have set up traffic lights that help people walk and text. A premise like Microsoft’s HoloLens is more aligned with the spatial environment and can actually help users while they move and use their phones, rather than absorb their attention and put them in danger.

当您专注于手机时，看不到其他任何东西。 您的视野有限。 这类似于虚拟现实(VR)用户体验原理中的视野。 这就是为什么有些城市为步行中使用手机的人创建了路线，并设置了交通信号灯来帮助人们行走和发短信。 像Microsoft的HoloLens这样的前提更加符合空间环境，实际上可以在用户移动和使用手机时为他们提供帮助，而不是吸引他们的注意力并使他们处于危险之中。

In July 2014, Amazon introduced the Fire Phone. It featured four cameras at the front of the phone. This was a breakthrough, even though it didn’t succeed. The four frontal cameras were used for scrolling and created 3D effects based on the accelerometer and users’ gaze. It was the first time that a phone used its front cameras as an input method from users.

2014年7月，亚马逊推出了Fire Phone 。 它在手机正面配有四个摄像头。 即使没有成功，这也是一个突破。 四个前置摄像头用于滚动，并根据加速度计和用户的视线创建3D效果。 这是手机第一次使用前置摄像头作为用户的输入方法。

On August 2016, Samsung’s Note 7 was launched. It allowed users to unlock their phones with iris scanning. Samsung resurrected a facial-recognition technology that had rested on the shelf for six years. Unfortunately, just looking at the tutorial can be vexing. Samsung didn’t do much user experience testing for that feature.

2016年8月， 三星推出了Note 7 。 它允许用户使用虹膜扫描解锁手机。 三星恢复了已经搁置了六年的面部识别技术。 不幸的是，仅仅看教程会很烦人。 三星并未对该功能进行太多的用户体验测试。

It is disturbing to hold this huge phone and put it at a 90° angle to your face. It is not something that anyone should be doing while they are walking on the street. It can work nicely for Saudi women who cover their faces, but because of manufacturing defects, many Note 7 phones overheated, combusted, or exploded. The concept of iris scanning was put on hold for another full year until the Note 8 came out.

拿起这部巨大的手机并将其与您的脸成90°角令人不安。 这不是任何人在街上行走时都应该做的事情。 它可以很好地遮盖住脸部的沙特女性，但由于制造缺陷，许多Note 7手机会过热，燃烧或爆炸。 虹膜扫描的概念又搁置了整整一年，直到Note 8出现。

But by that time no one mentioned iris scanning. The Note 8 mentioned that iris scanning is an another way of unlocking a phone in conjunction with the fingerprint sensor. It’s probably because the phone was not good enough or Samsung wasn’t able to make a decision (similar to the release of the Galaxy 6 and 6 Edge). For a product to succeed, it needs to have multiple functions, otherwise it risks being forgotten.

但是到那时，还没有人提到虹膜扫描。 Note 8提到虹膜扫描是与指纹传感器一起解锁手机的另一种方法。 可能是因为手机不够好，或者三星无法做出决定(类似于Galaxy 6和6 Edge的发布 )。 为了使产品成功，它必须具有多种功能，否则有被遗忘的风险。

Google took a break, and then in July 2017 released the second version of Google Glass as a business-to-business product. The use cases became more specific for some industries.

谷歌休息了一下，然后在2017年7月发布了第二版的谷歌眼镜作为企业对企业的产品。 用例对于某些行业变得更加具体。

Now Google is about to release the Google Lens to bring the initial Goggles use case to the present. It’s Google’s effort to learn how to use visual information with additional context, and to figure out what product to develop next. It appears that Google is leaning towards a camera that users can wear.

现在，Google即将发布Google Lens，以将最初的Goggles用例带到现在。 Google的工作是学习如何在其他上下文中使用视觉信息，并弄清楚下一步要开发什么产品。 Google似乎倾向于用户可以佩戴的相机。

There are other companies that are exploring visual input as well. For example, Pinterest is seeing a huge demand for its visual search lens, which its users are using to search for items to buy and to help people curate products and services online.

还有其他公司也在探索视觉输入。 例如， Pinterest对其视觉搜索镜头有巨大的需求，其用户正在使用该搜索镜头搜索要购买的商品并帮助人们在线策划产品和服务。

Snapchat’s spectacles allow users to record short videos easily (even though the upload process is cumbersome).

Snapchat的眼镜使用户可以轻松录制短视频(即使上传过程很繁琐)。

Now facial recognition is also on the Note 8 and Galaxy 8, but this feature is not panning out as well as hoped.

现在，Note 8和Galaxy 8上也采用了面部识别，但是这一功能并没有达到人们期望的水平。

To check out a facial recognition demonstration, click here.

要查看面部识别演示，请单击此处 。

Apple is slow to adopt new technologies relative to its competitors. But on the other hand, Apple commercializes its products, for example, the Apple Watch. The Watch was all about facial recognition and finite screen. There is no better way to make people use this feature than by removing all other options (like the Touch ID). It’s not surprising that Apple did this last year with wireless audio (Apple removed the headphone jack) and USB C on the MacBook Pro (by removing everything else).

与竞争对手相比，苹果在采用新技术方面进展缓慢。 但另一方面，苹果公司将其产品商业化，例如Apple Watch 。 手表全部关于面部识别和有限屏幕。 没有比删除所有其他选项(例如Touch ID )更好的方法来使人们使用此功能。 Apple去年在无线音频(Apple删除了耳机插Kong)和MacBook Pro上的USB C (通过删除所有其他配件)上做到了这一点也就不足为奇了。

There is a much bigger reason why Apple chose this technology at this time. It has to do with its augmented reality efforts.

苹果此时选择此技术的原因更大。 它与增强现实的努力有关。

Face ID has challenges that include recognizing users who wear the Niqāb (face covers), users who have had plastic surgery, and users who are changing physically because they are growing. But the bigger picture here is much more interesting. This is the first time that users can do something that they naturally do with no effort, while receiving data that is meaningful for the future of technology. I believe that a screen that can read fingerprints is better. It appears that Samsung is heading in that direction (although rumours has it that Apple tried and failed).

面部识别ID面临的挑战包括识别佩戴Niqāb (面部遮罩 )的用户，进行过整形手术的用户以及由于成长而身体发生变化的用户。 但是，这里的大局更加有趣。 这是用户首次毫不费力地执行他们自然可以做的事情，同时又接收到对未来技术有意义的数据。 我相信可以读取指纹的屏幕会更好。 三星似乎正朝着这个方向发展(尽管有传言称苹果曾尝试并失败了)。

那么这要去哪里呢？ 目标是什么？ (So where is this going? What’s the target?)

In the past, companies used special glasses and devices to perform user testing. The only output they could give were heat maps. Yet they weren’t able to document what the users’ were focusing on or their emotions and reactions.

过去，公司使用特殊的眼镜和设备进行用户测试。 他们唯一能提供的输出就是热图 。 但是他们无法记录用户关注的重点或他们的情绪和React。

Based on tech trends, it appears the future involves augmented reality and virtual reality. But in my opinion, it includes audio, 3D sound, and visual inputs combined. This would be a wonderful experience, which would allow users to look at anything, anywhere, and get information at the same time.

基于技术趋势，未来似乎涉及增强现实和虚拟现实。 但我认为，它包括音频，3D声音和视觉输入的组合。 这将是一次奇妙的体验，它将使用户可以在任何地方查看任何内容，并同时获取信息。

What if we are able to know where the users are looking at, and what they are focusing on? For years this is something that marketing and design professionals have tried to capture and analyze. What can do that better than the set of arrays a device like the iPhone X has as a starting point? Later on this evolved into glasses that can see what the user is focused on.

如果我们能够知道用户在看什么，他们在关注什么呢？ 多年来，市场营销和设计专业人员一直试图捕获和分析这些东西。 有什么比像iPhone X这样的设备作为起点更好地完成一组阵列的呢？ 后来演变成可以看到用户关注的眼镜。

React强烈且令人上瘾 (Reactions are powerful and addictive)

Reactions help people converse and increase retention and engagement. Some apps offer reactions to posts as messages that can be sent to friends. There are funny videos on YouTube that show the reactions of people who watch videos. There is even a TV show, called Gogglebox, that is dedicated to showing people watching TV.

React有助于人们交流，并增加保持力和参与度。 一些应用程序将帖子的响应作为消息发送给朋友。 YouTube上有一些有趣的视频，显示观看视频的人的React。 甚至还有一个电视节目，称为Gogglebox ，专门向人们展示看电视。

In IO Google, the annual developer festival, Google opened the option to pay creators for their platform. Its like what the brilliant Patreon site is doing but in a much more dominant way. A way that helps you to stand out from the crowd and grab the creator’s attention is SuperChat.

在一年一度的开发人员节Google IO中 ，Google公开了为平台的创建者付费的选项。 就像辉煌的Patreon网站正在做的事情一样，但是方式更占优势。 SuperChat是帮助您脱颖而出并吸引创作者注意的一种方法 。

In Chris Harrison’s student project from 2009, Harrison created a keyboard that had pressure sensing keys. Depending on the force with which users type, the keyboard reads the users’ emotions and determine if they are angry or excited. The letters too get bigger as a result. Now imagine combining it with a camera that sees the users’ facial expressions while they are typing, as people tend to express their emotions while they’re typing a message.

在2009年克里斯·哈里森(Chris Harrison)的学生项目中 ，哈里森(Harrison)创建了带有压力感应键的键盘。 根据用户键入的力度，键盘会读取用户的情绪并确定他们是生气还是激动。 结果，字母也变得更大。 现在想象一下，将其与摄像头结合使用，该摄像头可以在用户键入信息时看到用户的表情，因为人们在键入消息时往往会表达自己的情感。

这样的UX看起来如何 (How would such a UX look like)

Consider the pairing of a remote and center point in virtual reality. The center is our focus, but we also have a secondary focus point, which is where the remote points are. However, this type of user experience cannot work in augmented reality. To take advantage of augmented reality, which is a new focus for Apple, the user’s focus must be known.

考虑虚拟现实中远程和中心点的配对。 中心是我们的焦点，但我们还有一个辅助焦点，这是远程焦点所在的位置。 但是，这种类型的用户体验无法在增强现实中工作。 要利用增强现实(这是Apple的新焦点)的优势，必须了解用户的关注焦点。

What started as ARKit and Google’s ARCore SDK, both will be the future of development. This is because of the amazing output and input both can get from the front and back cameras combined. This will allow for a greater focus on the input.

从ARKit和Google的ARCore SDK开始 ，两者都是未来的发展。 这是因为，惊人的输出和输入都可以从正面和背面摄像头相结合得到的。 这样可以更专注于输入。

对未来发展的更批判性看法 (A more critical view on future developments)

While Apple opened the way for facial recognition and triggered reactionary Animoji, it is going to get interesting when other organizations start implementing Face ID. Currently, it is manifested in a basic and harmless way, but the goal remains to get more information. Information that will be used to keep track of and sell products to us. It will also allow us to learn more about us and gather our emotional data.

当苹果为面部识别开辟道路并引发反动Animoji时 ，当其他组织开始实施Face ID时，它将会变得有趣。 当前，它以一种基本且无害的方式表现出来，但目标仍然是获取更多信息。 将用于跟踪和向我们销售产品的信息。 它还将使我们能够更多地了解我们并收集我们的情感数据。

It is important to say that the front camera doesn’t come alone. It’s the expected result of Apple buying PrimeSense. The array of front-facing technology includes an infrared camera, depth sensor, etc. (I think they could do well with a heat sensor too.) It’s not that someone will keep videos of our faces using the phone, but rather there will be a scraper that will document all the information about our emotions.

重要的是要说前置摄像头并不孤单。 这是苹果购买PrimeSense的预期结果。 一系列前端技术包括红外摄像头，深度传感器等(我认为它们也可以与热传感器配合得很好。)不是有人会用手机保存我们面部的视频，而是会有记录所有有关我们情绪的信息的刮板。

摘要 (Summary)

It’s exciting that augmented reality have algorithms that can read faces. There are many books that talks about how to identify facial reactions, but now it’s time for technology to do this. It will be wonderful for many reasons. For example, robots can now see how we feel and react, or with glasses, we can get more context about what we need them to do. Relating to computers, it’s better to look at a combination of elements because that helps the machine to understand you better.

令人兴奋的是，增强现实技术具有可以读取人脸的算法。 有很多书籍讨论如何识别面部React，但是现在该是技术的时候了。 由于许多原因，这将是美好的。 例如，机器人现在可以看到我们的感觉和React，或者戴着眼镜，我们可以了解我们需要他们做什么。 对于计算机，最好将元素组合在一起，因为这有助于机器更好地了解您。

The things that you can do if you have the information about what the user is focusing on are endless. It’s the dream of every person who works with technology.

如果您掌握了用户所关注的内容，那么可以做的事情就是无穷无尽的。 这是每个从事技术工作的人的梦想。

This blog post was originally showed here.

该博客文章最初在此处显示。

翻译自: https://www.freecodecamp.org/news/facial-recognition-as-aux-driver-8a49dfd477ca/

增强现实采用什么相机