Large scale intelligent surveillance systems used by governments and corporates have attracted a lot of bad press and public outrage over privacy and data security concerns. They often do data collection as well as data processing for extracting key insights regarding the scenes captured.

由政府和企业是，l- ARGE规模智能监控系统已经吸引了隐私和数据安全问题有很多负面的报道和公众的愤怒。 他们经常进行数据收集和数据处理，以提取有关捕获场景的关键见解。

For example, these systems are alleged to do face recognition, matching, and information extraction of an individual or a group.

例如，据称这些系统对个人或群体进行人脸识别，匹配和信息提取。

Data privacy and the ethical aspects of these surveillance systems are a whole other debate and beyond the scope of this article. What is more important is to be aware of what goes on behind building such a system.

这些监视系统的数据隐私和道德方面是另一种辩论，超出了本文的范围。 更重要的是要意识到构建这样一个系统的背后是什么。

Have you ever wondered how these systems work in real life? If yes, then I suggest you read on to find out.

您是否想过这些系统在现实生活中如何工作？ 如果是，那么我建议您继续阅读以找出答案。

I have been working on large scale machine vision systems for the past 2 years and through this post, I would like to share with you some insights I gained while working on some projects which required me to architect systems involving 200+ cameras that need to be processed in realtime using an AI backend.

在过去的两年中，我一直在从事大型机器视觉系统的研究，在这篇文章中，我想与您分享我在一些项目中获得的一些见解，这些项目需要我设计涉及200多台摄像机的系统。使用AI后端实时处理。

The methods or knowledge I share are purely based on my learning and hands-on experience that I had and should not be considered as me saying its the absolute best practice or this is how every surveillance system out there works. Rather, it is a general template that showcases the potential of such pipelines and how the data gets handle in most scenarios.

我分享的方法或知识纯粹是基于我的学习和动手经验，不应将其视为绝对的最佳实践，否则这就是每个监视系统的工作方式。 相反，它是一个通用模板，展示了此类管道的潜力以及在大多数情况下如何处理数据。

So how does it all come together? To broadly put things to perspective, you have the cameras acting as the sensory aid for raw data capture, an artificial intelligence processing unit that could either be an edge computing device or a central processing server, a command center or control dashboard, typical networking infrastructure to stream the sensory data and a database to store all the vital info.

那么，这一切如何融合在一起？ 从广义上讲，您可以使用照相机充当原始数据捕获的感官帮助，可以是边缘计算设备或中央处理服务器的人工智能处理单元，命令中心或控制仪表板，典型的网络基础结构传输感官数据和数据库以存储所有重要信息。

Let’s take a recent project I worked on as an example so that I can explain the numbers and discuss the tech more easily.

让我们以我最近从事的项目为例，以便我可以解释数字并更轻松地讨论技术。

The project was part of an automation roadmap for smart city development. A typical smart city surveillance system would contain a minimum of 300 cameras strategically positioned in key areas to monitor what is happening.

该项目是智能城市发展自动化路线图的一部分。 一个典型的智能城市监控系统将至少包含300个策略性地放置在关键区域的摄像机，以监控正在发生的事情。

These cameras generate a lot of data every second and monitoring them manually becomes a huge task. That’s where my company comes in.

这些摄像机每秒产生大量数据，手动监视它们成为一项艰巨的任务。 那就是我公司进来的地方。

We develop vision processing systems supporting multiple compute hardware configurations to process these videos in real-time.

我们开发了支持多种计算硬件配置的视觉处理系统，以实时处理这些视频。

A typical smart city AI workload will contain human detection and tracking, vehicle monitoring with tagging and license plate recognition, anomaly detection, face detection with matching, etc to name a few. But all these workloads are pretty resource-intensive as is the case with any deep learning-based vision pipelines and would shoot up the price of the overall system as well.

典型的智慧城市AI工作负载将包括人工检测和跟踪，带有标签和车牌识别的车辆监控，异常检测，带有匹配项的面部检测等。 但是，所有这些工作负载都非常耗费资源，任何基于深度学习的视觉管道都是这种情况，并且还会抬高整个系统的价格。

An easy workaround is to avoid running all these workloads on every camera. Instead, we choose which cameras are relevant for different use cases and assign these cameras to each workload.

一个简单的解决方法是避免在每台摄像机上运行所有这些工作负载。 相反，我们选择哪些摄像头与不同用例相关，并将这些摄像头分配给每个工作负载。

As an example, the cameras placed at entrances of buildings are appropriate to run a face recognition workload as the visibility of faces is best in this scenario. A camera on the side of the road is ideal for watching pedestrian movement and vehicles.

例如，由于在这种情况下人脸的可见度最高，因此放置在建筑物入口处的摄像头适合进行人脸识别工作。 路边的摄像头非常适合观看行人运动和车辆。

Hardware specification for every camera is also determined on the basis of workload it needs to handle. Now, where do you put this processing hardware? Well, you can have a variety of configurations, but the best we found was the edge computing architecture. This can be seen in the image below.

每个摄像机的硬件规格还取决于其需要处理的工作量。 现在，您将该处理硬件放在哪里？ 好了，您可以有多种配置，但是我们发现的最好的就是边缘计算架构。 可以在下图中看到。

The above diagram showcases a barebones smart city deployment which has 3 main workloads. One to detect people moving across a camera view and count the total number of people who crossed that area in either direction. Another workload which detects and matches faces and finally a vehicle monitoring and sorting workload for traffic-related info.

上图展示了具有3个主要工作负载的准系统智能城市部署。 用于检测在摄像机视图中移动的人员并计算在任一方向上越过该区域的人员总数。 另一个工作负载用于检测和匹配人脸，最后是车辆监控和分类工作负载以获取与交通相关的信息。

Edge-based processing lets us tweak the software on camera-level and helps us scale the architecture easily.

基于边缘的处理使我们可以在相机级别调整软件，并帮助我们轻松扩展体系结构。

Notice the difference in the number of cameras allotted to an edge device. This is usually based on workload limitations. For example, the face detection and the matching use case has a comparatively higher workload requirement than the vehicle monitoring workload, so the former can handle lesser cameras than the latter for real-time processing.

请注意分配给边缘设备的摄像机数量的差异。 这通常基于工作负载限制。 例如，面部检测和匹配用例具有比车辆监视工作量更高的工作量要求，因此前者可以处理比后者少的相机进行实时处理。

More workloads will be added according to the requirement from the end client and by using edge hardware, we can horizontally scale with the addition of more cameras or inclusion of new use cases.

根据最终客户端的需求，将添加更多的工作负载，并且通过使用边缘硬件，我们可以通过添加更多的摄像机或包含新的用例来横向扩展。

As mentioned before the cameras for every workload are manually identified based on lots of different parameters, the major one being the total cost of ownership of such an infra.

如前所述，每个工作量的摄像机都是根据许多不同的参数手动识别的，主要的是此类基础设施的总拥有成本。

Every workload will be managed centrally, where the extracted information is streamed as small-sized data payloads to the datacentre server where they are preprocessed, sorted, and stored to a database. The stored data can be used by custom analytics engines to generate infographics like heatmaps, crowd analytics, flagging alerts based on facial matching, etc. This rule engine or analytics engine can be modified without risking downtime of any edge hardware, thus ensuring continuous operation.

将对每个工作负载进行集中管理，在此将提取的信息作为小型数据有效负载流式传输到数据中心服务器，在此对其进行预处理，分类并存储到数据库中。 定制的分析引擎可以使用存储的数据来生成信息图，例如热图，人群分析，基于面部匹配标记警报等。可以修改此规则引擎或分析引擎，而不会冒任何边缘硬件停机的风险，从而确保连续运行。

The edge hardware is a key piece of the puzzle. While we work with multiple computing modules, the best choice for large scale deployment like the smart city use case, we found that Intel or NVIDIA platforms are the best.

边缘硬件是难题的关键。 当我们使用多个计算模块时，这是大规模部署(如智能城市用例)的最佳选择，我们发现Intel或NVIDIA平台是最好的。

We have closely worked with both companies on multiple projects and have found that hardware from both vendors complement each other very well and gives good flexibility.

我们与两家公司在多个项目上密切合作，发现两家供应商的硬件可以很好地互补，并且具有良好的灵活性。

So to conclude, any typical large scale vision systems are an amalgamation of a lot of different technologies and AI is only a small part of the system. I hope this gives you a baseline understanding of an AI-powered surveillance system. Watch this space for more articles in the coming weeks, where I will be discussing topics related to AI development, architecture design, and software stacks for large scale vision systems.

综上所述，任何典型的大型视觉系统都是许多不同技术的结合，而AI只是系统的一小部分。 我希望这能使您对基于AI的监视系统有一个基本的了解。 在接下来的几周中，请观看此空间以获取更多文章，我将在其中讨论与AI开发，体系结构设计和大型视觉系统的软件堆栈相关的主题。