Artificial intelligence (AI) and machine learning (ML) have shown significant progress in recent years, and their development has enabled a wide range of beneficial applications. As they have started penetrating into more touchy areas, such as healthcare, more concerns have arisen as to their resilience to cyber-attacks. Like any other technology, AI and ML can be used to threaten the security or to improve it with the new means. In this post, we’ll discuss both sides of ML, as a tool for malicious use and a means to fight cyber-attacks.

近年来，人工智能(AI)和机器学习(ML)取得了长足的进步，其发展已实现了广泛的有益应用。 随着他们开始渗透到诸如医疗保健之类的更敏感的领域，人们对它们对网络攻击的抵御能力也引起了更多关注。 像任何其他技术一样，AI和ML可以用来威胁安全性或通过新手段对其进行改进。 在本文中，我们将讨论ML的双方，将其作为恶意使用的工具和应对网络攻击的手段。

From a security perspective, the rise of AI and ML is altering the landscape of risks for citizens, organizations, and states. Let’s take the ability to recognize a face and to navigate through space with the help of computer vision techniques and you can create an autonomous weapon system. NLG, the machine’s ability to generate text and speech, can be used to impersonate others online, or to sway public opinion.

从安全的角度来看，人工智能和机器学习的兴起正在改变着公民，组织和国家的风险格局。 让我们借助计算机视觉技术来识别人脸并在太空中导航，您可以创建自主武器系统。 NLG是该机器生成文本和语音的功能，可用于在网上模拟其他人或影响公众舆论。

AI安全威胁 (AI Security Threats)

First of all, let’s discuss what it is possible to do with AI-based systems. All cyberattacks can be divided into the most common triad of confidentiality, availability, and integrity, intertwined to form three main directions:

首先，让我们讨论一下基于AI的系统可以做什么。 所有网络攻击可以分为机密性，可用性和完整性这三个最常见的三元组，它们相互交织在一起，形成了三个主要方向：

Espionage, which in terms of cybersecurity means gleaning insights about the system and utilizing the received information for his or her own profit or plotting more advanced attacks. In other words, a hacker can use a ML-based engine to drill down and learn more about the internals like dataset.

间谍活动(Espionage) ，就网络安全而言，意味着收集有关系统的见解，并利用接收到的信息谋取自己的利益或策划更高级的攻击。 换句话说，黑客可以使用基于ML的引擎来深入挖掘并了解有关数据集之类的内部信息的更多信息。

Sabotage with the aim to disable functionality of an AI system by flooding AI with requests, or model modification

破坏活动旨在通过向AI泛滥请求或修改模型来禁用AI系统的功能

Fraud, which in AI means misclassifying tasks, such as introducing incorrect data in the training dataset (data poisoning) or interacting with a system at learning or production stage.

欺诈 ，在AI中意味着错误分类任务，例如在训练数据集中引入不正确的数据(数据中毒)或在学习或生产阶段与系统进行交互。

ML如何被滥用以进行攻击？ (How can ML be misused to carry out attacks?)

This is the question that worries everyone: from an old lady who was told that all her banking data will be processed digitally (even though she wouldn’t use the word “AI”) to the UN officials.

这个问题让每个人都感到担忧：从一位老太太那里被告知，她的所有银行数据都将以数字方式处理(即使她不会使用“ AI”一词)，再到联合国官员。

The truth is, AI systems have inherent characteristics that foster attacks. AI systems as a part of the digital world increase anonymity and psychological distance. We may automate a lot of tasks, but it also allows actors to experience a greater degree of psychological distance from the people they impact. For example, someone who uses an autonomous weapons system to carry out an assassination avoids the need to be present at the scene and the need to look at their victim.

事实是，人工智能系统具有助长攻击的固有特性。 作为数字世界一部分的AI系统增加了匿名性和心理距离。 我们可以使很多任务自动化，但是它也允许演员与他们所影响的人体验更大的心理距离。 例如，使用自动武器系统进行暗杀的某人避免了在现场出现和看待受害者的需要。

AI algorithms are open and can be reproduced with some skills. It is difficult and costly to obtain or reproduce the hardware, such as powerful computers or drones, but everyone can gain access to software and relevant scientific findings.

AI算法是开放的，可以通过一些技巧进行复制。 获得或复制硬件(例如功能强大的计算机或无人机)既困难又昂贵，但是每个人都可以使用软件和相关的科学发现。

On top of all, AI systems themselves suffer from a number of novel unresolved vulnerabilities, such as data poisoning attacks (introducing training data that causes a learning system to make mistakes), adversarial examples (inputs designed to be misclassified by machine learning systems), and the exploitation of flaws in the design of autonomous systems’ goals . These vulnerabilities differ from traditional software vulnerabilities (e.g. buffer overflows) and require immediate action to protect AI software.

最重要的是，人工智能系统本身也遭受许多新的未解决的漏洞的困扰，例如数据中毒攻击(引入导致学习系统出错的训练数据)，对抗性示例(旨在被机器学习系统错误分类的输入)，以及对自治系统目标设计中缺陷的利用。 这些漏洞不同于传统的软件漏洞(例如，缓冲区溢出)，需要立即采取行动来保护AI软件。

Malicious use of AI can threaten security in several ways:

恶意使用AI可以通过多种方式威胁安全性：

• digital security by hacking or socially engineering victims at human or superhuman levels of performance;通过以人类或超人类的水平入侵或对受害者进行社会工程设计来实现数字安全；
• physical security by affecting our personal safety with, for example weaponized drones; and通过影响我们的人身安全(例如带武器的无人机)来实现人身安全； 和
• political security by affecting the society through privacy-eliminating surveillance, profiling, and repression, or through automated and targeted disinformation campaigns.通过消除隐私的监视，配置文件和压制，或者通过针对性的自动虚假宣传活动来影响社会，从而实现政治安全。

数字安全 (Digital security)

• Automation of social engineering attacks: NLP tools allow mimicking the writing style of the victim’s contacts, so AI systems gather online information to automatically generate personalized malicious websites/emails/links that are more likely to be clicked on.

  自动化的社会工程学攻击： NLP工具允许模仿受害者联系人的写作风格，因此AI系统收集在线信息以自动生成个性化的恶意网站/电子邮件/链接，从而更有可能被点击。

• Automation of vulnerability discovery: Historical patterns of code vulnerabilities can help speed up the discovery of new vulnerabilities.

  漏洞发现的自动化 ：代码漏洞的历史模式可以帮助加快发现新漏洞的速度。

• Sophisticated hacking: AI can be used in hacking in many ways. It can offer automatic means to improve target selection and prioritization, evade detection, and creatively respond to changes in the target’s behavior and it can imitate human-like behavior driving the target system into a less secure state

  复杂的黑客攻击 ：AI可以多种方式用于黑客攻击。 它可以提供自动手段来改善目标的选择和优先级，逃避检测并创造性地响应目标行为的变化，并且可以模仿类似于人的行为，将目标系统驱动到不太安全的状态

• Automation of service tasks in criminal cyber-offense: AI techniques can automate various tasks that form the attack pipeline, such as payment processing or dialogue with ransomware victims.

  犯罪网络攻击中服务任务的自动化：AI技术可以自动化构成攻击管道的各种任务，例如付款处理或与勒索软件受害者的对话。

• Exploiting AI used in applications, especially in information security: Data poisoning attacks are used to surreptitiously maim or create backdoors in consumer machine learning models.

  利用在应用程序中使用的AI ，尤其是在信息安全中使用的AI ：数据中毒攻击用于在消费者机器学习模型中暗中破坏或创建后门。

人身安全 (Physical security)

• Terrorist repurposing: Commercial AI systems can be reused in harmful ways, such as using drones or self-driving cars to deliver explosives and cause crashes.

  恐怖袭击的目的 ：商业AI系统可能以有害的方式重复使用，例如使用无人机或自动驾驶汽车来运送爆炸物并导致坠毁。

• Attacks removed in time and space: As a result of automated operation, physical attacks are further removed from the attacker, including in environments where traditional remote communication with the system is not possible.

  在时间和空间上消除攻击 ：由于自动化操作，物理攻击会进一步从攻击者那里消除，包括在无法与系统进行传统远程通信的环境中。

• Swarming attacks: Distributed networks of autonomous robotic systems allow monitoring large areas and executing rapid, coordinated attacks.

  群集攻击：自主机器人系统的分布式网络允许监视大范围并执行快速，协调的攻击。

• Endowing low-skill individuals with high-skill capabilities: While in the past executing attacks required skills, such as those of a sniper, AI-enabled automation of such capabilities — such as using self-aiming, long-range sniper rifles — reduces the expertise required from the attacker.

  使低技能的人拥有高技能的能力 ：过去执行攻击所需的技能(如狙击手的技能)，但启用了AI的此类功能的自动化(例如使用自动瞄准的远程狙击步枪)降低了攻击能力。攻击者需要的专业知识。

政治安全 (Political security)

• State use of automated surveillance platforms: State surveillance powers are extended by AI-driven image and audio processing that permits the collection, processing, and exploitation of intelligence information at massive scales for myriad purposes, including the suppression of debate.

  州对自动监视平台的使用 ：通过AI驱动的图像和音频处理扩展了州监视的能力，该功能允许大规模收集，处理和利用情报信息，以实现多种目的，包括抑制辩论。

• Realistic fake news: Recent developments in image generation coupled with natural language generation techniques produce highly realistic videos of state leaders seeming to make inflammatory comments they never actually made.

  现实中的虚假新闻：图像生成的最新发展以及自然语言生成技术可生成非常真实的国家领导人视频，这些国家领导人似乎在发表实际上从未发表过的煽动性评论。

• Hyper-personalised disinformation and influence campaigns: AI-enabled analysis of social networks can identify key influencers to be approached with (malicious) offers or targeted with disinformation. On a larger scale, AI can analyse the struggles of specific communities to fed them personalised messages in order to affect their voting behavior.

  超个性化的虚假信息和影响力运动 ：启用AI的社交网络分析可以识别(恶意)要约或虚假信息针对的关键影响者。 在更大范围内，人工智能可以分析特定社区的挣扎，以向其提供个性化消息，从而影响其投票行为。

• Manipulation of information availability: Media platforms’ content curation algorithms are used to drive users towards or away from certain content to manipulate their behavior. One of the examples are bot-driven large-scale denial-of-information attacks that are leveraged to swamp information channels with noise, creating an obstacle to acquiring real information.

  操纵信息可用性：媒体平台的内容策划算法用于驱动用户移入或移出某些内容来操纵其行为。 机器人驱动的大规模拒绝信息攻击就是其中一个例子，这些攻击被利用来淹没带有噪声的信息通道，从而阻碍了获取真实信息。

Though there are lots of ways for AI to breach our safety and security, the question remains if it can be used also to forecast, prevent, and mitigate the harmful effects of malicious uses.

尽管人工智能有很多方法可以破坏我们的安全和保障，但是仍然存在问题，即人工智能是否还可以用于预测，预防和减轻恶意使用的有害影响。

机器学习如何帮助我们提高应用程序和网络的安全性？ (How can ML help us to increase the security of applications and networks?)

AI offers multiple opportunities for hackers and even terrorists, but at the same time, artificial intelligence and security were — in many ways — made for each other. Modern ML techniques seem to be arriving just in time to fill in the gaps of previous rule-based data security systems. In their essence, they try to fulfill several tasks that allow improving security systems and preventing attacks:

人工智能为黑客甚至恐怖分子提供了多种机会，但与此同时，人工智能和安全性(在许多方面)是相互兼容的。 现代ML技术似乎正好及时地填补了以前基于规则的数据安全系统的空白。 从本质上讲，他们试图完成一些任务，以改进安全系统并防止攻击：

• Anomaly detection — the task that defines normal behavior falling within a certain range and identifies every other behavior as an anomaly and thereby a potential threat;

  异常检测 -定义正常行为的任务，该行为在一定范围内，并将其他所有行为识别为异常，从而潜在威胁；

• Misuse detection — an opposite task that identifies malicious behavior is identified based on training with labeled data and allows through all traffic not classified as malicious;

  滥用检测 -根据对带有标签的数据的培训来识别可识别恶意行为的相反任务，并允许所有未分类为恶意的流量通过；

• Data exploration is a technique to identify characteristics of the data, often using visual exploration which directly assists security analysts by increasing the ‘readability’ of incoming requests.

  数据探查是一种识别数据特征的技术，通常使用视觉探查，它通过增加传入请求的“可读性”来直接帮助安全分析人员。

• Risk assessment is another task that estimates the probability of a certain user’s behavior to be malicious, which can either be done by attributing an absolute risk score or classifying users based on the probability that they are bad actors.

  风险评估是另一项估计特定用户行为为恶意行为的可能性的任务，可以通过归因于绝对风险评分或根据用户是不良行为者的可能性对用户进行分类来完成。

人工智能与安全应用 (Artificial Intelligence and Security Applications)

• Defense against hackers and software failures: The software that powers our computers and smart devices is subject to error in the code, as well as security vulnerabilities that can be exploited by human hackers. Modern AI-driven systems can search out and repair these errors and vulnerabilities, as well as defend against incoming attacks. For example, AI systems can find and determine whether the bug is exploitable. If found, the bot autonomously produces a “working control flow hijack exploit string” i.e. secures vulnerabilities. On the predictive side,such projects an artificial intelligence platform called AI2 predict cyber-attacks by continuously incorporating input from human experts.

  防御黑客和软件故障：为我们的计算机和智能设备提供动力的软件易受代码错误以及容易被人类黑客利用的安全漏洞的影响。 现代AI驱动的系统可以搜索并修复这些错误和漏洞，并防御传入的攻击。 例如，人工智能系统可以找到并确定该漏洞是否可利用。 如果找到，该漫游器会自动生成“工作控制流劫持漏洞利用字符串”，即保护漏洞。 在预测方面，此类项目通过不断整合人类专家的输入来预测一个称为AI2的人工智能平台，以预测网络攻击。

• Defense against zero-day exploits: Protection against such attacks is crucial since they are rarely noticed right away. It usually takes months to discover and address these breaches, and meanwhile large amounts of sensitive data is exposed. Machine Learning protect systems against such attacks by identifying malicious behavior by identifying abnormal data movement and help spot outliers

  防御零时差攻击 ：防范此类攻击至关重要，因为很少立即注意到它们。 发现和解决这些漏洞通常需要几个月的时间，同时还会暴露大量敏感数据。 机器学习通过识别异常数据移动来识别恶意行为，从而保护系统免受此类攻击，并帮助发现异常值

• Crime prevention: Predictive analytics and other AI-powered crime analysis tools have made significant strides. Game theory, for example can be used to predict when terrorists or other threats will strike a target.

  预防犯罪：预测分析和其他基于AI的犯罪分析工具取得了长足的进步。 例如，博弈论可用于预测恐怖分子或其他威胁何时袭击目标。

• Privacy protection: Differential privacy has been written about for some years, but it’s a relatively new approach with mixed feedback as to its scalability. It offers a way to maintain private data on a network, while providing targeted “provable assurances” to the protected subpopulation and using algorithms to investigate the targeted population. This type of solution can be used in trying to find patterns or indications of terrorists in a civilian population, find infected citizens within a larger healthy population, amongst other scenarios.

  隐私保护： 差异性隐私已经存在了很多年，但是它是一种相对较新的方法，其可扩展性得到了混合反馈。 它提供了一种方法来维护网络上的私有数据，同时为受保护的亚人群提供目标“可证明的保证”，并使用算法来调查目标人群。 此类解决方案可用于尝试找到平民人口中恐怖分子的模式或迹象，在更大的健康人口中找到受感染的公民以及其他场景。

AI在保护行业和消费者方面的潜在应用 (Potential applications of AI for protection of industry and consumers)

The field of artificial intelligence is growing constantly, embracing new techniques and creating new systems that could not be even imagined a decade ago.

人工智能领域不断发展，采用了新技术并创建了十年前甚至无法想象的新系统。

An example of such development is IoT-based security: The Internet of Things (IoT) is enabling cost-efficient implementation of condition-based maintenance for a number of complex assets, with ML playing a driving role in the analysis of incoming data. With the resources that IoT provides, the process of anomaly detection and, therefore, failure and crime prevention will become significantly more effective and rapid.

这种发展的一个例子是基于IoT的安全性：物联网(IoT)使许多复杂资产能够经济高效地实施基于状态的维护 ，而ML在输入数据的分析中起着推动作用。 借助物联网提供的资源，异常检测以及故障和预防犯罪的过程将变得更加有效和Swift。

The potential for the use of AI applications in improving security is limited only by our imagination, since AI can upgrade the existing approaches and come up with completely new ones. Just a few examples of application categories that can be examined:

使用AI应用程序来提高安全性的潜力仅受我们的想象力限制，因为AI可以升级现有方法并提出全新的方法。 可以检查的应用程序类别的几个示例：

• Spam filter applications;垃圾邮件过滤器应用；
• Network intrusion detection and prevention网络入侵检测与预防
• Credit scoring and next-best offers信用评分和次优报价
• Botnet detection僵尸网络检测
• Secure user authentication安全的用户身份验证
• Cyber security ratings网络安全等级
• Hacking incident forecasting, etc.入侵事件预测等

结论 (Conclusion)

AI is a dual-use area of technology: the same system that examines software for vulnerabilities can have both offensive and defensive applications, and there is little technical difference between the capabilities of a drone delivering packages and those of a drone delivering explosives. Since some tasks that require intelligence are benign and others are not, artificial intelligence is inherently dual — but so is human intelligence.

AI是技术的双重用途：检查软件是否存在漏洞的同一系统可以同时具有攻击性和防御性应用程序，交付包裹的无人机和交付爆炸物的无人机之间的技术差异不大。 由于某些需要智能的任务是良性的，而其他则不是，人工智能本质上是双重的，而人类的智能也是如此。