In mathematical optimization and computer science, heuristic (from Greek εὑρίσκω “I find, discover”) is a technique designed for solving a problem more quickly when classic methods are too slow or for finding an approximate solution when classic methods fail to find any exact solution. This is achieved by trading optimality, completeness, accuracy, or precision for speed. In a way, it can be considered a shortcut.

A heuristic function, also simply called a heuristic, is a function that ranks different in search algorithms at each branching step based on available information to decide which branch to follow. For example, it may approximate the exact solution.

1 Definition and motivation
2 Trade-off
3 Examples
- 3.1 Simpler problem
- 3.2 Travelling salesman problem
- 3.3 Search
- 3.4 Newell and Simon: heuristic search hypothesis
- 3.5 Antivirus software
4 Pitfalls
5 Etymology
6 See also

1 Definition and motivation

The objective of a heuristic is to produce a solution in a reasonable time frame that is good enough for solving the problem at hand. This solution may not be the best of all the solutions to this problem, or it may simply approximate the exact solution. But it is still valuable because finding it does not require a prohibitively long time.

Heuristics may produce results by themselves, or they may be used in conjunction with optimization algorithms to improve their efficiency (e.g., they may be used to generate good seed values).

Results about NP-hardness in theoretical computer science make heuristics the only viable option for a variety of complex optimization problems that need to be routinely solved in real-world applications.

Heuristics underlie the whole field of Artificial Intelligence and the computer simulation of thinking, as they may be used in situations where there are no known algorithms.

2 Trade-off

The trade-off criteria for deciding whether to use a heuristic for solving a given problem include the following:

Optimality: When several solutions exist for a given problem, does the heuristic guarantee that the best solution will be found? Is it actually necessary to find the best solution?
Completeness: When several solutions exist for a given problem, can the heuristic find them all? Do we actually need all solutions? Many heuristics are only meant to find one solution.
Accuracy and precision: Can the heuristic provide a confidence interval for the purported solution? Is the error bar on the solution unreasonably large?
Execution time: Is this the best known heuristic for solving this type of problem? Some heuristics converge faster than others. Some heuristics are only marginally quicker than classic methods, in which case the ‘overhead’ on calculating the heuristic might have negative impact.

In some cases, it may be difficult to decide whether the solution found by the heuristic is good enough, because the theory underlying heuristics is not very elaborate.

3 Examples

3.1 Simpler problem

One way of achieving the computational performance gain expected of a heuristic consists of solving a simpler problem whose solution is also a solution to the initial problem.

3.2 Travelling salesman problem

An example of approximation is described by Jon Bentley for solving the travelling salesman problem (TSP):

“Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city?”

so as to select the order to draw using a pen plotter. TSP is known to be NP-hard so an optimal solution for even a moderate size problem is difficult to solve. Instead, the greedy algorithm can be used to give a good but not optimal solution (it is an approximation to the optimal answer) in a reasonably short amount of time. The greedy algorithm heuristic says to pick whatever is currently the best next step regardless of whether that prevents (or even makes impossible) good steps later. It is a heuristic in the sense that practice indicates it is a good enough solution, while theory indicates that there are better solutions (and even indicates how much better, in some cases).

3.3 Search

Another example of heuristic making an algorithm faster occurs in certain search problems. Initially, the heuristic tries every possibility at each step, like the full-space search algorithm. But it can stop the search at any time if the current possibility is already worse than the best solution already found. In such search problems, a heuristic can be used to try good choices first so that bad paths can be eliminated early (see alpha-beta pruning). In the case of best-first search algorithms, such as A* search, the heuristic improves the algorithm’s convergence while maintaining its correctness as long as the heuristic is admissible.

3.4 Newell and Simon: heuristic search hypothesis

In their Turing Award acceptance speech, Allen Newell and Herbert A. Simon discuss the heuristic search hypothesis: a physical symbol system will repeatedly generate and modify known symbol structures until the created structure matches the solution structure. Each following step depends upon the step before it, thus the heuristic search learns what avenues to pursue and which ones to disregard by measuring how close the current step is to the solution. Therefore, some possibilities will never be generated as they are measured to be less likely to complete the solution.

A heuristic method can accomplish its task by using search trees. However, instead of generating all possible solution branches, a heuristic selects branches more likely to produce outcomes than other branches. It is selective at each decision point, picking branches that are more likely to produce solutions.

3.5 Antivirus software

Antivirus software often uses heuristic rules for detecting viruses and other forms of malware. Heuristic scanning looks for code and/or behavioral patterns common to a class or family of viruses, with different sets of rules for different viruses. If a file or executing process is found to contain matching code patterns and/or to be performing that set of activities, then the scanner infers that the file is infected. The most advanced part of behavior-based heuristic scanning is that it can work against highly randomized self-modifying/mutating (polymorphic) viruses that cannot be easily detected by simpler string scanning methods. Heuristic scanning has the potential to detect future viruses without requiring the virus to be first detected somewhere else, submitted to the virus scanner developer, analyzed, and a detection update for the scanner provided to the scanner’s users.

4 Pitfalls

Some heuristics have a strong underlying theory; they are either derived in a top-down manner from the theory or are arrived at based on either experimental or real world data. Others are just rules of thumb based on real-world observation or experience without even a glimpse of theory. The latter are exposed to a larger number of pitfalls.

When a heuristic is reused in various contexts because it has been seen to “work” in one context, without having been mathematically proven to meet a given set of requirements, it is possible that the current data set does not necessarily represent future data sets (see: overfitting) and that purported “solutions” turn out to be akin to noise.

Statistical analysis can be conducted when employing heuristics to estimate the probability of incorrect outcomes. To use a heuristic for solving a search problem or a knapsack problem, it is necessary to check that the heuristic is admissible. Given a heuristic function h(vi,vg){\displaystyle h(v_{i},v_{g})}h(vi,vg) meant to approximate the true optimal distance d⋆(vi,vg){\displaystyle d^{\star }(v_{i},v_{g})}d⋆(vi,vg) to the goal node vg{\displaystyle v_{g}}vg in a directed graph G{\displaystyle G}G containing n{\displaystyle n}n total nodes or vertices labeled v0,v1,⋯,vn{\displaystyle v_{0},v_{1},\cdots ,v_{n}}v0,v1,⋯,vn, “admissible” means roughly that the heuristic underestimates the cost to the goal or formally that h(vi,vg)≤d⋆(vi,vg)h(vi,vg){\displaystyle h(v_{i},v_{g})\leq d^{\star }(v_{i},v_{g})}h(v_{i},v_{g})h(vi,vg)≤d⋆(vi,vg)h(vi,vg) for all (vi,vg){\displaystyle (v_{i},v_{g})}(vi,vg) where i,g∈[0,1,...,n]{\displaystyle {i,g}\in [0,1,...,n]}i,g∈[0,1,...,n].

If a heuristic is not admissible, it may never find the goal, either by ending up in a dead end of graph G{\displaystyle G}G or by skipping back and forth between two nodes vi{\displaystyle v_{i}}vi and vj{\displaystyle v_{j}}vj where i,j≠g{\displaystyle {i,j}\neq g}i,j=g.

5 Etymology

Look up heuristic in Wiktionary, the free dictionary.
The word “heuristic” came into usage in the early 19th century. It is formed irregularly from the Greek word heuriskein, meaning “to find”.

6 See also

Algorithm
Constructive heuristic
Genetic algorithm
Heuristic
Heuristic routing
Heuristic evaluation: Method for identifying usability problems in user interfaces.
Metaheuristic: Methods for controlling and tuning basic heuristic algorithms, usually with usage of memory and learning.
Matheuristics: Optimization algorithms made by the interoperation of metaheuristics and mathematical programming (MP) techniques.
Reactive search optimization: Methods using online machine learning principles for self-tuning of heuristics.
Recursion (computer science)
Macro (computer science)

Heuristic (computer science)相关推荐

【Paper】【Compute Vision】Best Paper Awards in Computer Science (since 1996)
Best Paper Awards in Computer Science (since 1996) 转载之:http://jeffhuang.com/best_paper_awards.html 比 ...
计算机科学速成课】[40集全/精校] - Crash Course Computer Science
计算机科学速成课][40集全/精校] - Crash Course Computer Science Transistor have two electrodes separated by a mat ...
计算机科学生物技术,计算机科学（Computer Science）简介
原标题:计算机科学(Computer Science)简介计算机科学(Computer Science,缩写CS)是系统性研究信息与计算的理论基础,以及它们在计算机系统中如何实现与应用的实用技术的学 ...
Synchronization (computer science)
过程同步.数据同步. In computer science, synchronization refers to one of two distinct but related concepts: ...
达特茅斯计算机专业师资力量如何,达特茅斯学院计算机科学computer science专业排名第126~150名（2020THE泰晤士高等教育世界大学排名）...
2020年THE泰晤士高等教育世界大学计算机科学computer science专业排名公布,达特茅斯学院计算机科学专业排名第126~150位,达特茅斯学院计算机科学专业实力怎么样呢?下面美英港新留学 ...
am大学计算机科学,德克萨斯AM大学TAMU（Texas AM University ）计算机科学Computer Science专业排名第201-250位（2021年THE世界大学商科排名）...
2021年THE泰晤士高等教育计算机科学Computer Science专业世界大学排名公布,德克萨斯A&M大学TAMU 计算机科学世界排名第201-250位,德克萨斯A&M大学TAM ...
Computer Science Theory for the Information Age-3: 高维空间中的高斯分布和随机投影
Computer Science Theory for the Information Age-3: 高维空间中的高斯分布和随机投影高维空间中的高斯分布和随机投影 (一)在高维球体表面产生均匀分布点 ...
克莱姆森大学计算机排名,克莱姆森大学计算机科学computer science专业排名第401~500名（2020THE泰晤士高等教育世界大学排名）...
2020年THE泰晤士高等教育世界大学计算机科学computer science专业排名公布,克莱姆森大学计算机科学专业排名第401~500位,克莱姆森大学计算机科学专业实力怎么样呢?下面美英港新留学 ...
Java黑皮书课后题第1章：1.1（显示三条消息）编写程序，显示Welcome to Java、Welcome to Computer Science和Programming is fun
Java黑皮书第一章编程练习题:1.1(显示三条消息) 题目题目描述题目槽点代码片段零基础解释 ①print与println ②法2中的\n ③法2中的"+":连接两行 C ...

Heuristic (computer science)

Contents