Review of Algorithm (HITSZ) 含22年真题回忆

Review of Algorithm （HITSZ）含22年真题回忆

1. Time Analysis
- 1.1 Basic
- 1.2 Master Method
- 1.3 Recurrence Problems
2. Sorting Algorithm
- 2.1 Comparing Sort
- - 2.1.1 Insertion Sort
  - 2.1.2 Merge Sort
- 2.1.3 Shell Sort
- 2.1.4 Lower boundary of comparison sort
- 2.2 Linear Sort
- - 2.2.1 Counting Sort (Stable)
  - 2.2.2 Radix Sort (Stable)
  - 2.2.3 Bucket Sort
3. Hash Table
- 3.1 Hash method
- 3.2 Collision
4. BST
- 4.1 Normal BST
- 4.2 Treap
5. RB Tree
6. B Tree
7. Augmenting Data Structures
8. Skip Lists
9. Amortized Analysis
- 9.1 Basic Concept
- 9.2 Examples
- - 9.2.1 Dynamic tables
  - 9.2.2 Binary Counter
10. Dynamic Programming
11. Greedy Algorithm
- 11.1 Basic
- 11.2 Examples
12. Linear Programming
13. FFT
14. NP
- 14.1 P, NP, NPC, NPH
- 14.2 Solution for NPC problems
15. 2022年真题回忆
- 15.1 选择题
- 15.2 大题

1. Time Analysis

1.1 Basic

NOTE:

logab=logcblogcalog_ab=\frac{log_cb}{log_ca}logab=logcalogcb

lgn=log2nlgn=log_2nlgn=log2n

p(n)=O(f(n))p(n)=O(f(n))p(n)=O(f(n))

p(n)≤cf(n)p(n)\le cf(n)p(n)≤cf(n).
p(n)=Ω(f(n))p(n)=\Omega(f(n))p(n)=Ω(f(n))

p(n)≥cf(n)p(n)\ge cf(n)p(n)≥cf(n)
p(n)=Θ(f(n))p(n)=\Theta(f(n))p(n)=Θ(f(n))

c1f(n)≤p(n)≤c2f(n)c_1f(n)\le p(n)\le c_2f(n)c1f(n)≤p(n)≤c2f(n)
p(n)=o(f(n))p(n)=o(f(n))p(n)=o(f(n))

p(n)<cf(n)p(n)< cf(n)p(n)<cf(n)
p(n)=ω(f(n))p(n)=\omega(f(n))p(n)=ω(f(n))

p(n)>cf(n)p(n)> cf(n)p(n)>cf(n)

Solution: Calculating limn→∞p(n)f(n)lim_{n\rightarrow\infty}\frac{p(n)}{f(n)}limn→∞f(n)p(n).

If limn→∞p(n)f(n)=a≠0lim_{n\rightarrow\infty}\frac{p(n)}{f(n)}=a\ne0limn→∞f(n)p(n)=a=0, p(n)=Θ(f(n))p(n)=\Theta(f(n))p(n)=Θ(f(n))
If limn→∞p(n)f(n)=t≤alim_{n\rightarrow\infty}\frac{p(n)}{f(n)}=t\le alimn→∞f(n)p(n)=t≤a, p(n)=O(f(n))p(n)=O(f(n))p(n)=O(f(n)), since p(n)f(n)<δ+t≤δ+a\frac{p(n)}{f(n)}<\delta+t\le\delta+af(n)p(n)<δ+t≤δ+a.
If limn→∞p(n)f(n)=t≥alim_{n\rightarrow\infty}\frac{p(n)}{f(n)}=t\ge alimn→∞f(n)p(n)=t≥a, p(n)=Ω(f(n))p(n)=\Omega(f(n))p(n)=Ω(f(n)), since p(n)f(n)>t−δ≥a−δ\frac{p(n)}{f(n)}>t-\delta\ge a-\deltaf(n)p(n)>t−δ≥a−δ
If limn→∞p(n)f(n)=0lim_{n\rightarrow\infty}\frac{p(n)}{f(n)}= 0limn→∞f(n)p(n)=0, p(n)=o(f(n))p(n)=o(f(n))p(n)=o(f(n)), since p(n)f(n)<δ\frac{p(n)}{f(n)}<\deltaf(n)p(n)<δ.
If limn→∞p(n)f(n)=∞lim_{n\rightarrow\infty}\frac{p(n)}{f(n)}= \inftylimn→∞f(n)p(n)=∞, p(n)=ω(f(n))p(n)=\omega(f(n))p(n)=ω(f(n)), since p(n)f(n)>N0\frac{p(n)}{f(n)}>N_0f(n)p(n)>N0.

1.2 Master Method

Given equation like T(n)=aT(nb)+f(n)T(n)=aT(\frac{n}{b})+f(n)T(n)=aT(bn)+f(n)

If f(n)=O(nlogb(a)−ϵ)f(n)=O(n^{log_b(a)-\epsilon})f(n)=O(nlogb(a)−ϵ), T(n)=Θ(nlogb(a))T(n)=\Theta(n^{log_b(a)})T(n)=Θ(nlogb(a))
If f(n)=Θ(nlogb(a))f(n)=\Theta(n^{log_b(a)})f(n)=Θ(nlogb(a)), T(n)=Θ(nlogb(a)∗lgn)T(n)=\Theta(n^{log_b(a)}*lgn)T(n)=Θ(nlogb(a)∗lgn)
If f(n)=Ω(nlogb(a)+ϵ)f(n)=\Omega(n^{log_b(a)+\epsilon})f(n)=Ω(nlogb(a)+ϵ), and af(n/b)≤cf(n),∃c<1af(n/b)\le cf(n), \exist c<1af(n/b)≤cf(n),∃c<1, then T(n)=Θ(f(n))T(n)=\Theta(f(n))T(n)=Θ(f(n))

1.3 Recurrence Problems

Big Integer Multiplication
XY=ac2n+(ab+cd)2n/2+bdXY=ac2^n+(ab+cd)2^{n/2}+bd XY=ac2n+(ab+cd)2n/2+bd
Time complexity:
T(n)=4T(n/2)+O(n)T(n)=4T(n/2)+O(n) T(n)=4T(n/2)+O(n)

Divide and conquer:
XY=ac2n+((a+c)(b+d)−ac−bd)2n/2+bdXY=ac2^n+((a+c)(b+d)-ac-bd)2^{n/2}+bd XY=ac2n+((a+c)(b+d)−ac−bd)2n/2+bd
Time complexity:
T(n)=3T(n/2)+O(n)T(n)=3T(n/2)+O(n) T(n)=3T(n/2)+O(n)
FFT
Chess Board Cover

Put the “L” into center.
Binary Search

2. Sorting Algorithm

2.1 Comparing Sort

2.1.1 Insertion Sort

Insert a number into a sorted array. Need to move the element.

Time complexity:
T(n)=∑i=1i=n(i−1)=O(n2)T(n)=\sum_{i=1}^{i=n} (i-1) = O(n^2) T(n)=i=1∑i=n(i−1)=O(n2)

2.1.2 Merge Sort

Divide the array into two parts, sort them separately and merge them

Time complexity:
T(n)=2T(n2)+O(n)T(n)=2T(\frac{n}{2})+O(n) T(n)=2T(2n)+O(n)
According to master method: a=2,b=2,f(n)=cna=2,b=2,f(n)=cna=2,b=2,f(n)=cn. Thus nlogba=n=Θ(n)n^{log_ba}=n=\Theta(n)nlogba=n=Θ(n). So that
T(n)=Θ(nlgn)T(n)=\Theta(nlgn) T(n)=Θ(nlgn)

2.1.3 Shell Sort

希尔排序动图_哔哩哔哩_bilibili

Time complexity:
≈O(N1.25)\approx O(N^{1.25}) ≈O(N1.25)

2.1.4 Lower boundary of comparison sort

Θ(nlgn)\Theta(nlgn) Θ(nlgn)

2.2 Linear Sort

2.2.1 Counting Sort (Stable)

Flow:
- Input array AAA.
- Assigning 0 to array CCC. (O(k)O(k)O(k))
- Counting each element in an array AAA, recording in CCC. (O(n)O(n)O(n))
- Accumulating array CCC. (O(k)O(k)O(k))
- Sorting in array BBB. (O(n)O(n)O(n))
Time complexity: O(n+k)O(n + k)O(n+k), where nnn is the input number, kkk is equivalent to value of the maximal number.

2.2.2 Radix Sort (Stable)

Run Counting Sort in group
Time complexity: O(br(n+2r))=O(n)O(\frac{b}{r}(n+2^r))=O(n)O(rb(n+2r))=O(n). Where bbb is the bits number of the compared numbers, rrr is the bits number of per group number. Thus there are br\frac{b}{r}rb groups. We do not want r>lgnr>lgnr>lgn.

2.2.3 Bucket Sort

Divide input array into several buckets, running insert sort in each bucket and concatenate them together.
Expected time: Θ(n)\Theta(n)Θ(n)
Worst-case performance: O(n2)O(n^2)O(n2)
How to improve? Change insert sort to other efficient sort. Like, merge sort.

3. Hash Table

3.1 Hash method

Division method: h(k)=kmodmh(k)=k\mod mh(k)=kmodm. Do not choose m=2rm=2^rm=2r, this not uses all of bits in kkk.
Multiplication method: h(k)=(Akmod2r)rsh(w−r)h(k)=(Ak\mod 2^r) rsh(w-r)h(k)=(Akmod2r)rsh(w−r). AAA is an odd integer in the range 2w−1≤A≤2w2^{w-1}\le A\le2^w2w−1≤A≤2w
An unsuccessful search requires: 1+α1+\alpha1+α, where α=m/n\alpha=m/nα=m/n, refer to load factor.

3.2 Collision

NOTE:

AmodB=A−(A÷B)∗BA \mod B = A - (A \div B)*BAmodB=A−(A÷B)∗B

(A+B)modN=(AmodN+BmodN)modN(A+B)\mod N=(A\mod N + B\mod N) \mod N(A+B)modN=(AmodN+BmodN)modN

Assuming h(k)=kmodmh(k)=k\mod mh(k)=kmodm.
Learning probing: h′(k,i)=(kmodm+i)modmh'(k,i)=(k \mod m+i) \mod mh′(k,i)=(kmodm+i)modm
Quadratic probing: h′(k,i)=(kmodm+c1i+c2i2)modmh'(k,i)=(k \mod m+c_1i+c_2i^2) \mod mh′(k,i)=(kmodm+c1i+c2i2)modm
Double hash probing: h′(k,i)=(kmodm+i∗h2(k))modmh'(k,i)=(k \mod m+i*h_2(k)) \mod mh′(k,i)=(kmodm+i∗h2(k))modm
Theorem: Open-addressed hash table with load factor α\alphaα, the expected number of probes in an unsuccessful search is at most 1/(1−α)1/(1-\alpha)1/(1−α) . That is to say:
- If the table is half full, then the expected number of probes is 1/(1–0.5)=21/(1–0.5) = 21/(1–0.5)=2.
- If the table is 90% full, then the expected number of probes is 1/(1–0.9)=101/(1–0.9) = 101/(1–0.9)=10.

4. BST

4.1 Normal BST

Left subtree is less than root
Right subtree is large than root
Deletion:
- No children: Delete directly.
- One child: Delete and make the child as the parent’s child.
- Two child: Replace with the successor. Apply 1 or 2 to delete it.
Randomly built BST has O(lgn)O(lgn)O(lgn) search time complexity.

4.2 Treap

Treap is the BST with priority. This helps build a randomly BST.

5. RB Tree

Node is either red or black.
Root is always black.
Black height has only one.
Terminate with two black nodes.
Red node cannot connect with red node.
Theorem: A RB tree with nnn internal nodes has height h≤2lg(n+1)h\le2lg(n+1)h≤2lg(n+1). h=O(lgn)h=O(lgn)h=O(lgn).
- In my opinion, N≤n+n−1N\le n+n-1N≤n+n−1, so that h≤lg(2n−1)=O(lgn)h\le lg(2n-1)=O(lgn)h≤lg(2n−1)=O(lgn)

6. B Tree

Degree nnn (out degree): allows at least n−1n-1n−1 keys, at most 2n−12n-12n−1 keys.
Split when insertion.
h≤logt((n+1)2)h\le log_t(\frac{(n+1)}{2})h≤logt(2(n+1)). See proof below.

Deletion: Make sure each node has at least t−1t-1t−1 keys and merge those nodes can be merged. i.e., both t−1t-1t−1.

7. Augmenting Data Structures

Order-Statistic Tree (OS-Tree): Recording the number of elements of subtree.
Interval Tree: Each node represents an interval [a,b][a,b][a,b], and maintains a maxmaxmax field, which satisfies:

max=max({node.highnode.left.maxnode.right.max)max=max(\left\{ \begin{aligned} node.high \\ node.left.max \\ node.right.max \end{aligned} \right.) max=max(⎩⎪⎨⎪⎧node.highnode.left.maxnode.right.max)

Search interval i in the Interval Tree:
- Check if root is overlapped. If it is overlapped, then return root.
- Check whether root.left.max ≥ i.left, go to left. Recursively run process.
- Otherwise, go to right. Recursively run process.

8. Skip Lists

Each node is promoted with 1/21/21/2 probability.
Multiple indexing
Idea: Skip list with lgnlgnlgn linked lists is like a binary tree (B+ tree).
With high probability, every search in an n-element skip list costs O(lgn)O(lg n)O(lgn)

9. Amortized Analysis

9.1 Basic Concept

Aggregated:
T=∑i=1nunitnT=\frac {\sum_{i=1}^n unit}{n} T=n∑i=1nunit
Accounting: Store ccc for each normal case operation, and each normal case operation spends mmm. Thus, you store c−mc-mc−m to the bank for each normal case operation. When the cost operation is coming, you spend all of your money in the bank. Make sure that the bank always has positive money.
T=∑i=1ncn=cT=\frac{\sum_{i=1}^n c}{n}=c T=n∑i=1nc=c
Potential: Design a potential function ϕ(Di)\phi(D_i)ϕ(Di), s.t., ci′=ci+ϕ(Di)−ϕ(Di−1)c'_i=c_i+\phi(D_i)-\phi(D_{i-1})ci′=ci+ϕ(Di)−ϕ(Di−1). Make sure that ϕ(D0)=0\phi(D_0)=0ϕ(D0)=0, and ϕ(Di)≥0\phi(D_i)\ge 0ϕ(Di)≥0

9.2 Examples

9.2.1 Dynamic tables

Aggregated
ci={i,i−1=2n1,otherwisec_i=\left\{ \begin{aligned} i, i-1=2^n \\ 1, otherwise \\ \end{aligned} \right. ci={i,i−1=2n1,otherwise

∑i=1nci≤n+∑j=1lg(n−1)2j≤3n=Θ(n)\sum_{i=1}^nc_i\le n+\sum_{j=1}^{lg(n-1)} 2^j\le3n=\Theta(n) i=1∑nci≤n+j=1∑lg(n−1)2j≤3n=Θ(n)

Thus,
ci′=Θ(n)/n=Θ(1)c_i'=\Theta(n)/n= \Theta(1) ci′=Θ(n)/n=Θ(1)
Accounting

Charge ci′=3c_i'=3ci′=3 for each operation, so that 111 for insertion, 222 is stored for extending. When extending the table, 111 for inserting old item. 111 for inserting new item. See the figure below

Potential

Define
ϕ(Di)=2i−2⌈lgi⌉\phi(D_i)=2i-2^{\lceil lgi\rceil } ϕ(Di)=2i−2⌈lgi⌉
So that
ci′=ci+ϕ(Di)−ϕ(Di−1)c_i'=c_i+\phi(D_i)-\phi(D_{i-1}) ci′=ci+ϕ(Di)−ϕ(Di−1)
Thus, we have ci′=3c_i'=3ci′=3.

9.2.2 Binary Counter

Aggregated

The summation of flip frequency of each bit.
n2+n4+n8+...+n2n=2n\frac{n}{2}+\frac{n}{4}+\frac{n}{8}+...+\frac{n}{2^n}=2n 2n+4n+8n+...+2nn=2n
So that ci′=2c_i'=2ci′=2.
Accounting

Operation 0→10\rightarrow 10→1 store 222, and pay for 111. Operation 1→01\rightarrow 01→0 pay for 111 store 0. Thus, for nnn operations, we must store 2n2n2n fee. Thus ci′=2c_i'=2ci′=2.
Potential

Define
ϕ(Di)=numberof1incounter\phi(D_i)= number\ of\ 1\ in \ counter ϕ(Di)=number of 1 in counter
Thus,
ci′=ci+ϕ(Di)−ϕ(Di−1)c_i'=c_i+\phi(D_i)-\phi(D_{i-1}) ci′=ci+ϕ(Di)−ϕ(Di−1)
Let operation iii changes tit_iti 111 to 000. And there only one 000 is changed to 111 (But be careful to the situation that the counter is overflow). Thus
ϕ(Di)≤ϕ(Di−1)−ti+1\phi(D_i)\le\phi(D_{i-1})-t_i+1 ϕ(Di)≤ϕ(Di−1)−ti+1
So that
ci′≤ti+1+1−ti=2c_i'\le t_i+1 +1-t_i=2 ci′≤ti+1+1−ti=2

10. Dynamic Programming

Matrix-chain Multiplication

The minimal multiplication times of A1∗A2∗...∗AnA_1*A_2*...*A_nA1∗A2∗...∗An

Let Min[i,j]Min[i,j]Min[i,j] denote the minimal multiplication times of Ai∗...∗AjA_i*...*A_jAi∗...∗Aj
Min[i,j]={0i=jmini≤k≤j{Min[i,j],Min[i,k]+Min[k+1,j]+pi∗pk∗pj}i≠jMin[i,j]= \begin{cases} 0 & \text{ $ i=j $ } \\ min_{i\le k\le j}\{Min[i,j], Min[i,k]+Min[k+1,j]+p_i*p_k*p_j\} & \text{ $ i\ne j $ } \end{cases} Min[i,j]={0mini≤k≤j{Min[i,j],Min[i,k]+Min[k+1,j]+pi∗pk∗pj} i=j i=j
LCS problem

The longest common sub sequence of X=x1x2...xmX=x_1x_2...x_mX=x1x2...xm and Y=y1y2...ynY=y_1y_2...y_nY=y1y2...yn

Let LCS[i,j]LCS[i,j]LCS[i,j] denote the length of the longest common sequence of Xi=x1...xiX_i=x1...x_iXi=x1...xi and Yj=y1...yjY_j=y_1...y_jYj=y1...yj

Thus,
LCS[i,j]={0i=0orj=0LCS[i−1,j−1]+1Xi=Yjmax{LCS[i−1,j],LCS[i,j−1]}Xi≠YjLCS[i,j]= \begin{cases} 0 & \text{ $ i =0 \ or\ j=0 $} \\ LCS[i-1,j-1]+1 & \text{ $ X_i=Y_j $ } \\ max\{ LCS[i-1,j],LCS[i,j-1]\} & \text{ $ X_i\ne Y_j $ } \end{cases} LCS[i,j]=⎩⎪⎨⎪⎧0LCS[i−1,j−1]+1max{LCS[i−1,j],LCS[i,j−1]} i=0 or j=0 Xi=Yj Xi=Yj
Triangle Decomposition of Convex Polygon

Find the minimal summation of triangle decomposition.

Let Min[i,j]Min[i,j]Min[i,j] denote the minimal summation triangle decomposition of vertex from iii to jjj.

Thus,
Min[i,j]={0i=jmini<k<j{Min[i,k]+Min[k,j]+w(vivjvk)}i≠jMin[i,j]= \begin{cases} 0 & \text{ $ i=j $ } \\ min_{i< k< j}\{Min[i,k]+Min[k,j]+w(v_i v_j v_k)\} & \text{ $ i\ne j $ } \end{cases} Min[i,j]={0mini<k<j{Min[i,k]+Min[k,j]+w(vivjvk)} i=j i=j

11. Greedy Algorithm

11.1 Basic

To prove a greedy algorithm is workable, you should:

First, show the optimal structure: Given an optimal solution of the problem, prove that it’s subproblem is also optimal. i.e., applying the recursive function like DP problem.
Second, show that the greedy selection must be in the final optimal solution.

11.2 Examples

Activities arrangement
0-1 Knapsack

12. Linear Programming

Simplex Algorithm
- Build the slack form, where each variable is constrained by ≥0\ge 0≥0.
- Find the most constrained variable, replacing it with the slack variable. Where slack variable can reach to 000.
- Recursive to find the anwser.

13. FFT

To solve a FFT problem, for example:

Compute DFT of input (x(0),x(1),x(2),x(3),x(4),x(5),x(6),x(7))(x(0),x(1),x(2),x(3),x(4),x(5),x(6),x(7))(x(0),x(1),x(2),x(3),x(4),x(5),x(6),x(7)) by using FFT

First, determine leaves. In this example, leaves are

(x(0),x(4))(x(0),x(4))(x(0),x(4)), (x(2),x(6))(x(2),x(6))(x(2),x(6)), (x(1),x(5))(x(1),x(5))(x(1),x(5)), (x(3),x(7))(x(3),x(7))(x(3),x(7))
Second, draw a butterfly. i.e,
There are two things you should note:
1. Recursively. W80=W40W_8^0=W_4^0W80=W40, W82=W41W_8^2=W_4^1W82=W41, and 1=W201=W_2^01=W20
2. Positive for even number and Negative for odd number.

Third, compute from leaves, for example
X(0)=A(0)+W80B(0)=(C(0)+D(0))+W80(E(0)+F(0))=(x(0)+x(4)+x(2)+x(6))+W80(x(1)+x(5)+x(3)+x(7))=∑i=0i=7x(i)\begin{aligned} X(0) &=A(0)+W_8^0B(0)\\ &=(C(0)+D(0))+W_8^0(E(0)+F(0))\\ &=(x(0)+x(4)+x(2)+x(6))+W_8^0(x(1)+x(5)+x(3)+x(7))\\ &=\sum_{i=0}^{i=7} x(i) \end{aligned} X(0)=A(0)+W80B(0)=(C(0)+D(0))+W80(E(0)+F(0))=(x(0)+x(4)+x(2)+x(6))+W80(x(1)+x(5)+x(3)+x(7))=i=0∑i=7x(i)

14. NP

14.1 P, NP, NPC, NPH

P: Solve and Verify in polynomial time.
NP: Verify in polynomial time.
NP-hard: All NP problems can be reduce to the problem.
NPC: ① It is a NP problem ② Any NP problem can be reduced to the problem.
The relations
- If NP≠PNP\ne PNP=P: NPHNPHNPH can be reduced to either NPNPNP problems or other non−Pnon-Pnon−P problems.
- If NP=PNP=PNP=P: NPHNPHNPH can be reduce to NP=PNP=PNP=P problems or other problems. Thus if one problem is NPNPNP, it is a NPHNPHNPH problem. Thus, it is a NPCNPCNPC problem.

14.2 Solution for NPC problems

Exponential solution
- Backtracking (DFS + prune)
- Dynamic programming
Approximate algorithm
- Approximate ratio
  
  Assume the best solution is c∗c^*c∗, and the approximate solution is ccc. The approximate ratio is:
  max{c∗c,cc∗}≤ρ(n)max\{\frac{c^*}{c},\frac{c}{c^*}\} \le \rho(n) max{cc∗,c∗c}≤ρ(n)
  ρ(n)\rho(n)ρ(n) is approximate ratio.
- Relative error ϵ(n)\epsilon(n)ϵ(n)
  ∣c−c∗c∗∣≤ϵ(n)|\frac{c-c^*}{c^*}|\le \epsilon(n) ∣c∗c−c∗∣≤ϵ(n)

15. 2022年真题回忆

选择 + 大题

15.1 选择题

共5题

FFT时间复杂度 (OlgnOlgnOlgn)
红黑树插入时间复杂度（lgnlgnlgn）
B树度为t, 判断关键字数、儿子数
哪种形式是hash的线性探测
忘记了

15.2 大题

树堆（Treap）插入
用FFT计算DFT
递归树计算T(n)=3T(n/4)+cn2T(n)=3T(n/4)+cn^2T(n)=3T(n/4)+cn2
线性规划，求解18x+12.5y18x+12.5y18x+12.5y的最大值
证明活动选择问题，选择最晚开始的活动
BST与红黑树问题：给一个树形结构填值，使之符合BST；红黑树黑高度为k，最大、最少内部节点数；BST旋转一下变成红黑树
B树+Augumenting
动规问题，合并石堆

总的来说不算难，但是第7题的证明题是真难想，希望有分~~