随机过程、马尔可夫链详解

学校的概率统计课程没有讲到这一部分的内容，但是又比较重要，因此来填个小坑

PS：参考概率论与数理统计第五版浙大盛骤谢式千潘承毅、Blibili@FunInCode

一、随机过程

1.概念

用TTT表示一无限实数集，我们把依赖于参数t∈Tt \in Tt∈T的一族随机变量{Xt,t∈T}\{X_t, t \in T\}{Xt,t∈T}，称为随机过程。

参数集：把TTT(无限实数集)称为参数集

状态：通常把t∈Tt \in Tt∈T看作时间，称对XtX_tXt的观察值xxx为ttt时过程的状态

状态空间：随机过程{Xt,t∈T}\{X_t, t \in T\}{Xt,t∈T}所有可能的取的状态全体称为随机过程的状态空间

样本函数/样本曲线：对{Xt,t∈T}\{X_t, t \in T\}{Xt,t∈T}进行一次实验(即在TTT上进行一次全程观测)，得到的一个函数x(t),t∈Tx(t), t \in Tx(t),t∈T，称为样本函数或样本曲线。

随机过程可以看作是多维随机变量的延伸，随机过程与其样本函数的关系和数理统计中总体于样本的关系是类似的。

随机过程可根据其在任一时刻ttt的状态XXX，是连续型随机变量或离散型随机变量而分为连续型随机过程或离散型随机过程。随机过程还可以依时间参数分类，当TTT是有限区间或无限区间时称Xt,t∈T{X_t, t\in T}Xt,t∈T为连续参数随机过程。对于连续参数随机过程我们常用记号X(t)=XX(t) = XX(t)=X来表示其对参数ttt的函数依赖关系。当TTT为离散型随机变量的时候称之为离散参数随机过程或时间序列。

2.随机过程的统计描述

(一)、随机过程的分布函数族

给定随机过程{X(t),t∈T}\{X(t), t \in T\}{X(t),t∈T}，对于每一个固定的的ttt，随机变量X(t)X(t)X(t)的随机分布函数一般与ttt有关，记为：
FX(x;t)=P(X(t)≤x),x∈RF_X(x;t) = P(X(t) \leq x), x \in R FX(x;t)=P(X(t)≤x),x∈R
称为随机过程{X(t),t∈T}\{X(t), t\in T\}{X(t),t∈T}的一维分布函数，而FX(x;t),t∈TF_X(x;t), t \in TFX(x;t),t∈T称为一维分布函数族。

一维分布函数族刻画了随机过程在各个个别时刻的统计特性，为了描述随机过程在不同时刻状态之间的统计联系，一般可以对∀n(n=2,3,…)\forall n (n = 2,3,\dots)∀n(n=2,3,…)个不同时刻t1,t2,…,tn∈Tt_1, t_2, \dots , t_n \in Tt1,t2,…,tn∈T引入随机变量(X(t1),X(t2),…,X(tn))(X(t_1), X(t_2), \dots , X(t_n))(X(t1),X(t2),…,X(tn))，它的分布函数记为：
F(x1,x2,…,xn;t1,t2,…,tn)=P{X(t1)≤x1,X(t2)≤x2,…,X(tn)≤xn},xi∈R,i=1,2,…,n.F(x_1, x_2, \dots, x_n; t_1, t_2, \dots, t_n) = P\{X(t_1) \leq x_1, X(t_2) \leq x_2, \dots, X(t_n) \leq x_n\}, x_i \in R, i = 1, 2, \dots, n. F(x1,x2,…,xn;t1,t2,…,tn)=P{X(t1)≤x1,X(t2)≤x2,…,X(tn)≤xn},xi∈R,i=1,2,…,n.
对于固定的nnn，称{F(x1,x2,…,xn;t1,t2,…,tn);t1∈T}\{F(x_1, x_2, \dots, x_n; t_1, t_2, \dots, t_n);t_1 \in T\}{F(x1,x2,…,xn;t1,t2,…,tn);t1∈T}为随机过程{X(t),t∈T}\{X(t), t \in T\}{X(t),t∈T}的nnn维分布函数族。

当nnn充分大的时候，nnn维分布函数族能够近似的描述随机过程的统计特性，显然nnn取得越大，则nnn维分布函数描述随机过程的统计特性也越趋完善。

(二)、随机规程的数字特征

在实际的应用中，单纯的通过观察难以确定随机过程的有限维分布函数族，因此需要引入随机过程的的数字特征

均值函数：对于给定的随机过程{X(t),t∈T}\{X(t) , t \in T\}{X(t),t∈T}，固定t∈T,X(t)t \in T, X(t)t∈T,X(t)为一维随机变量，它的均值一般与ttt有关，记作：μx(t)=E[X(t)]\mu_x(t) = E[X(t)]μx(t)=E[X(t)]，称之为随即过程{X(t),t∈T}\{X(t) , t \in T\}{X(t),t∈T}的均值函数。

均值函数μx(t)\mu_x(t)μx(t)描述了随机过程X(t)X(t)X(t)在各个时刻ttt的摆动中心

均方值函数/方差函数、标准差函数：将随机变量X(t)X(t)X(t)的二阶原点矩和二阶中心距分别记为ψX2(t)=E[X2(t)]\psi^2_X(t) = E[X^2(t)]ψX2(t)=E[X2(t)]和σX2(t)=DX(t)=Var[X(t)]=E{[X(t)−μX(t)]2}\sigma^2_X(t) = D_X(t) = Var[X(t)] = E\{[X(t) - \mu_X(t)]^2\}σX2(t)=DX(t)=Var[X(t)]=E{[X(t)−μX(t)]2}，并分别称之为随机过程{X(t),t∈T}\{X(t) , t \in T\}{X(t),t∈T}的均方值函数和方差函数，方差函数的平方根σX(t)\sigma_X(t)σX(t)称为随机过程的标准差函数

标准差函数反映了随机过程X(t)X(t)X(t)在时刻ttt对于均值函数μX(t)\mu_X(t)μX(t)的平均偏离程度。

自相关系数/相关函数：对于∀t1,t2∈T\forall t_1,t_2 \in T∀t1,t2∈T，将随机变量X(t1)X(t_1)X(t1)和X(t2)X(t_2)X(t2)的二阶混合原点矩记作RXX(t1,t2)=E(X(t1)X(t2))R_{XX}(t_1, t_2) = E(X(t_1)X(t_2))RXX(t1,t2)=E(X(t1)X(t2))，并称之为随机过程{X(t),t∈T}\{X(t), t \in T\}{X(t),t∈T}的自相关系数，简称相关系数。

自协方差函数/协方差函数：对于∀t1,t2∈T\forall t_1,t_2 \in T∀t1,t2∈T，将随机变量X(t1)X(t_1)X(t1)和X(t2)X(t_2)X(t2)的二阶混合中心矩记作CXX(t1,t2)=Cov[X(t1),X(t2)]=E{[X(t1)−μX(t1)][X(t2)−μX(t2)]}C_{XX}(t_1, t_2)=Cov[X(t_1), X(t_2)] = E\{[X(t_1) - \mu_X(t_1)][X(t_2) - \mu_X(t_2)]\}CXX(t1,t2)=Cov[X(t1),X(t2)]=E{[X(t1)−μX(t1)][X(t2)−μX(t2)]}，并称之为随机过程{X(t),t∈T}\{X(t), t \in T\}{X(t),t∈T}的自协方差系数，简称协方差系数。

运算关系：σX2(t)=CXX(t,t)=RXX(t,t)−μX2(t)\sigma^2_X(t) = C_{XX}(t, t) = R_{XX}(t, t) - \mu^2_X(t)σX2(t)=CXX(t,t)=RXX(t,t)−μX2(t)

正态过程：当随机过程{X(t),t∈T}\{X(t), t \in T\}{X(t),t∈T}的每一个有限维分布都是正态分布，即对于∀\forall∀正整数n≥1n \geq 1n≥1以及∀t1,t2,…,tn∈T,(X(t1),X(t2),…,X(tn))\forall t_1, t_2, \dots, t_n \in T, (X(t_1), X(t_2),\dots, X(t_n))∀t1,t2,…,tn∈T,(X(t1),X(t2),…,X(tn))服从nnn维正态分布，则称该随机过程为正态过程。

二、马尔可夫链

1.基本概念

对于一个由NNN步构成的随机过程，每一步的结构仅与上一步有关，与之后的步骤、上一步之前的过程均无关，那么该过程即为马尔可夫过程，一般称为马尔科夫链。

马尔科夫链具有三个核心要素：

状态空间(States Space)
无记忆性(Memorylessness) (P(X∣Xt−1,Xt−1=2,…)=P(X∣Xt−1)P(X|X_{t - 1},X_{t - 1=2},\dots) = P(X|X_{t - 1})P(X∣Xt−1,Xt−1=2,…)=P(X∣Xt−1))
转移矩阵(Transition Matrix)

2.马尔可夫链的状态概率分布推演及稳态分布

(1).状态概率分布推演与稳态分布

我们以一个简单的例子来进行分析说明：假设有两个选择AAA、BBB，遵从以下关系：

#mermaid-svg-h3IlYGkht9B17ddY .label{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);fill:#333;color:#333}#mermaid-svg-h3IlYGkht9B17ddY .label text{fill:#333}#mermaid-svg-h3IlYGkht9B17ddY .node rect,#mermaid-svg-h3IlYGkht9B17ddY .node circle,#mermaid-svg-h3IlYGkht9B17ddY .node ellipse,#mermaid-svg-h3IlYGkht9B17ddY .node polygon,#mermaid-svg-h3IlYGkht9B17ddY .node path{fill:#ECECFF;stroke:#9370db;stroke-width:1px}#mermaid-svg-h3IlYGkht9B17ddY .node .label{text-align:center;fill:#333}#mermaid-svg-h3IlYGkht9B17ddY .node.clickable{cursor:pointer}#mermaid-svg-h3IlYGkht9B17ddY .arrowheadPath{fill:#333}#mermaid-svg-h3IlYGkht9B17ddY .edgePath .path{stroke:#333;stroke-width:1.5px}#mermaid-svg-h3IlYGkht9B17ddY .flowchart-link{stroke:#333;fill:none}#mermaid-svg-h3IlYGkht9B17ddY .edgeLabel{background-color:#e8e8e8;text-align:center}#mermaid-svg-h3IlYGkht9B17ddY .edgeLabel rect{opacity:0.9}#mermaid-svg-h3IlYGkht9B17ddY .edgeLabel span{color:#333}#mermaid-svg-h3IlYGkht9B17ddY .cluster rect{fill:#ffffde;stroke:#aa3;stroke-width:1px}#mermaid-svg-h3IlYGkht9B17ddY .cluster text{fill:#333}#mermaid-svg-h3IlYGkht9B17ddY div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);font-size:12px;background:#ffffde;border:1px solid #aa3;border-radius:2px;pointer-events:none;z-index:100}#mermaid-svg-h3IlYGkht9B17ddY .actor{stroke:#ccf;fill:#ECECFF}#mermaid-svg-h3IlYGkht9B17ddY text.actor>tspan{fill:#000;stroke:none}#mermaid-svg-h3IlYGkht9B17ddY .actor-line{stroke:grey}#mermaid-svg-h3IlYGkht9B17ddY .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333}#mermaid-svg-h3IlYGkht9B17ddY .messageLine1{stroke-width:1.5;stroke-dasharray:2, 2;stroke:#333}#mermaid-svg-h3IlYGkht9B17ddY #arrowhead path{fill:#333;stroke:#333}#mermaid-svg-h3IlYGkht9B17ddY .sequenceNumber{fill:#fff}#mermaid-svg-h3IlYGkht9B17ddY #sequencenumber{fill:#333}#mermaid-svg-h3IlYGkht9B17ddY #crosshead path{fill:#333;stroke:#333}#mermaid-svg-h3IlYGkht9B17ddY .messageText{fill:#333;stroke:#333}#mermaid-svg-h3IlYGkht9B17ddY .labelBox{stroke:#ccf;fill:#ECECFF}#mermaid-svg-h3IlYGkht9B17ddY .labelText,#mermaid-svg-h3IlYGkht9B17ddY .labelText>tspan{fill:#000;stroke:none}#mermaid-svg-h3IlYGkht9B17ddY .loopText,#mermaid-svg-h3IlYGkht9B17ddY .loopText>tspan{fill:#000;stroke:none}#mermaid-svg-h3IlYGkht9B17ddY .loopLine{stroke-width:2px;stroke-dasharray:2, 2;stroke:#ccf;fill:#ccf}#mermaid-svg-h3IlYGkht9B17ddY .note{stroke:#aa3;fill:#fff5ad}#mermaid-svg-h3IlYGkht9B17ddY .noteText,#mermaid-svg-h3IlYGkht9B17ddY .noteText>tspan{fill:#000;stroke:none}#mermaid-svg-h3IlYGkht9B17ddY .activation0{fill:#f4f4f4;stroke:#666}#mermaid-svg-h3IlYGkht9B17ddY .activation1{fill:#f4f4f4;stroke:#666}#mermaid-svg-h3IlYGkht9B17ddY .activation2{fill:#f4f4f4;stroke:#666}#mermaid-svg-h3IlYGkht9B17ddY .mermaid-main-font{font-family:"trebuchet ms", verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-h3IlYGkht9B17ddY .section{stroke:none;opacity:0.2}#mermaid-svg-h3IlYGkht9B17ddY .section0{fill:rgba(102,102,255,0.49)}#mermaid-svg-h3IlYGkht9B17ddY .section2{fill:#fff400}#mermaid-svg-h3IlYGkht9B17ddY .section1,#mermaid-svg-h3IlYGkht9B17ddY .section3{fill:#fff;opacity:0.2}#mermaid-svg-h3IlYGkht9B17ddY .sectionTitle0{fill:#333}#mermaid-svg-h3IlYGkht9B17ddY .sectionTitle1{fill:#333}#mermaid-svg-h3IlYGkht9B17ddY .sectionTitle2{fill:#333}#mermaid-svg-h3IlYGkht9B17ddY .sectionTitle3{fill:#333}#mermaid-svg-h3IlYGkht9B17ddY .sectionTitle{text-anchor:start;font-size:11px;text-height:14px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-h3IlYGkht9B17ddY .grid .tick{stroke:#d3d3d3;opacity:0.8;shape-rendering:crispEdges}#mermaid-svg-h3IlYGkht9B17ddY .grid .tick text{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-h3IlYGkht9B17ddY .grid path{stroke-width:0}#mermaid-svg-h3IlYGkht9B17ddY .today{fill:none;stroke:red;stroke-width:2px}#mermaid-svg-h3IlYGkht9B17ddY .task{stroke-width:2}#mermaid-svg-h3IlYGkht9B17ddY .taskText{text-anchor:middle;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-h3IlYGkht9B17ddY .taskText:not([font-size]){font-size:11px}#mermaid-svg-h3IlYGkht9B17ddY .taskTextOutsideRight{fill:#000;text-anchor:start;font-size:11px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-h3IlYGkht9B17ddY .taskTextOutsideLeft{fill:#000;text-anchor:end;font-size:11px}#mermaid-svg-h3IlYGkht9B17ddY .task.clickable{cursor:pointer}#mermaid-svg-h3IlYGkht9B17ddY .taskText.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-h3IlYGkht9B17ddY .taskTextOutsideLeft.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-h3IlYGkht9B17ddY .taskTextOutsideRight.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-h3IlYGkht9B17ddY .taskText0,#mermaid-svg-h3IlYGkht9B17ddY .taskText1,#mermaid-svg-h3IlYGkht9B17ddY .taskText2,#mermaid-svg-h3IlYGkht9B17ddY .taskText3{fill:#fff}#mermaid-svg-h3IlYGkht9B17ddY .task0,#mermaid-svg-h3IlYGkht9B17ddY .task1,#mermaid-svg-h3IlYGkht9B17ddY .task2,#mermaid-svg-h3IlYGkht9B17ddY .task3{fill:#8a90dd;stroke:#534fbc}#mermaid-svg-h3IlYGkht9B17ddY .taskTextOutside0,#mermaid-svg-h3IlYGkht9B17ddY .taskTextOutside2{fill:#000}#mermaid-svg-h3IlYGkht9B17ddY .taskTextOutside1,#mermaid-svg-h3IlYGkht9B17ddY .taskTextOutside3{fill:#000}#mermaid-svg-h3IlYGkht9B17ddY .active0,#mermaid-svg-h3IlYGkht9B17ddY .active1,#mermaid-svg-h3IlYGkht9B17ddY .active2,#mermaid-svg-h3IlYGkht9B17ddY .active3{fill:#bfc7ff;stroke:#534fbc}#mermaid-svg-h3IlYGkht9B17ddY .activeText0,#mermaid-svg-h3IlYGkht9B17ddY .activeText1,#mermaid-svg-h3IlYGkht9B17ddY .activeText2,#mermaid-svg-h3IlYGkht9B17ddY .activeText3{fill:#000 !important}#mermaid-svg-h3IlYGkht9B17ddY .done0,#mermaid-svg-h3IlYGkht9B17ddY .done1,#mermaid-svg-h3IlYGkht9B17ddY .done2,#mermaid-svg-h3IlYGkht9B17ddY .done3{stroke:grey;fill:#d3d3d3;stroke-width:2}#mermaid-svg-h3IlYGkht9B17ddY .doneText0,#mermaid-svg-h3IlYGkht9B17ddY .doneText1,#mermaid-svg-h3IlYGkht9B17ddY .doneText2,#mermaid-svg-h3IlYGkht9B17ddY .doneText3{fill:#000 !important}#mermaid-svg-h3IlYGkht9B17ddY .crit0,#mermaid-svg-h3IlYGkht9B17ddY .crit1,#mermaid-svg-h3IlYGkht9B17ddY .crit2,#mermaid-svg-h3IlYGkht9B17ddY .crit3{stroke:#f88;fill:red;stroke-width:2}#mermaid-svg-h3IlYGkht9B17ddY .activeCrit0,#mermaid-svg-h3IlYGkht9B17ddY .activeCrit1,#mermaid-svg-h3IlYGkht9B17ddY .activeCrit2,#mermaid-svg-h3IlYGkht9B17ddY .activeCrit3{stroke:#f88;fill:#bfc7ff;stroke-width:2}#mermaid-svg-h3IlYGkht9B17ddY .doneCrit0,#mermaid-svg-h3IlYGkht9B17ddY .doneCrit1,#mermaid-svg-h3IlYGkht9B17ddY .doneCrit2,#mermaid-svg-h3IlYGkht9B17ddY .doneCrit3{stroke:#f88;fill:#d3d3d3;stroke-width:2;cursor:pointer;shape-rendering:crispEdges}#mermaid-svg-h3IlYGkht9B17ddY .milestone{transform:rotate(45deg) scale(0.8, 0.8)}#mermaid-svg-h3IlYGkht9B17ddY .milestoneText{font-style:italic}#mermaid-svg-h3IlYGkht9B17ddY .doneCritText0,#mermaid-svg-h3IlYGkht9B17ddY .doneCritText1,#mermaid-svg-h3IlYGkht9B17ddY .doneCritText2,#mermaid-svg-h3IlYGkht9B17ddY .doneCritText3{fill:#000 !important}#mermaid-svg-h3IlYGkht9B17ddY .activeCritText0,#mermaid-svg-h3IlYGkht9B17ddY .activeCritText1,#mermaid-svg-h3IlYGkht9B17ddY .activeCritText2,#mermaid-svg-h3IlYGkht9B17ddY .activeCritText3{fill:#000 !important}#mermaid-svg-h3IlYGkht9B17ddY .titleText{text-anchor:middle;font-size:18px;fill:#000;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-h3IlYGkht9B17ddY g.classGroup text{fill:#9370db;stroke:none;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);font-size:10px}#mermaid-svg-h3IlYGkht9B17ddY g.classGroup text .title{font-weight:bolder}#mermaid-svg-h3IlYGkht9B17ddY g.clickable{cursor:pointer}#mermaid-svg-h3IlYGkht9B17ddY g.classGroup rect{fill:#ECECFF;stroke:#9370db}#mermaid-svg-h3IlYGkht9B17ddY g.classGroup line{stroke:#9370db;stroke-width:1}#mermaid-svg-h3IlYGkht9B17ddY .classLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5}#mermaid-svg-h3IlYGkht9B17ddY .classLabel .label{fill:#9370db;font-size:10px}#mermaid-svg-h3IlYGkht9B17ddY .relation{stroke:#9370db;stroke-width:1;fill:none}#mermaid-svg-h3IlYGkht9B17ddY .dashed-line{stroke-dasharray:3}#mermaid-svg-h3IlYGkht9B17ddY #compositionStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-h3IlYGkht9B17ddY #compositionEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-h3IlYGkht9B17ddY #aggregationStart{fill:#ECECFF;stroke:#9370db;stroke-width:1}#mermaid-svg-h3IlYGkht9B17ddY #aggregationEnd{fill:#ECECFF;stroke:#9370db;stroke-width:1}#mermaid-svg-h3IlYGkht9B17ddY #dependencyStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-h3IlYGkht9B17ddY #dependencyEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-h3IlYGkht9B17ddY #extensionStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-h3IlYGkht9B17ddY #extensionEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-h3IlYGkht9B17ddY .commit-id,#mermaid-svg-h3IlYGkht9B17ddY .commit-msg,#mermaid-svg-h3IlYGkht9B17ddY .branch-label{fill:lightgrey;color:lightgrey;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-h3IlYGkht9B17ddY .pieTitleText{text-anchor:middle;font-size:25px;fill:#000;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-h3IlYGkht9B17ddY .slice{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-h3IlYGkht9B17ddY g.stateGroup text{fill:#9370db;stroke:none;font-size:10px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-h3IlYGkht9B17ddY g.stateGroup text{fill:#9370db;fill:#333;stroke:none;font-size:10px}#mermaid-svg-h3IlYGkht9B17ddY g.statediagram-cluster .cluster-label text{fill:#333}#mermaid-svg-h3IlYGkht9B17ddY g.stateGroup .state-title{font-weight:bolder;fill:#000}#mermaid-svg-h3IlYGkht9B17ddY g.stateGroup rect{fill:#ECECFF;stroke:#9370db}#mermaid-svg-h3IlYGkht9B17ddY g.stateGroup line{stroke:#9370db;stroke-width:1}#mermaid-svg-h3IlYGkht9B17ddY .transition{stroke:#9370db;stroke-width:1;fill:none}#mermaid-svg-h3IlYGkht9B17ddY .stateGroup .composit{fill:white;border-bottom:1px}#mermaid-svg-h3IlYGkht9B17ddY .stateGroup .alt-composit{fill:#e0e0e0;border-bottom:1px}#mermaid-svg-h3IlYGkht9B17ddY .state-note{stroke:#aa3;fill:#fff5ad}#mermaid-svg-h3IlYGkht9B17ddY .state-note text{fill:black;stroke:none;font-size:10px}#mermaid-svg-h3IlYGkht9B17ddY .stateLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.7}#mermaid-svg-h3IlYGkht9B17ddY .edgeLabel text{fill:#333}#mermaid-svg-h3IlYGkht9B17ddY .stateLabel text{fill:#000;font-size:10px;font-weight:bold;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-h3IlYGkht9B17ddY .node circle.state-start{fill:black;stroke:black}#mermaid-svg-h3IlYGkht9B17ddY .node circle.state-end{fill:black;stroke:white;stroke-width:1.5}#mermaid-svg-h3IlYGkht9B17ddY #statediagram-barbEnd{fill:#9370db}#mermaid-svg-h3IlYGkht9B17ddY .statediagram-cluster rect{fill:#ECECFF;stroke:#9370db;stroke-width:1px}#mermaid-svg-h3IlYGkht9B17ddY .statediagram-cluster rect.outer{rx:5px;ry:5px}#mermaid-svg-h3IlYGkht9B17ddY .statediagram-state .divider{stroke:#9370db}#mermaid-svg-h3IlYGkht9B17ddY .statediagram-state .title-state{rx:5px;ry:5px}#mermaid-svg-h3IlYGkht9B17ddY .statediagram-cluster.statediagram-cluster .inner{fill:white}#mermaid-svg-h3IlYGkht9B17ddY .statediagram-cluster.statediagram-cluster-alt .inner{fill:#e0e0e0}#mermaid-svg-h3IlYGkht9B17ddY .statediagram-cluster .inner{rx:0;ry:0}#mermaid-svg-h3IlYGkht9B17ddY .statediagram-state rect.basic{rx:5px;ry:5px}#mermaid-svg-h3IlYGkht9B17ddY .statediagram-state rect.divider{stroke-dasharray:10,10;fill:#efefef}#mermaid-svg-h3IlYGkht9B17ddY .note-edge{stroke-dasharray:5}#mermaid-svg-h3IlYGkht9B17ddY .statediagram-note rect{fill:#fff5ad;stroke:#aa3;stroke-width:1px;rx:0;ry:0}:root{--mermaid-font-family: '"trebuchet ms", verdana, arial';--mermaid-font-family: "Comic Sans MS", "Comic Sans", cursive}#mermaid-svg-h3IlYGkht9B17ddY .error-icon{fill:#522}#mermaid-svg-h3IlYGkht9B17ddY .error-text{fill:#522;stroke:#522}#mermaid-svg-h3IlYGkht9B17ddY .edge-thickness-normal{stroke-width:2px}#mermaid-svg-h3IlYGkht9B17ddY .edge-thickness-thick{stroke-width:3.5px}#mermaid-svg-h3IlYGkht9B17ddY .edge-pattern-solid{stroke-dasharray:0}#mermaid-svg-h3IlYGkht9B17ddY .edge-pattern-dashed{stroke-dasharray:3}#mermaid-svg-h3IlYGkht9B17ddY .edge-pattern-dotted{stroke-dasharray:2}#mermaid-svg-h3IlYGkht9B17ddY .marker{fill:#333}#mermaid-svg-h3IlYGkht9B17ddY .marker.cross{stroke:#333}:root { --mermaid-font-family: "trebuchet ms", verdana, arial;}#mermaid-svg-h3IlYGkht9B17ddY {color: rgba(0, 0, 0, 0.75);font: ;}

60%

50%

40%

50%

设At−1A_{t - 1}At−1、Bt−1B_{t - 1}Bt−1为当前状态的前一个状态，AtA_tAt、BtB_tBt为当前状态的前一个状态，那么可以列出状态转移矩阵：

At−1Bt−1At0.40.5Bt0.60.5\begin{matrix} & A_{t - 1} & B_{t - 1}\\ A_t & 0.4 & 0.5\\ B_t & 0.6 & 0.5 \end{matrix} AtBtAt−10.40.6Bt−10.50.5

实际上，一般将状态转移矩阵直接写为：

[0.40.50.60.5]\begin{bmatrix} 0.4 & 0.5\\ 0.6 & 0.5 \end{bmatrix} [0.40.60.50.5]

当我们进行状态概率分布推演的时候，假设前一个状态(假设为初始状态)选择AAA，则将当前的状态概率分布记作：

[10]T\begin{bmatrix} 1 & 0\\ \end{bmatrix}^T [10]T

将其与状态转移矩阵相乘，得到当前状态(初始状态的下一个状态)为：

[0.40.50.60.5]×[10]T=[0.40.6]T\begin{bmatrix} 0.4 & 0.5\\ 0.6 & 0.5 \end{bmatrix} \times \begin{bmatrix} 1 & 0\\ \end{bmatrix}^T= \begin{bmatrix} 0.4 & 0.6\\ \end{bmatrix}^T [0.40.60.50.5]×[10]T=[0.40.6]T

相似的，我们继续左乘状态转移矩阵，可得：

[0.40.50.60.5]2×[10]T=[0.460.54]T\begin{bmatrix} 0.4 & 0.5\\ 0.6 & 0.5 \end{bmatrix}^2 \times \begin{bmatrix} 1 & 0\\ \end{bmatrix}^T= \begin{bmatrix} 0.46 & 0.54\\ \end{bmatrix}^T [0.40.60.50.5]2×[10]T=[0.460.54]T

如果我们反复进行状态推演，可以发现，在经过有限次矩阵左乘后，结果将不再发生改变，即：

[0.40.50.60.5]n×[10]T=[0.4545450.545455]T\begin{bmatrix} 0.4 & 0.5\\ 0.6 & 0.5 \end{bmatrix}^n \times \begin{bmatrix} 1 & 0\\ \end{bmatrix}^T= \begin{bmatrix} 0.454545 & 0.545455\\ \end{bmatrix}^T [0.40.60.50.5]n×[10]T=[0.4545450.545455]T

我们再设初始状态选择BBB，进行如上的状态推演，可以得到：

[0.40.50.60.5]n×[01]T=[0.4545450.545455]T\begin{bmatrix} 0.4 & 0.5\\ 0.6 & 0.5 \end{bmatrix}^n \times \begin{bmatrix} 0 & 1\\ \end{bmatrix}^T= \begin{bmatrix} 0.454545 & 0.545455\\ \end{bmatrix}^T [0.40.60.50.5]n×[01]T=[0.4545450.545455]T

容易发现：初始选择AAA或BBB，最终的状态概率分布式相同的。

此时的状态概率分布即为稳态分布(Steady state distribution)，且在该例子中，稳态分布是唯一的。

同时我们发现：马尔可夫链模型的状态转移矩阵收敛到的稳定概率分布与初始状态概率分布无关。

(2).稳态推导

不难发现，设转移矩阵为MMM，则马尔可夫链的稳态分布满足：
M×X=XM \times X = X M×X=X
那么我们可以通过变换：
(M−E)×X=0(M - E) \times X = 0 (M−E)×X=0
然后只需要求解XXX便可得到稳态分布。

(3).遍历性、稳态唯一性问题

但实际上，并不是所有马尔可夫链都具有唯一的稳态分布，比如如下所示的过程：

#mermaid-svg-nYWSXXzkcrAI0Vkt .label{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);fill:#333;color:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .label text{fill:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .node rect,#mermaid-svg-nYWSXXzkcrAI0Vkt .node circle,#mermaid-svg-nYWSXXzkcrAI0Vkt .node ellipse,#mermaid-svg-nYWSXXzkcrAI0Vkt .node polygon,#mermaid-svg-nYWSXXzkcrAI0Vkt .node path{fill:#ECECFF;stroke:#9370db;stroke-width:1px}#mermaid-svg-nYWSXXzkcrAI0Vkt .node .label{text-align:center;fill:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .node.clickable{cursor:pointer}#mermaid-svg-nYWSXXzkcrAI0Vkt .arrowheadPath{fill:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .edgePath .path{stroke:#333;stroke-width:1.5px}#mermaid-svg-nYWSXXzkcrAI0Vkt .flowchart-link{stroke:#333;fill:none}#mermaid-svg-nYWSXXzkcrAI0Vkt .edgeLabel{background-color:#e8e8e8;text-align:center}#mermaid-svg-nYWSXXzkcrAI0Vkt .edgeLabel rect{opacity:0.9}#mermaid-svg-nYWSXXzkcrAI0Vkt .edgeLabel span{color:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .cluster rect{fill:#ffffde;stroke:#aa3;stroke-width:1px}#mermaid-svg-nYWSXXzkcrAI0Vkt .cluster text{fill:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt div.mermaidTooltip{position:absolute;text-align:center;max-width:200px;padding:2px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);font-size:12px;background:#ffffde;border:1px solid #aa3;border-radius:2px;pointer-events:none;z-index:100}#mermaid-svg-nYWSXXzkcrAI0Vkt .actor{stroke:#ccf;fill:#ECECFF}#mermaid-svg-nYWSXXzkcrAI0Vkt text.actor>tspan{fill:#000;stroke:none}#mermaid-svg-nYWSXXzkcrAI0Vkt .actor-line{stroke:grey}#mermaid-svg-nYWSXXzkcrAI0Vkt .messageLine0{stroke-width:1.5;stroke-dasharray:none;stroke:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .messageLine1{stroke-width:1.5;stroke-dasharray:2, 2;stroke:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt #arrowhead path{fill:#333;stroke:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .sequenceNumber{fill:#fff}#mermaid-svg-nYWSXXzkcrAI0Vkt #sequencenumber{fill:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt #crosshead path{fill:#333;stroke:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .messageText{fill:#333;stroke:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .labelBox{stroke:#ccf;fill:#ECECFF}#mermaid-svg-nYWSXXzkcrAI0Vkt .labelText,#mermaid-svg-nYWSXXzkcrAI0Vkt .labelText>tspan{fill:#000;stroke:none}#mermaid-svg-nYWSXXzkcrAI0Vkt .loopText,#mermaid-svg-nYWSXXzkcrAI0Vkt .loopText>tspan{fill:#000;stroke:none}#mermaid-svg-nYWSXXzkcrAI0Vkt .loopLine{stroke-width:2px;stroke-dasharray:2, 2;stroke:#ccf;fill:#ccf}#mermaid-svg-nYWSXXzkcrAI0Vkt .note{stroke:#aa3;fill:#fff5ad}#mermaid-svg-nYWSXXzkcrAI0Vkt .noteText,#mermaid-svg-nYWSXXzkcrAI0Vkt .noteText>tspan{fill:#000;stroke:none}#mermaid-svg-nYWSXXzkcrAI0Vkt .activation0{fill:#f4f4f4;stroke:#666}#mermaid-svg-nYWSXXzkcrAI0Vkt .activation1{fill:#f4f4f4;stroke:#666}#mermaid-svg-nYWSXXzkcrAI0Vkt .activation2{fill:#f4f4f4;stroke:#666}#mermaid-svg-nYWSXXzkcrAI0Vkt .mermaid-main-font{font-family:"trebuchet ms", verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-nYWSXXzkcrAI0Vkt .section{stroke:none;opacity:0.2}#mermaid-svg-nYWSXXzkcrAI0Vkt .section0{fill:rgba(102,102,255,0.49)}#mermaid-svg-nYWSXXzkcrAI0Vkt .section2{fill:#fff400}#mermaid-svg-nYWSXXzkcrAI0Vkt .section1,#mermaid-svg-nYWSXXzkcrAI0Vkt .section3{fill:#fff;opacity:0.2}#mermaid-svg-nYWSXXzkcrAI0Vkt .sectionTitle0{fill:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .sectionTitle1{fill:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .sectionTitle2{fill:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .sectionTitle3{fill:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .sectionTitle{text-anchor:start;font-size:11px;text-height:14px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-nYWSXXzkcrAI0Vkt .grid .tick{stroke:#d3d3d3;opacity:0.8;shape-rendering:crispEdges}#mermaid-svg-nYWSXXzkcrAI0Vkt .grid .tick text{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-nYWSXXzkcrAI0Vkt .grid path{stroke-width:0}#mermaid-svg-nYWSXXzkcrAI0Vkt .today{fill:none;stroke:red;stroke-width:2px}#mermaid-svg-nYWSXXzkcrAI0Vkt .task{stroke-width:2}#mermaid-svg-nYWSXXzkcrAI0Vkt .taskText{text-anchor:middle;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-nYWSXXzkcrAI0Vkt .taskText:not([font-size]){font-size:11px}#mermaid-svg-nYWSXXzkcrAI0Vkt .taskTextOutsideRight{fill:#000;text-anchor:start;font-size:11px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-nYWSXXzkcrAI0Vkt .taskTextOutsideLeft{fill:#000;text-anchor:end;font-size:11px}#mermaid-svg-nYWSXXzkcrAI0Vkt .task.clickable{cursor:pointer}#mermaid-svg-nYWSXXzkcrAI0Vkt .taskText.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-nYWSXXzkcrAI0Vkt .taskTextOutsideLeft.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-nYWSXXzkcrAI0Vkt .taskTextOutsideRight.clickable{cursor:pointer;fill:#003163 !important;font-weight:bold}#mermaid-svg-nYWSXXzkcrAI0Vkt .taskText0,#mermaid-svg-nYWSXXzkcrAI0Vkt .taskText1,#mermaid-svg-nYWSXXzkcrAI0Vkt .taskText2,#mermaid-svg-nYWSXXzkcrAI0Vkt .taskText3{fill:#fff}#mermaid-svg-nYWSXXzkcrAI0Vkt .task0,#mermaid-svg-nYWSXXzkcrAI0Vkt .task1,#mermaid-svg-nYWSXXzkcrAI0Vkt .task2,#mermaid-svg-nYWSXXzkcrAI0Vkt .task3{fill:#8a90dd;stroke:#534fbc}#mermaid-svg-nYWSXXzkcrAI0Vkt .taskTextOutside0,#mermaid-svg-nYWSXXzkcrAI0Vkt .taskTextOutside2{fill:#000}#mermaid-svg-nYWSXXzkcrAI0Vkt .taskTextOutside1,#mermaid-svg-nYWSXXzkcrAI0Vkt .taskTextOutside3{fill:#000}#mermaid-svg-nYWSXXzkcrAI0Vkt .active0,#mermaid-svg-nYWSXXzkcrAI0Vkt .active1,#mermaid-svg-nYWSXXzkcrAI0Vkt .active2,#mermaid-svg-nYWSXXzkcrAI0Vkt .active3{fill:#bfc7ff;stroke:#534fbc}#mermaid-svg-nYWSXXzkcrAI0Vkt .activeText0,#mermaid-svg-nYWSXXzkcrAI0Vkt .activeText1,#mermaid-svg-nYWSXXzkcrAI0Vkt .activeText2,#mermaid-svg-nYWSXXzkcrAI0Vkt .activeText3{fill:#000 !important}#mermaid-svg-nYWSXXzkcrAI0Vkt .done0,#mermaid-svg-nYWSXXzkcrAI0Vkt .done1,#mermaid-svg-nYWSXXzkcrAI0Vkt .done2,#mermaid-svg-nYWSXXzkcrAI0Vkt .done3{stroke:grey;fill:#d3d3d3;stroke-width:2}#mermaid-svg-nYWSXXzkcrAI0Vkt .doneText0,#mermaid-svg-nYWSXXzkcrAI0Vkt .doneText1,#mermaid-svg-nYWSXXzkcrAI0Vkt .doneText2,#mermaid-svg-nYWSXXzkcrAI0Vkt .doneText3{fill:#000 !important}#mermaid-svg-nYWSXXzkcrAI0Vkt .crit0,#mermaid-svg-nYWSXXzkcrAI0Vkt .crit1,#mermaid-svg-nYWSXXzkcrAI0Vkt .crit2,#mermaid-svg-nYWSXXzkcrAI0Vkt .crit3{stroke:#f88;fill:red;stroke-width:2}#mermaid-svg-nYWSXXzkcrAI0Vkt .activeCrit0,#mermaid-svg-nYWSXXzkcrAI0Vkt .activeCrit1,#mermaid-svg-nYWSXXzkcrAI0Vkt .activeCrit2,#mermaid-svg-nYWSXXzkcrAI0Vkt .activeCrit3{stroke:#f88;fill:#bfc7ff;stroke-width:2}#mermaid-svg-nYWSXXzkcrAI0Vkt .doneCrit0,#mermaid-svg-nYWSXXzkcrAI0Vkt .doneCrit1,#mermaid-svg-nYWSXXzkcrAI0Vkt .doneCrit2,#mermaid-svg-nYWSXXzkcrAI0Vkt .doneCrit3{stroke:#f88;fill:#d3d3d3;stroke-width:2;cursor:pointer;shape-rendering:crispEdges}#mermaid-svg-nYWSXXzkcrAI0Vkt .milestone{transform:rotate(45deg) scale(0.8, 0.8)}#mermaid-svg-nYWSXXzkcrAI0Vkt .milestoneText{font-style:italic}#mermaid-svg-nYWSXXzkcrAI0Vkt .doneCritText0,#mermaid-svg-nYWSXXzkcrAI0Vkt .doneCritText1,#mermaid-svg-nYWSXXzkcrAI0Vkt .doneCritText2,#mermaid-svg-nYWSXXzkcrAI0Vkt .doneCritText3{fill:#000 !important}#mermaid-svg-nYWSXXzkcrAI0Vkt .activeCritText0,#mermaid-svg-nYWSXXzkcrAI0Vkt .activeCritText1,#mermaid-svg-nYWSXXzkcrAI0Vkt .activeCritText2,#mermaid-svg-nYWSXXzkcrAI0Vkt .activeCritText3{fill:#000 !important}#mermaid-svg-nYWSXXzkcrAI0Vkt .titleText{text-anchor:middle;font-size:18px;fill:#000;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-nYWSXXzkcrAI0Vkt g.classGroup text{fill:#9370db;stroke:none;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family);font-size:10px}#mermaid-svg-nYWSXXzkcrAI0Vkt g.classGroup text .title{font-weight:bolder}#mermaid-svg-nYWSXXzkcrAI0Vkt g.clickable{cursor:pointer}#mermaid-svg-nYWSXXzkcrAI0Vkt g.classGroup rect{fill:#ECECFF;stroke:#9370db}#mermaid-svg-nYWSXXzkcrAI0Vkt g.classGroup line{stroke:#9370db;stroke-width:1}#mermaid-svg-nYWSXXzkcrAI0Vkt .classLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.5}#mermaid-svg-nYWSXXzkcrAI0Vkt .classLabel .label{fill:#9370db;font-size:10px}#mermaid-svg-nYWSXXzkcrAI0Vkt .relation{stroke:#9370db;stroke-width:1;fill:none}#mermaid-svg-nYWSXXzkcrAI0Vkt .dashed-line{stroke-dasharray:3}#mermaid-svg-nYWSXXzkcrAI0Vkt #compositionStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-nYWSXXzkcrAI0Vkt #compositionEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-nYWSXXzkcrAI0Vkt #aggregationStart{fill:#ECECFF;stroke:#9370db;stroke-width:1}#mermaid-svg-nYWSXXzkcrAI0Vkt #aggregationEnd{fill:#ECECFF;stroke:#9370db;stroke-width:1}#mermaid-svg-nYWSXXzkcrAI0Vkt #dependencyStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-nYWSXXzkcrAI0Vkt #dependencyEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-nYWSXXzkcrAI0Vkt #extensionStart{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-nYWSXXzkcrAI0Vkt #extensionEnd{fill:#9370db;stroke:#9370db;stroke-width:1}#mermaid-svg-nYWSXXzkcrAI0Vkt .commit-id,#mermaid-svg-nYWSXXzkcrAI0Vkt .commit-msg,#mermaid-svg-nYWSXXzkcrAI0Vkt .branch-label{fill:lightgrey;color:lightgrey;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-nYWSXXzkcrAI0Vkt .pieTitleText{text-anchor:middle;font-size:25px;fill:#000;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-nYWSXXzkcrAI0Vkt .slice{font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-nYWSXXzkcrAI0Vkt g.stateGroup text{fill:#9370db;stroke:none;font-size:10px;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-nYWSXXzkcrAI0Vkt g.stateGroup text{fill:#9370db;fill:#333;stroke:none;font-size:10px}#mermaid-svg-nYWSXXzkcrAI0Vkt g.statediagram-cluster .cluster-label text{fill:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt g.stateGroup .state-title{font-weight:bolder;fill:#000}#mermaid-svg-nYWSXXzkcrAI0Vkt g.stateGroup rect{fill:#ECECFF;stroke:#9370db}#mermaid-svg-nYWSXXzkcrAI0Vkt g.stateGroup line{stroke:#9370db;stroke-width:1}#mermaid-svg-nYWSXXzkcrAI0Vkt .transition{stroke:#9370db;stroke-width:1;fill:none}#mermaid-svg-nYWSXXzkcrAI0Vkt .stateGroup .composit{fill:white;border-bottom:1px}#mermaid-svg-nYWSXXzkcrAI0Vkt .stateGroup .alt-composit{fill:#e0e0e0;border-bottom:1px}#mermaid-svg-nYWSXXzkcrAI0Vkt .state-note{stroke:#aa3;fill:#fff5ad}#mermaid-svg-nYWSXXzkcrAI0Vkt .state-note text{fill:black;stroke:none;font-size:10px}#mermaid-svg-nYWSXXzkcrAI0Vkt .stateLabel .box{stroke:none;stroke-width:0;fill:#ECECFF;opacity:0.7}#mermaid-svg-nYWSXXzkcrAI0Vkt .edgeLabel text{fill:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .stateLabel text{fill:#000;font-size:10px;font-weight:bold;font-family:'trebuchet ms', verdana, arial;font-family:var(--mermaid-font-family)}#mermaid-svg-nYWSXXzkcrAI0Vkt .node circle.state-start{fill:black;stroke:black}#mermaid-svg-nYWSXXzkcrAI0Vkt .node circle.state-end{fill:black;stroke:white;stroke-width:1.5}#mermaid-svg-nYWSXXzkcrAI0Vkt #statediagram-barbEnd{fill:#9370db}#mermaid-svg-nYWSXXzkcrAI0Vkt .statediagram-cluster rect{fill:#ECECFF;stroke:#9370db;stroke-width:1px}#mermaid-svg-nYWSXXzkcrAI0Vkt .statediagram-cluster rect.outer{rx:5px;ry:5px}#mermaid-svg-nYWSXXzkcrAI0Vkt .statediagram-state .divider{stroke:#9370db}#mermaid-svg-nYWSXXzkcrAI0Vkt .statediagram-state .title-state{rx:5px;ry:5px}#mermaid-svg-nYWSXXzkcrAI0Vkt .statediagram-cluster.statediagram-cluster .inner{fill:white}#mermaid-svg-nYWSXXzkcrAI0Vkt .statediagram-cluster.statediagram-cluster-alt .inner{fill:#e0e0e0}#mermaid-svg-nYWSXXzkcrAI0Vkt .statediagram-cluster .inner{rx:0;ry:0}#mermaid-svg-nYWSXXzkcrAI0Vkt .statediagram-state rect.basic{rx:5px;ry:5px}#mermaid-svg-nYWSXXzkcrAI0Vkt .statediagram-state rect.divider{stroke-dasharray:10,10;fill:#efefef}#mermaid-svg-nYWSXXzkcrAI0Vkt .note-edge{stroke-dasharray:5}#mermaid-svg-nYWSXXzkcrAI0Vkt .statediagram-note rect{fill:#fff5ad;stroke:#aa3;stroke-width:1px;rx:0;ry:0}:root{--mermaid-font-family: '"trebuchet ms", verdana, arial';--mermaid-font-family: "Comic Sans MS", "Comic Sans", cursive}#mermaid-svg-nYWSXXzkcrAI0Vkt .error-icon{fill:#522}#mermaid-svg-nYWSXXzkcrAI0Vkt .error-text{fill:#522;stroke:#522}#mermaid-svg-nYWSXXzkcrAI0Vkt .edge-thickness-normal{stroke-width:2px}#mermaid-svg-nYWSXXzkcrAI0Vkt .edge-thickness-thick{stroke-width:3.5px}#mermaid-svg-nYWSXXzkcrAI0Vkt .edge-pattern-solid{stroke-dasharray:0}#mermaid-svg-nYWSXXzkcrAI0Vkt .edge-pattern-dashed{stroke-dasharray:3}#mermaid-svg-nYWSXXzkcrAI0Vkt .edge-pattern-dotted{stroke-dasharray:2}#mermaid-svg-nYWSXXzkcrAI0Vkt .marker{fill:#333}#mermaid-svg-nYWSXXzkcrAI0Vkt .marker.cross{stroke:#333}:root { --mermaid-font-family: "trebuchet ms", verdana, arial;}#mermaid-svg-nYWSXXzkcrAI0Vkt {color: rgba(0, 0, 0, 0.75);font: ;}

50%

不同的初始状态可以导致不同的稳态：
A1=[010]T→[010]TA2=[001]T→[001]TA_1=\begin{bmatrix} 0 & 1 & 0\\ \end{bmatrix}^T \rightarrow \begin{bmatrix} 0 & 1 & 0\\ \end{bmatrix}^T\\ A_2 = \begin{bmatrix} 0 & 0 & 1\\ \end{bmatrix}^T \rightarrow \begin{bmatrix} 0 & 0 & 1\\ \end{bmatrix}^T A1=[010]T→[010]TA2=[001]T→[001]T
那么马尔科夫链的稳态分布在何种情况下唯一呢？首先需要引入马尔可夫链的遍历性

如果马尔可夫链的全部状态节点满足以下三条件：

常返性：如果从某个节点XXX出发，在一定条件下通过有限次状态转移能够返回XXX，那么称节点XXX具有常返性
非周期性：马尔科夫链中的某一状态节点，可以经过KaTeX parse error: Undefined control sequence: \and at position 18: …\times N(N > 0 \̲a̲n̲d̲ ̲K = 2,3,4 \dots…个周期返回原点，那么就称该状态满足周期性。
两两联通：状态AAA和状态BBB之间具有直接往返路径。

那么该马尔科夫链具有遍历性，且具有唯一的稳态分布。