贝叶斯(Baysian)分类器[1]是一种理论上比较简单的分类器。但是结合不同的网络结构和概率模形，它又可以演化成非常复杂的分类体系。本短文主要演示Baysian + Gaussian如何解两类问题。

其中，分母部分主要用于归一化。p(y)为先验概率(prior)，　p(x|y)为条件概率或称之为类概率密度(即已知x是哪一类的情况下p(x)的概率密度)。　在本文中，假设p(x|y)是高斯分布，即[2]：

而p(y)则采用伯努利(Bernoulli)分布[3]:

其中最大似然估计后得到的\eta即为第０类中训练样本的个数占总样本数的百分比。　求得五个参数后，就可能通过比较后验概率得到任意样本x的类别：

.

当f(x) 大于０时即表示

,

此时把样本x归为第０类，否则归为第１类。

下面通过Matlab程序进行演示：

训练的代码：

function [model_pos,model_neg ] = FindGuassianModel( x,y )

%FINDGUASSIANMODULE Summary of this function goes here

%   Detailed explanation goes here

x_pos = x(:,y==1);

model_pos.mu = mean(x_pos,2);

model_pos.var = cov(x_pos');

model_pos.prior = length(x_pos)/length(x);

x_neg = x(:,y~=1);

model_neg.mu = mean(x_neg,2);

model_neg.var = cov(x_neg');

model_neg.prior = length(x_neg)/length(x);

end

计算分类误差：

function [err,h] = FindModelError(model_pos,model_neg, x,y )

%FINDGUASSIANMODULE Summary of this function goes here

%   Detailed explanation goes here

mu1 = model_pos.mu;

sigma1 = model_pos.var;

p1 = model_pos.prior;

mu2 = model_neg.mu;

sigma2 = model_neg.var;

p2 = model_neg.prior;

bias = 0.5*log(det(sigma2))-0.5*log(det(sigma1))+log(p1/p2);

err = 0;

h = zeros(size(y));

for i=1:length(y)

c = bias + 0.5*(x(:,i)-mu2)'/sigma2*(x(:,i)-mu2) - 0.5*(x(:,i)-mu1)'/sigma1*(x(:,i)-mu1);

if c > 0

h(i) = 1;

else

h(i) = -1;

end

if h(i)~=y(i)

err = err + 1;

end

end

end

演示主程序:

%%

clc;

clear;

close all;

%% generate random data

shift =3.0;

n = 2;%2 dim

sigma = 1;

N = 500;

x = [randn(n,N/2)-shift, randn(n,N/2)*sigma+shift];

y = [ones(N/2,1);-ones(N/2,1)];

%show the data

figure;

plot(x(1,1:N/2),x(2,1:N/2),'rs');

hold on;

plot(x(1,1+N/2:N),x(2,1+N/2:N),'go');

title('2d training data');

legend('Positve samples','Negative samples','Location','SouthEast');

% model fitting using maximum likelihood

[model_pos,model_neg] = FindGuassianModel(x,y);

%% test on new dataset, same distribution

n = 2;%2 dim

%y = 1./exp(-w'*x+b)

sigma = 2;

N = 500;

x = [randn(n,N/2)-shift, randn(n,N/2)*sigma+shift];

y = [ones(N/2,1);-ones(N/2,1)];figure;plot(x(1,1:N/2),x(2,1:N/2),'rs');

hold on;

plot(x(1,1+N/2:N),x(2,1+N/2:N),'go');

title('2d testing data');

hold on;

%% gaussian model as a baseline

[err,h] = FindModelError(model_pos,model_neg,x,y);

fprintf('Baysian error on test data set: %f\n',err/N);

x_pos = x(:,h==1);

x_neg = x(:,h~=1);

plot(x_pos(1,:),x_pos(2,:),'r.');

hold on;

plot(x_neg(1,:),x_neg(2,:),'g.');

legend('Positve samples','Negative samples','Positve samples as predicted','Negative samples as predicted','Location','SouthEast');

最后的测试结果：

从测试结是上看，大部分样本都能分类正确(同色的点在同色的圆或方框中)，只有0.8％的点分类错误。