重学statistics，Cha3 Descriptive Statistics: numerical measures

这一章学什么？

1. 一些很简单的统计数据：Mean, Median, Mode, Percentiles, Variance, Standard Deviation

2. 用数据描述图形的形状：z-score, skewness

3. 最后的两个变量之间的 Covariance 和 Correlation Coefficient。

这三个方面都是为以后的章节打下基础。

If the measures are computed for data from a sample, they are called sample statistics.

If the measures are computed for data from a population, they are called population parameters.

3.1 Measure of Location

Sample Mean :

Population Mean：

Median：Arrange the data in ascending order (smallest value to largest value).

(a) For an odd number of observations, the median is the middle value.

(b) For an even number of observations, the median is the average of the two middle values.

Mode：The mode is the value that occurs with greatest frequency.

Percentiles: The pth percentile is a value such that at least p percent of the observations are less than or equal to this value and at least (100-p) percent of the observations are greater than or equal to this value. 第 p 的 percentiles 是至少 p%的数据是小于等于这个值的，而且至少100-p 的值是大于或者等于这个值。

怎么计算 Percentile ？

Step 1. Arrange the data in ascending order (smallest value to largest value).

Step 2. Compute an index i= p/100 *n where p is the percentile of interest and n is the number of observations.

Step 3. (a) If i is not an integer, round up. The next integer greater than i denotes the position of the pth percentile.

(b) If i is an integer, the pth percentile is the average of the values in po- sitions i and i+1.

如果计算出 i 不是一个整数，那么就 round up,下一个数字就是第 p 的 pencentile。

如果计算出 i 是一个整数，那么第 p 的 pencentile 就是第 i个数据和第 i+1个数据的平均数

Quartiles: Q1 = first quartile, or 25th percentile Q2 = second quartile, or 50th percentile (also the median) Q3 = third quartile, or 75th percentile

3.2 Measures of Variability

Range = Largest Value - Smallest Value

Interquartile Range = IQR = Q3 - Q1

Population Variance: σ² = Σ ( X_i - X )² / N

Sample Variance: s² = Σ ( x_i - x )² / ( n - 1 )

Why Sample Variance is n-1?

因为用 n 的时候，计算出来的方差会偏小，所以用 n-1，具体过程还可以用数学证明，具体见：https://www.zhihu.com/question/20099757

Sample standard deviation： s = sqrt [ Σ ( x_i - x )² / ( n - 1 ) ]

Population standard deviation： σ = sqrt [ Σ ( X_i - X )² / N ]

Coefficient of Variation = (standard deviation/ Mean） * 100% 这个数字可以用来比较不同数据集中数据的变化程度 (Variability)

3.3 Measures of Distribution Shape, Relative Location, and Detective Outliers

1. Distribution Shape

Skewed Left (Skewness <0 ), median > mean

Skewed Right (Skewness >0), mean > median

Symmetric (Skewness = 0), mean = median

2. Z-score:

determine how far a particular value is from the mean Zi = (Xi -X)/s

The z-score, zi , can be interpreted as the number of standard deviations xi is from the meanX

3. CHEBYSHEV’S THEOREM

At least (1-1/z2) of the data values must be within z standard deviations of the mean, where z is any value greater than 1.

4. Empirical Rule

For data having a bell-shaped distribution:

• Approximately 68% of the data values will be within one standard deviation of the mean.

• Approximately 95% of the data values will be within two standard deviations of the mean.

• Almost all of the data values will be within three standard deviations of the mean.

5. Detecting Outliers

we recommend treating any data value with a z-score less than 3 or greater than 3 as an outlier.

3.4 Exploratory Data Analysis

1. Five Number Summary

Smallest value
First quartile (Q1)
Median (Q2)
Third quartile (Q3)
Largest value

2. Box Plot

3.5 Measures of Association Between two variables

Sample Covariance :

Population Covariance:

第一象限和第三象限才是正向的

Covariance 也有局限性，第一，数值结果会变得很大，最好能变成[-1,1]的区间，第二，当把 x 由米转成厘米的时候，结果应该是一样的。

所以要用到：Correlation Coefficient

Correlation Coefficient: 这个的范围是从 -1到1，The correlation coefficient ranges from 1 to 1. Values close to 1 or 1 indicate a strong linear relationship. The closer the correlation is to zero, the weaker the relationship.

3.6 The Weighted Mean and Working with Grouped Data

1. Weighted Mean

2. Grouped Data

重学statistics，Cha3 Descriptive Statistics: numerical measures相关推荐

R语言数据描述性统计（Descriptive statistics）实战：数据全局描述信息、数值数据的描述性统计（Numerical data）、离散型数据的描述性统计（Categorical）
R语言数据描述性统计(Descriptive statistics)实战:数据全局描述信息.数值数据的描述性统计(Numerical data).离散型数据的描述性统计(Categorical) 目录
概率论与数理统计 1 Overview and Descriptive Statistics(概述和描述性统计) （上篇）
概率论_1.1_1.2 1.1 Populations,Samples,and Processes Branches of Statistics the Scope of Modern Statist ...
pandas使用groupby函数和describe函数计算不同分组的描述性统计（descriptive statistics of each group in dataframe）
pandas使用groupby函数和describe函数计算不同分组的描述性统计(descriptive statistics of each group in dataframe) 目录
R语言编写自定义描述统计计算函数、使用doBy包的summaryBy函数计算不同分组（group）的描述性统计值(Descriptive statistics by group、样本个数、均值、标准)
R语言编写自定义描述统计计算函数.使用doBy包的summaryBy函数计算不同分组(group)的描述性统计值(Descriptive statistics by group using summa ...
什么是描述性统计分析( descriptive statistics)？主要包含哪些内容？
什么是描述性统计分析( descriptive statistics)?主要包含哪些内容? 描述统计是通过图表或数学方法,对数据资料进行整理.分析,并对数据的分布状态.数字特征和随机变量之间关系进行估 ...
描述统计学（descriptive statistics）
描述统计学(descriptive statistics)是一门统计学领域的学科,主要研究如何取得反映客观现象的数据,并以图表形式对所搜集的数据进行处理和显示,最终对数据的规律.特征做出综合性的描述分 ...
【笔记】重学前端-winter
本文为:winter 发布在极客时间 [重学前端]系列课程的的笔记和总结支持正版哦: https://time.geekbang.org/col... 导语如果深入进去了解,你会发现,表面上看他们 ...
重学JavaScript深入理解系列（一）
JavaScript深入理解--执行上下文(Execution Context) 定义每当控制器到达ECMAScript可执行代码的时候,控制器就进入了一个执行上下文. 执行上下文(简称:EC) 以 ...
重学前端-学习笔记-JavaScript对象
说明重学前端是程劭非(winter)在极客时间开的一个专栏,在此主要整理我的学习笔记.如有侵权,请联系我,谢谢. javascript对象特征对象具有唯一标识性:完全相同的两个对象,也不是同一个对 ...
重学hadoop技术
最近因为做了些和hadoop相关的项目(虽然主要是运维),但是这段经历让我对hadoop的实际运用有了更加深入的理解. 相比以前自学hadoop,因为没有实战场景以及良好的大数据学习氛围,现在回顾下的 ...

重学statistics，Cha3 Descriptive Statistics: numerical measures

重学statistics，Cha3 Descriptive Statistics: numerical measures相关推荐

最新文章

热门文章