UA MATH567 高维统计I 概率不等式0 Markov不等式、Chebyshev不等式与取等条件

UA MATH567 高维统计I 概率不等式0 Markov不等式、Chebyshev不等式与取等条件

Markov不等式与Chebyshev不等式

假设 g g g是一个取值为正的函数,定义
m B = inf ⁡ { g ( t ) : t ∈ B } m_B = \inf\{g(t):t \in B\} mB=inf{g(t):tB}

从而
E g ( X ) ≥ E [ g ( X ) I B ( X ) ] ≥ E [ m B I B ( X ) ] = m B P ( X ∈ B ) Eg(X) \ge E[g(X)I_B(X)] \ge E[m_B I_B(X)] = m_B P(X \in B) Eg(X)E[g(X)IB(X)]E[mBIB(X)]=mBP(XB)

如果 g g g是单调递增的函数, B = [ x , + ∞ ) B = [x,+\infty) B=[x,+),则上式可以写成
E g ( X ) ≥ g ( x ) P ( X > x ) ⇒ P ( X > x ) ≤ E [ g ( X ) ] g ( x ) Eg(X) \ge g(x) P(X >x) \Rightarrow P(X >x ) \le \frac{E[g(X)]}{g(x)} Eg(X)g(x)P(X>x)P(X>x)g(x)E[g(X)]

这个不等式被称为Markov不等式。在Markov不等式中,假设 X = ∣ Y − E Y ∣ X = |Y-EY| X=YEY g ( x ) = x 2 g(x)=x^2 g(x)=x2,则
P ( ∣ Y − E Y ∣ > x ) ≤ V a r Y x 2 , ∀ x > 0 P(|Y-EY|>x) \le \frac{VarY}{x^2},\forall x>0 P(YEY>x)x2VarY,x>0

这个不等式叫Chebyshev不等式。

Sharpness of Chebyshev

概率不等式是用来bound一些tail probability的,如果某种概率不等式定义的tail probability bound非常接近tail probability本身,就说这种概率不等式比较sharp。

我们以Chebyshev不等式为例,考虑非常简化的形式:
P ( ∣ X ∣ ≥ a ) ≤ E X 2 a 2 P(|X|\ge a) \le \frac{EX^2}{a^2} P(Xa)a2EX2

在某些情况下,Chebyshev不等式取等,这种时候Chebyshev不等式就非常sharp。我们可以构造一个特殊的分布:

X − α -\alpha α0 α \alpha α
P c / 2 c/2 c/2 1 − c 1-c 1c c / 2 c/2 c/2

可以计算 E X 2 = c α 2 EX^2=c\alpha^2 EX2=cα2,取 α = a \alpha=a α=a,显然
P ( ∣ X ∣ ≥ a ) = E X 2 a 2 P(|X|\ge a) = \frac{EX^2}{a^2} P(Xa)=a2EX2

也就是Chebyshev不等式对于随机变量 X X X而言是strictly sharp或者就是说是等式成立的。

我们可以试图抽象化这个特例,假设 P ( ∣ X ∣ ≥ a ) = c P(|X|\ge a)=c P(Xa)=c,则Chebyshev不等式的取等条件为
{ P ( ∣ X ∣ ≥ a ) = c E X 2 = c a 2 \begin{cases}P(|X|\ge a)=c \\ EX^2 = ca^2 \end{cases} {P(Xa)=cEX2=ca2

基于这组条件构造使得Chebyshev不等式取等的分布是比较容易的,下面是一种一般性的操作:

Let Y Y Y denote a r.v. on ( Ω , F , P ) (\Omega,\mathcal{F},P) (Ω,F,P) such that Y Y Y follows Bernoulli distribution with parameter b 2 / a 2 b^2/a^2 b2/a2. Let A = Y − 1 ( { 1 } ) A=Y^{-1}(\{1\}) A=Y1({1}). Construct X : Ω → R X:\Omega \to \mathbb{R} X:ΩR such that { w : ∣ X ( w ) ∣ ≥ x } = A \{w:|X(w)|\ge x\}=A {w:X(w)x}=A and { w : ∣ X ( w ) ∣ < x } = A C \{w:|X(w)|< x\}=A^C {w:X(w)<x}=AC, where x x x can be arbitrary number. We see P ( ∣ X ∣ ≥ x ) = b 2 / a 2 P(|X|\geq x)= b^2/a^2 P(Xx)=b2/a2. Let’s show X X X is a r.v. Let E = ( − ∞ , − a ] ∪ [ a , + ∞ ) E=(-\infty,-a]\cup[a,+\infty) E=(,a][a,+), ∀ B ∈ B ( R ) \forall B \in \mathcal{B}(\mathbb{R}) BB(R), B = ( B ∩ E ) ∪ ( B ∩ E C ) B=(B\cap E)\cup(B\cap E^C) B=(BE)(BEC).
X − 1 ( B ) = X − 1 ( ( B ∩ E ) ∪ ( B ∩ E C ) ) = X − 1 ( B ∩ E ) ∪ X − 1 ( B ∩ E C ) X^{-1}(B)=X^{-1}((B\cap E)\cup(B\cap E^C))=X^{-1}(B\cap E)\cup X^{-1}(B\cap E^C) X1(B)=X1((BE)(BEC))=X1(BE)X1(BEC)

Notice X − 1 ( B ∩ E ) ⊂ A , X − 1 ( B ∩ E C ) ⊂ A C X^{-1}(B\cap E) \subset A,X^{-1}(B\cap E^C) \subset A^C X1(BE)A,X1(BEC)AC, so X − 1 ( B ∩ E ) , X − 1 ( B ∩ E C ) ∈ F X^{-1}(B\cap E),X^{-1}(B\cap E^C) \in \mathcal{F} X1(BE),X1(BEC)F, and X − 1 ( B ) ∈ F X^{-1}(B) \in \mathcal{F} X1(B)F. Hence, X X X is a r.v. Assume E X 2 = c 2 < ∞ EX^2=c^2<\infty EX2=c2<. Construct Z = b c X Z=\frac{b}{c}X Z=cbX, and E Z 2 = b 2 EZ^2=b^2 EZ2=b2. Obviously, Z Z Z is a r.v.
{ w : ∣ X ∣ ≥ x } = { w : ∣ Z ∣ ≥ b c x } \{w:|X|\ge x\}=\{w:|Z| \ge \frac{b}{c}x\} {w:Xx}={w:Zcbx}

Let x = a b c x=\frac{ab}{c} x=cab, and we see Z Z Z is a r.v. such that E Z 2 = b 2 E Z^2 = b^2 EZ2=b2 and P ( ∣ Z ∣ ≥ a ) = b 2 / a 2 P(|Z|\geq a)= b^2/a^2 P(Za)=b2/a2.

以上的讨论是针对小样本Chebyshev不等式的表现的,我们同样需要考虑大样本时Chebyshev不等式的效果,然而事实上我们可以证明:
P ( ∣ X ∣ ≥ a ) = o ( E X 2 / a 2 ) P(|X| \ge a) = o(EX^2/a^2) P(Xa)=o(EX2/a2)

因此大样本时Chebyshev bound diminish的速度远远赶不上tail probability diminish的速度,因此样本时Chebyshev不等式效果并不好。最后附一个简单证明:

Let U = a 2 1 ∣ X ∣ ≥ a , V = X 2 U=a^2\textbf{1}_{|X|\ge a},V=X^2 U=a21Xa,V=X2. Definitely we have ∣ U ∣ ≤ V |U|\le V UV and E V < ∞ EV<\infty EV<. Note that U → 0   a . s . U \to 0\ a.s. U0 a.s. as a → ∞ a\to \infty a. Then, by dominated convergence theorem, E U → 0 EU \to 0 EU0 indicating
lim ⁡ a → ∞ E U = lim ⁡ a → ∞ a 2 P ( ∣ X ∣ ≥ a ) = 0 ⇒ lim ⁡ a → ∞ a 2 P ( ∣ X ∣ ≥ a ) / E X 2   =   0 \lim_{a \to \infty} EU = \lim_{a\to \infty} a^2P(|X| \ge a) = 0 \Rightarrow \lim_{a\rightarrow\infty} a^2P(|X|\geq a)/EX^2 \ = \ 0 alimEU=alima2P(Xa)=0alima2P(Xa)/EX2 = 0

相关推荐
©️2020 CSDN 皮肤主题: 撸撸猫 设计师:马嘣嘣 返回首页