stochastic sequence

每個數值都是隨機變數。所有數值形成隨機過程。

請見本站文件「stochastic number」。

noise

「雜訊」或「噪訊」。雜訊沒有明確定義。目前大家認為雜訊包含幾種意義：一、不被需要的訊號、冗餘的訊號。二、沒有規律的訊號、不可預測的訊號。三、無法擬合數學模型的訊號。

大家觀察各種真實現象，命名為各種雜訊。大家也利用浮動訊號建立數學模型，創造出各種雜訊。維基百科整理了一份列表：

例如「Gaussian white noise」：高斯白雜訊。每個隨機變數都是常態分布，其平均數（浮動中心）和變異數（浮動範圍）均相同。恰好屬於白雜訊。

noise的頻譜

隨機過程，實施「傅立葉轉換」，從時域變頻域。

普通的隨機過程：無解析解，頻譜混亂。

weakly stationary process：有解析解。兩兩的共相關數，可求得頻譜。具備傅立葉轉換的相關數學特性，諸如線性、卷積乘法對偶、能量守恆。

然而，weakly stationary process，就是每個數字幾乎一樣的數列。缺乏討論意義，也無法解決現實問題。數學家目前僅發現weakly stationary process，尚未發現更具討論意義的隨機過程。

noise的頻譜

大家仿照光譜由紅到紫的特性，嘗試分類雜訊。

 white: 振幅為常數
  grey: 振幅符合人類聽覺曲線。（不那麼白）
   red: 振幅正比於頻率倒數平方。　　以頻率對數為座標軸，漸減6dB。
  pink: 振幅正比於頻率倒數。　　　　以頻率對數為座標軸，漸減3dB。（不那麼紅）
violet: 振幅負正比於頻率倒數平方。　以頻率對數為座標軸，漸增6dB。
  blue: 振幅負正比於頻率倒數。　　　以頻率對數為座標軸，漸增3dB。（不那麼紫）

http://vellocet.com/dsp/noise/VRand.html

noise reduction（denoising）

分離並消除隨機訊號，通常稱作「去雜訊」或「降噪」。

假設雜訊是高頻，於是套用low-pass filter。轉換到頻域、刪除高頻振幅、轉換回時域。

https://www.mathworks.com/help/wavelet/ug/wavelet-denoising.html

system with noise

stochastic process

sequence           x[n] = (x[0], x[1], x[2], ...)
stochastic process {Xₙ} = {X₀, X₁, X₂, ...}

訊號的數值，從固定的改成浮動的。
一個數值從固定數字改成浮動數字（隨機變數）。
一道訊號從固定數列改成浮動數列（隨機過程）。
標記方式完全不同。自己保重。

stochastic system

stochastic system:
   all signals are stochastic processes
=> all signals are sequences with noise

                               noise       noise
                               X-E[X]      Y-E[Y]
                                 │           │
       ┌───┐            signal   ↓+  ┌───┐   ↓+  signal
X ────→│ f │────→ Y  =   E[X] ──→⊕──→│ f │──→⊕──→ E[Y]
       └───┘                    +    └───┘  +

浮動數字擁有指標。大家習慣只看平均數和變異數。
浮動數列則是有平均數數列和變異數數列。
大家習慣抽取平均數們，成為固定數字，當作訊號。
剩餘的數值們，仍是浮動數字，其平均數們全是零，當作雜訊。

noise

noise assumptions:
1. stationary process (stochastic time-invariant)
2. white (power spectrum is uniform distribution)

大家習慣假設：
一、雜訊不隨時間而變。
二、雜訊的強度頻譜平方是常數函數（均勻分布）。
三、雜訊的平均值是零、變異數是常數。

stochastic time-invariant <=> stationary process

1. zero mean & constant variance

stationary process => constant mean and variance (if exist)

現實世界當中，系統受到其他因素影響。
訊號不是固定數字，而是浮動數字。
當其他因素不隨時間而變，
那麼平均數（浮動基準）、變異數（浮動範圍）也不隨時間而變。
兩者都是常數。
順帶一提，平均數被抽取了，所以是零。

white <=> uncorrelated between samples

mutually independent process => uncorrelated
獨立則不相關。反方向不一定。

noise
1. staiotnary process
2. uncorrelated process

Gaussian noise:
1. staiotnary process
2. mutually independent process

for Gaussian distribution,
independent <=> uncorrelated

disturbance

noise: affect signal
disturbance: affect system

stochastic process

專著《Introduction to Statistical Signal Processing》。

non-stochastic process / stochastic process

訊號的數值，固定的與浮動的。

deterministic process / random process

deterministic process = evolution rule is fixed
random process = evolution rule is random

訊號的數值的數學公式，確定的和隨機的。
確定的：訊號數值有明確數學公式。甚至使用了前後數值。
隨機的：訊號數值亂七八糟。甚至看不出是否使用了前後數值。

數學公式使用了過往數值，此時稱作causal。
數學公式使用了過往數值，而且每個數值的數學公式都相同，此時形成recurrence。

constant function / time-invariant function

in K-12 mathematics,
(1) constant                = value remains unchanged
(2) constant/variable       = value is known/unknown

in mathematics,
(1) constant function       = output remains unchanged
                              by different input
(2) time-invariant function = function remains unchanged
                              by different time

in signals and systems,
(1) constant input/output   = signal remains unchanged
(2) time-invariant system   = system remains unchanged

stationary process / mutually independent process

stationary ≈ non-changing (aperiodic)
mutually independent ≈ non-regressive (acausal)
等號兩側其實不一樣，只是表面上看起來很可能是那樣。

stationary: independent between variables and time.
mutually independent: independent between variable and variable.

stationary => identically distributed
mutually independent => pairwise independent

identically distributed process = time-invariant distribution
X₁ = X₂ = X₃ = ...

stationary process = time-invariant joint distribution
X₁ = X₂ = X₃ = ...
(X₁, X₂) = (X₂, X₃) = (X₃, X₄) = ...
(X₁, X₂, X₃) = (X₂, X₃, X₄) = (X₃, X₄, X₅) = ...
      :

pairwise independent process = constant marginal distribution
                               in 2-joint distribution
X₁ = (X₁, X₂ = x) for all x
X₁ = (X₁, X₃ = x) for all x
X₁ = (X₁, X₄ = x) for all x
 :         :
X₂ = (X₂, X₁ = x) for all x
X₂ = (X₂, X₃ = x) for all x
X₂ = (X₂, X₄ = x) for all x
 :         :

mutually independent process = constant marginal distribution
                               in joint distribution
X₁ = (X₁, X₂ = x₂) for all x₂
X₁ = (X₁, X₃ = x₃) for all x₃
X₁ = (X₁, X₄ = x₄) for all x₄
 :
X₁ = (X₁, X₂ = x₂, X₃ = x₃) for all x₂, x₃
X₁ = (X₁, X₂ = x₂, X₄ = x₄) for all x₂, x₄
X₁ = (X₁, X₃ = x₃, X₄ = x₄) for all x₃, x₄
 :

https://math.stackexchange.com/questions/3269920/
https://math.stackexchange.com/questions/1920473/

independent and identically distributed process

原本定義是
i.i.d. = identically distributed & mutually independent

但是有一個定理是
i.i.d. => stationary

因此實際上是
i.i.d. = stationary & mutually independent

https://www.statlect.com/glossary/stationary-sequence

moment / coherence

moment: statistics of one random variable
coherence: statistics of two random varibles

indentical => time-invariant moment
stationary => time-invariant moment and coherence
pairwise independent => no 1st coherence
mutually independent => no higher-order coherence

nth moment
E[Xⁿ]

nth central moment
E[(X-X̄)ⁿ]   where X̄ = E[X] is mean

1st coherence
E[XY]

1st standardized coherence
    E[(X-X̄)(Y-Ȳ)]
———————————————————————
(E[(X-X̄)²]E[(Y-Ȳ)²])¹⸍²

higher-order coherence
E[(XY,YZ,XZ)]            C(3,2)
E[(AB,AC,AD,BC,BD,CD)]   C(4,2)

moment: mean
central moment: zero, variance, skewness
coherence: correlation
standardized coherence: correlation coefficient

constant moment / uncorrelated

1. stationary <=> constant moment (if moment exists)
2. independent => uncorrelated

Cauchy distribution: all moments are undefined

weakly stationary process

weakly stationary process

教科書習慣採用weakly stationary達成time-invariant autocorrelation。
但是(1)是多餘的，成為歷史共業。

identically distributed process = time-invariant distribution
stationary process = time-invariant joint distribution

stationary process:
(1) time-invariant moment (mean/variance/skewness/...)
(2) time-invariant coherence (autocorrelation/...)

stationary process example:
1. i.i.d. process
2. steady states

weakly stationary process:
(1) time-invariant 1st moment (mean)
(2) time-invariant 1st coherence (autocorrelation)
性質更弱。失去了高階動差和高階相干。

property:
(3) time-invariant 2nd central moment (variance)
(1) & (2) => (3)。證明省略。也有人將(3)直接納入定義。

weakly stationary process example:
1. white noise
2. random phase cosine wave

white noise

white noise:
power spectrum is uniform <=> no 1st coherence
(pairwise independent => no 1st coherence)

white noise in time domain:
never discovered

a sufficient but not necessary condition:
weakly stationary process with Wiener–Khinchin theorem

white noise in theory:
(1) time-invariant 1st moment (mean)
(2) time-invariant 1st coherence (autocorrelation)
(3) autocorrelation approaches zero at infinite lag
其中(2)和(3)足以實施逆向傅立葉轉換。

white noise in practice:
(1)   constant mean
      E[x[n]] = μ for all n
(1&2) constant variance
      Var[x[n]] = σ² for all n
(2&3) impulse autocorrelation (uncorrelated)
      Rx[t] = σ² δ[t]
實務上無視高階動差，只處理mean和variance。
實務上無視高階相干，只處理autocorrelation，並且視作不相關。

white noise example:
1. white Gaussian noise N(μ,σ²)
2. white uniform noise  U(a,b)

stable system

stable system

definition:
1. stable:     difference of two sequences is bounded
2. convergent: sequence approaches constant (steady state)
3. stationary: sequence is constant

theorem:
convergent => stationary at steady states

(talking about a trivial thing like a theorem)

for LTI system,

theorem:
1. stable:     sequence is bounded
2. stable <=> Re[pᵢ] < 0 <=> convergent
3. convergent: sequence approaches zero (steady state)

大家習慣省略主角convergent。喪心病狂。
1. stable => stationary at steady states
2. stable <=> Re[pᵢ] < 0

(people pretend not to see 'convergent')
(textbooks replace all 'convergent' with 'stable')

statistics

統計學基礎問題。

https://tcs.nju.edu.cn/wiki/index.php?title=概率论与数理统计_(Spring_2025)

conditional probability 只關注某一塊子集合  代入身分建立視點
Bayes's theorem 切換視點
independence 即便到了子集合裡面比例也一樣，換句話說，在宇集合內很均勻
             p_xy(x,y) = p_x|y(x,y) p_y(y) = p_x(x) p_y(y)
             p_x|y(x,y) = p_x(x)
correlation 正比反比關係
causation   因果關係
association 上述所有東西的泛稱

normal distribution

central limit theorem 加在一起，平均數接近常態分布
Lévy–Cramér theorem 多個獨立變數，當總和是常態分布，則各自是常態分布
Darmois–Skitovitch theorem 線性組合互相獨立，必是常態分布

martingale

一個隨機過程，最新的K個隨機變數，期望值是定值。

https://en.wikipedia.org/wiki/Doob_martingale
https://en.wikipedia.org/wiki/Azuma's_inequality

When analyzing sums, random walks, or other additive functions of independent
random variables, one can often apply the central limit theorem, law of large
numbers, Chernoff's inequality, Chebyshev's inequality or similar tools. When
analyzing similar objects where the differences are not independent, the main
tools are martingales and Azuma's inequality.

Azuma's inequality applied to the Doob martingale gives the method of bounded
differences (MOBD) which is common in the analysis of randomized algorithms.

concentration

一個隨機變數，實際取樣集中於平均值附近。

generated by ChatGPT。

https://en.wikipedia.org/wiki/Concentration_inequality
https://en.wikipedia.org/wiki/Concentration_of_measure

concentration inequality

Markov's inequality:
Gives an upper bound on the probability that a
non-negative random variable exceeds a certain value.

Chebyshev's inequality:
Provides a bound on how much a random variable
deviates from its mean, in terms of its variance.

Chernoff bounds:
Provide exponentially decreasing bounds on the tail
probabilities of sums of independent random variables.

Hoeffding's inequality:
Gives a concentration bound for the sum of
bounded independent random variables.        

concentration of measure

Chernoff Bound:
Gives exponential decay bounds for the tail probabilities
of sums of independent random variables,
often used in concentration of measure results.

McDiarmid's Inequality:
Provides concentration bounds for functions of independent random variables, stating that a function of independent
variables is unlikely to deviate too far from its expected
value if each variable has only a small effect on the function.

Gaussian concentration:
For Gaussian distributions,
the concentration of measure shows that the probability
that a Gaussian random variable deviates by more than
a fixed amount from its mean is exponentially small.