Statistics
H Information Theory & Stochastic Processes
Measuring uncertainty in bits, and modelling systems that evolve randomly over time — entropy, channels, random walks, and Markov chains.
4 concepts— start at the top and work your way down
- 1→
Random Walks
Processes built from repeated random steps, modeled with transition matrices, absorbing boundaries, and diffusion-like spread.
- 2→
Markov Chains
State-based stochastic processes governed by transition matrices, stationary distributions, and long-run convergence.
- 3→
Markov Chain Monte Carlo
Sampling from a posterior distribution you can't write down in closed form, by building a random walk whose long-run behavior matches that distribution.
- 4→
Law of Large Numbers
As the number of trials grows, the sample mean converges to the true population mean — the mathematical foundation of why probability works.