Probability

Modern machine learning is incredible at finding statistical patterns in large datasets. Neural networks regularly reach or beat human performance in vision, speech, and language tasks. However, relying only on statistical correlations comes with a major catch: standard models assume that the test data follows the exact same distribution as the training data. In the real world, data distributions shift. When this occurs, purely statistical models often fail. To build models that are robust to distribution shifts, we have to look beyond pure correlation and turn to causal inference. ...

In standard machine learning, the starting point typically consists of empirical loss functions such as Mean Squared Error or Cross-Entropy. Later, probabilistic ML introduces concepts like priors, likelihoods, and posteriors, which can make the field appear fragmented. In reality, these approaches are mathematically equivalent. The relationship becomes clear when structured into three progressive levels: Maximum Likelihood (MLE), Maximum A Posteriori (MAP), and full Bayesian inference. 1. The Core Mechanism: Updating Beliefs Bayes’ theorem formalizes how an initial belief regarding a parameter or hypothesis $\theta$ is updated once dataset $\mathcal{D}$ has been observed: ...