Posts

Modern machine learning is incredible at finding statistical patterns in large datasets. Neural networks regularly reach or beat human performance in vision, speech, and language tasks. However, relying only on statistical correlations comes with a major catch: standard models assume that the test data follows the exact same distribution as the training data. In the real world, data distributions shift. When this occurs, purely statistical models often fail. To build models that are robust to distribution shifts, we have to look beyond pure correlation and turn to causal inference. ...

Disclaimer: This is a vibe-coding project. I used Claude Code for the vast majority of the implementation (especially the wgpu rendering pipeline and noise-based terrain generation). My primary focus was exploring the architecture around chunk management, spatial indexing, and meshing. 😊 Live Demo · GitHub Repository The Software Stack With WebGPU maturing and WebAssembly integration in Rust becoming remarkably smooth, I wanted to see how far browser-based 3D engines could be pushed: ...

Abstract High-quality multilingual training data is essential for effectively pretraining large language models (LLMs). Yet, the availability of suitable open-source multilingual datasets remains limited. Existing state-of-the-art datasets mostly rely on heuristic filtering methods, restricting both their cross-lingual transferability and scalability. Here, we introduce JQL, a systematic approach that efficiently curates diverse and high-quality multilingual data at scale while significantly reducing computational demands. JQL distills LLMs’ annotation capabilities into lightweight annotators based on pretrained multilingual embeddings. These models exhibit robust multilingual and cross-lingual performance, even for languages and scripts unseen during training. Evaluated empirically across 35 languages, the resulting annotation pipeline substantially outperforms current heuristic filtering methods like Fineweb2. JQL notably enhances downstream model training quality and increases data retention rates. Our research provides practical insights and valuable resources for multilingual data curation, raising the standards of multilingual dataset development. ...

In standard machine learning, the starting point typically consists of empirical loss functions such as Mean Squared Error or Cross-Entropy. Later, probabilistic ML introduces concepts like priors, likelihoods, and posteriors, which can make the field appear fragmented. In reality, these approaches are mathematically equivalent. The relationship becomes clear when structured into three progressive levels: Maximum Likelihood (MLE), Maximum A Posteriori (MAP), and full Bayesian inference. 1. The Core Mechanism: Updating Beliefs Bayes’ theorem formalizes how an initial belief regarding a parameter or hypothesis $\theta$ is updated once dataset $\mathcal{D}$ has been observed: ...