Series

EXPLAIN, indexes, WHERE vs HAVING — diagnose and optimize slow queries yourself

Derives DDPM's forward process from the beta schedule, the closed form that jumps straight to any timestep, the reverse process and its posterior, the ELBO, and the reparameterization to noise prediction. Every equation worked through rather than quoted.

NVIDIA Nemotron 3 Nano, Qwen 3.5, and Mamba-3 independently converge on 75% linear layers + 25% attention + MoE. 88% KV-cache reduction, O(n) complexity for long-context processing.

Why can't traditional RAG answer "What are the main themes in these documents?" Microsoft Research's GraphRAG reveals the secret of community-based search.

I trained BDH and counted more than 4.8 billion activations across three controls. Training moved its single-latent sparsity from 49.98% to 81.65%, but a similar-budget ReLU baseline reached 91.04%. Sparsity is real; by itself, it is not evidence of interpretability.

From DDPM to LLaDA 2.1 -- everything about diffusion-based LLMs. Masked Diffusion, Token Editing, and MoE scaling dissected across 4 parts.

karpathy-skills, claude-mem, Cognee — comparing 3 approaches to solving the AI memory problem.

Understand the foundational ReAct pattern. The difference between chatbots and agents, the Thought-Action-Observation loop, and why ReAct falls short in production.

Install AgentScope, learn 5 core concepts (Agent, Model, Memory, Toolkit, Formatter), and build a tool-using ReAct agent.

A condensed walk through the 4-hour video: building GPT-2 (124M), taking a step from 1,000 ms to 90 ms on one A100, the GPT-3 training settings, and a 10B-token run that beat OpenAI's checkpoint on HellaSwag. Plus one check we ran: nanoGPT's exact GELU moves logits by up to 5 against GPT-2's.

From LoRA theory to Qwen 2.5 7B model setup. 99.8% parameter reduction and 86% memory savings vs full fine-tuning, explained with code.

A code-level deep dive into Karpathy's autoresearch. Dissecting train.py, BPE tokenizer, MuonAdamW optimizer, and the agent protocol design.

Diagnose naive RAG limitations, classify query intent, and route to the optimal retrieval source with LangGraph. Implement adaptive retrieval that skips unnecessary searches.

DeerFlow 2.0 architecture, setup, and first task execution. A SuperAgent runtime with 9 agent nodes, 5 tool sources, and Docker sandboxes.

Every piece of text Claude generates now carries an invisible watermark — with nothing added to the text. A step-by-step walkthrough of SynthID-Text: the secret key, tournament sampling, detection, and the honest limits.

Build Self-Attention from scratch. Compare MHA → GQA → MQA evolution in code. KV Cache mechanics and Prefill vs Decode analysis.

Stanford researchers analyzed 500+ papers to systematically map LLM reasoning failures. From cognitive biases to the reversal curse, discover where and why AI reasoning breaks down.

One CLAUDE.md file can dramatically change your AI coding assistant's performance. Learn how to keep Claude on track in large-scale projects.

What happens inside an LLM between input and output? Logit Lens and Tuned Lens let us observe how Transformers build predictions layer by layer.

We reproduced Table 5 of arXiv:2609.04575 on one A100 and measured the throughput the paper leaves out: nothing in HF transformers, nothing at batch 1 in stock vLLM, 1.35× at batch 8.

We measure, on a local model and on GPT-4o-mini, exactly how much answers change with sampling temperature. Temperature doesn't create diversity -- it amplifies what's already there.

Every list of KV cache techniques presents a dozen of them as peers. On one A100 the spread is enormous: prefix reuse cut a shared-prefix workload from 59.4s to 6.2s, while tuning the paged block size moved capacity by under 1%. The reason you cannot rank them on one number is that they pay out in different units.

Common patterns across AI, RAG, and ML projects — why does "it worked fine" fall apart in production?