Find your topic. Follow your curiosity.
Choose a model family or a technical topic to see the relevant articles together. For a guided introduction, follow a reading path below.
Learning & fundamentals
9 articlesUnderstand AI step by step: visual explanations, attention, KV cache and quantization.
- Attention Explained Visually: Queries, Keys and Values
- How LLMs Actually Work — An Animated Walkthrough
- KV Cache Explained Visually: Prefill, Decode and Memory
- KV Cache Explained: Memory Formula, GQA, MQA & MLA
- KV-Cache Glossary: GQA, MLA, TTFT and Eviction
- LLM Quantization: FP8, INT4, GGUF & VRAM Requirements
- LLM VRAM Requirements: Weights, Quantization & KV Cache
- NVFP4 vs INT4: A Visual Guide to 4-Bit Quantization
- Visual Tech Learning: From AI Basics to Systems
AI models
36 articlesModel families, architectures and practical applications, from Nemotron and Qwen to speech, images and embeddings.
Architecture, use cases and measurements are grouped by model family.
NVIDIA Nemotron
Qwen
- DSpine causal feature bridge and draft limits
- GPU for Qwen3 Inference: VRAM, Bandwidth & Model Fit
- MI350P vLLM Optimization: Kernels and Measurements
- Qwen-Image 2.1: Attention, KV Cache and Turbo Math
- Qwen3-Next-80B-A3B: Architecture and Serving Limits
- Qwen3.8 Flash Next: GDN, Attention and N-gram Memory
Beam
Cloudflare Clef
DeepSeek
EmbeddingGemma
Fibo
GLM
IBM Granite Speech
JetBrains Mellum
Kimi
Kolibri
Ling
Liquid AI
Muse
NVIDIA Parakeet
OlmoEarth
Perplexity pplx-embed
Ripple
StepFun
TypeSafe Jev
Inference & optimization
60 articlesServing, memory, quantization, GPU kernels and costs for operating AI systems.
KV cache & memory
- Agent Inference: KV-Cache Traffic and Memory Tiers
- Beam 1.0: NoPE, Softmax Scaling and KV Memory
- BOOST: Host Memory and HBM for LLM Decode
- CXL-SSD Prefix Caching: Why KV Chunk Awareness Matters
- DeepSeek V4.1 Flash: KV-Cache Compression
- DSpine causal feature bridge and draft limits
- Embedding value residuals and hybrid cache
- FlashLoop: Looped Transformers and KV-Cache Savings
- HBF for KV Cache: Capacity vs Latency
- HBM Capacity and KV-Cache Economics
- K3.1: DSA indexer and cache
- KITE: two-tower MoE and KV cache (research)
- KV Cache Explained Visually: Prefill, Decode and Memory
- KV Cache Explained: Memory Formula, GQA, MQA & MLA
- KV Cache Memory Growth Explorer
- KV-Cache Compression: Tensor Rank and Error
- KV-Cache Glossary: GQA, MLA, TTFT and Eviction
- KVSET: Size an LLM Prefix Cache with LRU Traces
- LFM2.5-24B-A2B: Routing and Cache Architecture
- Ling-3.1-flash: KDA, MLA and Actual Cache Layouts
- LLM Prefix Caching: LRU Eviction and Its Limits
- Mellum2.1: MoE, Sliding Attention and Selective YaRN
- Nemotron 3.5 Super VL: Visual Tokens and Video Pruning
- Qwen-Image 2.1: Attention, KV Cache and Turbo Math
- Random Attention vs Score-Based KV Eviction
- Risk-Controlled KV Eviction and Full-Cache Fallback
- Step-5-Preview: Sparse GQA, Indexer and KV Memory
- UNISON: Scheduling KV Cache for Paused Agents
- Visual Tech Learning: From AI Basics to Systems
Serving & deployment
- DeepSeek V4 Pro: Self-Hosting VRAM and Cost Math
- GLM-5.3 MTP on B300: Acceptance Math and NCCL NVLS
- llama.cpp vs Ollama vs vLLM: Local LLM Serving Compared
- LLM Inference Performance: Roofline Math & GPU Bandwidth
- LLM Serving Engines 2026: Upgrades and Version Pinning
- Nemotron E4B DFlash: Block Speculation and Serving Math
- NVIDIA and Hugging Face: Serving Dependencies
- vLLM vs SGLang: Serving, Caching & When to Use Each
Quantization & fine-tuning
- 4-Bit Quantization: Why Retrieval Rankings Change
- BF16 vs FP16: Why Greedy LLM Outputs Can Change
- Kolibri 1 Architecture: Hybrid Attention, MoE and FP8
- LLM Quantization: FP8, INT4, GGUF & VRAM Requirements
- LLM Quantization: Separate Prefill and Decode Formats
- LoRA and QLoRA Fine-Tuning: The Actual Memory Math
- Nemotron 3.5 Bolt: LatentMoE, IQ2 and Memory Budgets
- Nemotron 3.5 Embed: compatible multimodal retrieval
- NVFP4 vs INT4: A Visual Guide to 4-Bit Quantization
- OlmoEarth v1.3: pooler, Matryoshka student, Searchlight
- PTQ: Estimate Quantization Error Before Deployment
GPU kernels & scaling
Costs & capacity
Embeddings & RAG
6 articlesEmbedding models, vector search, retrieval quality and memory budgets.
Hardware & systems
17 articlesCompare GPUs, memory and networking, and set up Linux, drivers and containers.
GPUs & systems
- AMD Instinct vs NVIDIA: Memory and Serving Limits
- AMD MI455X: Examining the 34× Throughput Claim
- GPU Buying Guide for LLM Inference (2026)
- GPU for Qwen3 Inference: VRAM, Bandwidth & Model Fit
- MLPerf Inference: Hardware vs Software Gains
- NVIDIA B200 vs GB200: Differences & MLPerf Benchmarks
- NVIDIA Groq 3 LPX: SRAM, Bandwidth and LLM Decode
Memory & data movement
Networking & interconnects
Linux, drivers & containers
Benchmarks & evaluation
7 articlesAssess measurements, model evaluations and datasets, and interpret results.
- AMD MI455X: Examining the 34× Throughput Claim
- KREX: Reliable Kernel Timing on Shared GPUs
- LLM Leaderboards: Hidden Selection and Score Margins
- MLPerf Inference: Hardware vs Software Gains
- Nemotron 3.5 Transcribe: Architecture and Dataset Auditing
- NVIDIA B200 vs GB200: Differences & MLPerf Benchmarks
- Uncheatable Eval: LLM Scoring by Text Compression
AI regulation & transparency
5 articlesEU AI Act, data protection, disclosure and the technical limits of watermarking.
Calculators & tools
3 articlesCalculate VRAM, KV-cache growth and inference costs interactively.
Learn step by step
Start with a question. Each reading path puts the foundations, practical tools and deeper analyses in a useful order.
New to AI? See how it works.
Start with attention, KV cache and quantization. Continue with an individual visual explanation in every technical article, from model architectures to GPUs and Linux.
- Visual Tech Learning: From AI Basics to Systems
Choose your starting point; no machine-learning background needed.
- Attention Explained Visually: Queries, Keys and Values
Select a token and calculate how it mixes information from the context.
- KV Cache Explained Visually: Prefill, Decode and Memory
Watch which computations are reused as a model generates text.
- NVFP4 vs INT4: A Visual Guide to 4-Bit Quantization
Move a number to compare INT4 and NVFP4 rounding and scales.
- How LLMs Actually Work — An Animated Walkthrough
Connect the pieces: tokens, embeddings, Transformer blocks and sampling.
How much VRAM does my LLM need?
Start with the memory budget, then calculate weights, context and runtime overhead. Fine-tuning has a separate budget.
- LLM VRAM Requirements: Weights, Quantization & KV Cache
Understand the inference memory budget and its assumptions.
- LLM VRAM Calculator: GPU Memory & Inference Estimates
Calculate model fit for a chosen GPU, precision and context.
- LLM Quantization: FP8, INT4, GGUF & VRAM Requirements
Separate weight compression from quality and runtime constraints.
- KV Cache Explained: Memory Formula, GQA, MQA & MLA
Work out the memory consumed by context and concurrent requests.
- LoRA and QLoRA Fine-Tuning: The Actual Memory Math
Plan adapters, optimizer states and activations for fine-tuning.
Which GPU and interconnect fit my workload?
Use model fit and bandwidth for inference planning. Read training benchmarks as system results and distinguish them from purchasing evidence.
- GPU Buying Guide for LLM Inference (2026)
Compare specifications, quotes and workload requirements.
- GPU for Qwen3 Inference: VRAM, Bandwidth & Model Fit
Apply roofline math to concrete Qwen3 and GPU pairings.
- NVIDIA B200 vs GB200: Differences & MLPerf Benchmarks
Compare published B200 and GB200 training submissions fairly.
- NVIDIA NVLink: Bandwidth, NVSwitch & GPU Topology
Understand what an NVLink domain connects.
- Scale-Up vs Scale-Out: NVLink and All-Reduce
Calculate communication within and between GPU servers.
- AMD Instinct vs NVIDIA: Memory and Serving Limits
Compare AMD and NVIDIA memory specifications and their limits.
How do I choose and operate an LLM serving engine?
Keep local deployment, engine selection, release maintenance and cost calculations separate. Each guide answers a different operational question.
- llama.cpp vs Ollama vs vLLM: Local LLM Serving Compared
Choose between local runtimes and GPU serving.
- vLLM vs SGLang: Serving, Caching & When to Use Each
Compare vLLM and SGLang for your request patterns.
- LLM Serving Engines 2026: Upgrades and Version Pinning
Budget for upgrades, version pinning and rollback.
- LLM Inference Performance: Roofline Math & GPU Bandwidth
Understand the formulas behind decode and prefill limits.
- MI350P vLLM Optimization: Kernels and Measurements
Audit an inference campaign and follow GDN, QSA and MoE kernel changes.
- LLM Inference Economics: Costs & Cloud Break-Even
Compare self-hosting costs with an API under explicit assumptions.
How do I size and manage a KV cache?
Move from per-token memory to working sets and prefix reuse. Sizing, eviction and offload solve different problems.
- KV Cache Explained Visually: Prefill, Decode and Memory
See prefill, decode and cache reuse one step at a time.
- KV Cache Explained: Memory Formula, GQA, MQA & MLA
Learn the per-token formula and attention-layout differences.
- KV Cache Memory Growth Explorer
Explore how growing context consumes free GPU memory.
- KV-Cache Glossary: GQA, MLA, TTFT and Eviction
Look up serving terms and their precise definitions.
- KVSET: Size an LLM Prefix Cache with LRU Traces
Derive a cache-capacity curve from an LRU trace.
- LLM Prefix Caching: LRU Eviction and Its Limits
Evaluate LRU eviction for different workloads and capacities.
- Agent Inference: KV-Cache Traffic and Memory Tiers
Calculate agent working sets and memory-tier traffic.
What changes between model architectures?
Learn the basic token-to-output flow, then compare released language and multimodal checkpoints. Architecture guides describe mechanisms and serving limits.
- Attention Explained Visually: Queries, Keys and Values
Calculate one attention head before exploring whole architectures.
- How LLMs Actually Work — An Animated Walkthrough
Follow embeddings, attention and sampling in an animated walkthrough.
- Qwen-Image 2.1: Attention, KV Cache and Turbo Math
Inspect block-causal diffusion attention, prefix caching and the Turbo schedule.
- Fibo 2.0 Architecture: Gist Tokens and Editing Costs
Trace adaptive gist tokens, dual text injection and clean-latent editing costs.
- OlmoEarth v1.3: pooler, Matryoshka student, Searchlight
Trace spatial attention pooling, nested distillation and seam-aware embedding serving.
- Qwen3-Next-80B-A3B: Architecture and Serving Limits
Inspect Qwen3-Next hybrid attention and sparse experts.
- LFM2.5-24B-A2B: Routing and Cache Architecture
Separate biased expert selection from mixture weights and derive hybrid LFM2.5 cache memory.
- Mellum2.1: MoE, Sliding Attention and Selective YaRN
Derive Mellum2.1 local/global attention, selective YaRN and hybrid cache memory.
- Beam 1.0: NoPE, Softmax Scaling and KV Memory
Trace local RoPE and global NoPE, derive SSMax entropy effects and distinguish KV payload from allocation.
- Ripple-65B: Value Residuals and Hybrid KV Cache
Trace embedding-seeded value residuals, PeriNorm and the difference between windowed reads and compact cache storage.
- Kolibri 1 Architecture: Hybrid Attention, MoE and FP8
Read Kolibri 1 architecture and its serving caveats.
- Kimi K3 Architecture: KDA, MLA and LatentMoE
Read Kimi K3 recurrence, MLA and LatentMoE routing.
- Kimi K3.1 Pro: DSA Indexer and KV-Cache Limits
Inspect K3.1 NoPE indexing, sparse MLA reads and the distinction between read width and retained history.
- Ling-3.1-flash: KDA, MLA and Actual Cache Layouts
Derive Ling grouped routing and the difference between expanded and absorbed MLA caches.
- Step-5-Preview: Sparse GQA, Indexer and KV Memory
Trace Step-5 block indexing and separate sparse K/V reads from retained cache capacity.
- Nemotron 3.5 Super VL: Visual Tokens and Video Pruning
Trace Super VL visual patches, paired frames and change-based pruning; separate token savings from vision work.
- Nemotron 3.5 Bolt: LatentMoE, IQ2 and Memory Budgets
Inspect Bolt heterogeneous LatentMoE, packed IQ2 weights and the separate runtime memory budget.
- Nemotron E4B DFlash: Block Speculation and Serving Math
Derive E4B DFlash block proposals, exact verification and the latency/concurrency break-even point.
- Qwen3.8 DSpine: Causal Drafting, Refinement and Cache
Trace adjacent causal draft writes, serial candidate refinement and the dense-cache serving boundary.
- Muse Spark Open: Residual Gates and Normalized Experts
Distinguish Muse attention, routing and residual gates, and derive the limits of norm-preserving mixing.
- Inside GLM-5.3-Flash: KDA, Sparse MLA, mHC, and 8-of-288 MoE
Inspect GLM-5.3-Flash attention, residuals and cache shapes.
- Cloudflare Clef: Qwen Backbones and Schema Scoring
Understand Clef decision scoring and its prefill-only design.
- Liquid d1-3B: Shared-State Branch Inference
Derive shared-state branch inference and exact split-attention merging in d1-3B.
- Clef Omni: Audio, Video, MoE and Schema Scoring
Follow Clef Omni audio/video fusion, sparse experts and schema scoring.
- Granite Speech 5.0 TurboCTC Architecture
Trace pyramidal Conformer shapes, block attention and encoder-only CTC decoding.
- Nemotron 3.5 Transcribe: Architecture and Dataset Auditing
Connect promptable speech architecture to applications and evidence-based audio–transcript audits.
How do I compare embeddings for retrieval?
Start with storage and latency budgets, compare single-vector and late-interaction approaches, then examine quantization limits.
- RAG Memory Math: Embeddings, HNSW and Rerankers
Budget embedding weights, vector-index storage and retrieval latency.
- EmbeddingGemma 2 Architecture: Soft Tokens, PLE and MRL
Inspect an embedding encoder, pooling and production limits.
- Nemotron 3.5 Embed: compatible multimodal retrieval
Trace frozen text-space compatibility, visual alignment, distillation and serving dimensions.
- pplx-embed-v2: 9B documents, 0.6B queries
Compare late interaction and asymmetric document/query encoding.
- 4-Bit Quantization: Why Retrieval Rankings Change
Distinguish retrieval-ranking changes from losses in relevance.
How do I get NVIDIA GPUs working on Ubuntu?
Separate installation and container verification from CUDA error diagnosis. A reported workaround is not a universal fix.
- NVIDIA GPU Containers on Ubuntu 24.04
Install the host driver and verify GPU access inside Docker.
- NVIDIA Error 802 on Ubuntu: Diagnose Before Trying nokaslr
Diagnose CUDA error 802 before considering nokaslr.