Topics & Tags
All TechHub articles, grouped by tag. Click through the topics.
ai91 articles
- About Florian Zimmermeister
- Qwen3-Next-80B-A3B: Architecture and Serving Limits
- TypeSafe Jev: What the Public Evidence Supports
- Nemotron 3.5 Transcribe: From Recorded Speech to Dataset Audits
- Agent-Fleet Token Economics Is Cache Economics: Decomposing the 60x Dollar Spread
- Your Agent Fleet Is a Storage Array: The KV-Cache Tiering Math Behind Agentic Inference
- EU AI Act Article 50: Marking and Its Technical Limits
- EU AI Act after the Digital Omnibus: What Moved
- EU AI Act and GDPR for Self-Hosted LLMs
- Granite Speech 5.0 TurboCTC: Pyramidal Conformer Serving
- Parakeet TDT vs Encoder–Decoder ASR: Production Trade-offs
- Random Attention vs Score-Based KV Eviction on Reasoning Traces
- Attention Explained Visually
- Beam 1.0: Local RoPE, Global NoPE and Scalable Softmax
- BOOST: Concurrent Host and HBM Access for LLM Decode
- The Cache-Read Price War of September 2026 That Wasn't Three-Sided
- Inside Cloudflare Clef: Prefill-Only Qwen and a Joint Schema Head
- Clef Omni: Audio, Video, Sparse Experts and Schema Decisions
- CXL-SSDs For LLM Prefix Caching: Byte-Addressability Buys Nothing Without Chunk Awareness
- CXMT at 10% of DRAM Revenue: What the Figures Show
- Self-hosting DeepSeek V4 Pro: the actual math
- DeepSeek-V4.1-Flash KV-Cache Compression: What the Reported 890 Bytes per Token Includes
- Disaggregated Quantization: Splitting the Prefill and Decode Price System Across Two Formats
- Inside EmbeddingGemma 2: Soft Tokens, PLE, Hybrid Attention, and MRL
- Fibo 2.0: Gist Compression, Dual Text Injection and Clean-Latent Editing
- FlashLoop, or: Looping a Transformer Quadruples the Bill and Then Throws Most of It Away
- Inside GLM-5.3-Flash: KDA, Sparse MLA, mHC, and 8-of-288 MoE
- GPU Buying Guide for LLM Inference (2026)
- Selecting the Right GPU for Qwen3 Inference
- Gumbel Watermarking Finally Shipped — and the Compliance Asymmetry Nobody Priced
- HBF as the Third KV Tier: 24x Sessions or 5x Worse Latency - the Medium Is Fine, the Placement Policy Decides
- HBM Capacity and KV-Cache Economics
- How LLMs Actually Work — An Animated Walkthrough
- KernelOPT's Agentic Kernel Search: The Verification Cascade Works, The Speedup Fades With Difficulty
- Germany's KI-MIG: Who Enforces the AI Act?
- Inside Kimi K3: Delta Attention, Depth Attention, and LatentMoE
- Kimi K3.1 Pro: Sparse Reads, Not a Smaller History
- Inside Kolibri 1: Hybrid RNoPE Attention, Sparse MoE, and FP8 Serving
- KREX: Shared-GPU Kernel Benchmarking Without Corrupting the Agent's Search
- The KV Cache: Bit-Exact Memory Math, GQA vs MQA vs MLA, and PagedAttention
- KV-Cache Glossary: every term in the serving stack
- Who Pays for the KV Cache? Metering Rules Are a Pricing Decision in Disguise
- The KV Cache Explained Visually
- Risk-Controlled KV Eviction: The Reliability Contract and Full-KV Fallback
- KITE: Scale the Model, Freeze the KV Bill — the Prefill Invariant arXiv:2609.27294 Actually Proves (and the Three Things It Does Not)
- KV-Cache Tensor Decomposition: Full-Rank Head and Layer Modes in Two Models
- KVSET: LRU Stack Distance for LLM Prefix-Cache Sizing
- Leaderboard Margins vs Hidden Model Selection: Which Gains Survive k Secret Variants?
- LFM2.5-24B-A2B: Biased Routing, Short Convolutions and Ten KV Caches
- Ling-3.1-flash: Grouped Routing and Two Different MLA Caches
- Liquid d1-3B: Branching Decisions Without Decoding
- llama.cpp / Ollama vs vLLM: Which One, When, and Why
- LLM Inference Math: From Theory to Hardware
- LLM VRAM Requirements: A Mathematical Deep Dive
- LoRA and QLoRA Fine-Tuning: The Actual Memory Math
- Mellum2.1: Sparse Experts, Local Attention and Selective YaRN
- Serving Qwen3.8 on MI350P: Kernels, Failures, and Measurements
- Muse Spark Open: Norm-Preserving Gates Are Not a Norm Guarantee
- Nemotron 3.5 Bolt E4B: Block-Parallel Speculation with DFlash
- Nemotron 3.5 Bolt: Heterogeneous LatentMoE and the 24 GB Budget
- Nemotron 3.5 Embed: adding images without replacing the text index
- Nemotron 3.5 Super VL: Where Visual Token Compression Happens
- NVIDIA Groq 3 LPX: The SRAM Decode Engine and the Arithmetic NVIDIA Won't Do
- NVIDIA Agrees to Buy Hugging Face: The Dependency Math Behind the $12.93B Headline
- OlmoEarth v1.3: Attention Pooling, Matryoshka Distillation, and Searchlight
- pplx-embed-v2: 9B documents, 0.6B queries
- Greedy Decoding Is Not Precision-Invariant: BF16-vs-FP16 Flips, Margin Gating, and the Kernel-Order Axis
- LLM Prefix Caching: Where LRU Holds Up and Where It Falls Short
- PTQ Configuration Pricing: Predicting the Quantization Price Before You Build the Model
- LLM Quantization: Bit Layouts, Block Scales, and the VRAM Math
- Quantization Can Change Retrieval Results While Classification Accuracy Holds
- NVFP4 vs INT4 Explained Visually
- Qwen-Image 2.1: Block-Causal Attention, Prefix KV Cache, and Turbo
- DSpine for Qwen3.8-27B: Causal Features Through Depth
- Inside Qwen3.8-Flash-Next: Gated DeltaNet, Sparse Attention, and N-gram Memory
- RAG Retrieval Math: Embedding Memory, Vector Index Size, and the Latency Budget Nobody Calculates
- Ripple-65B: Embedding Value Residuals and Hybrid Attention
- Expert Parallelism: The Router's Dilemma
- Pipeline Parallelism: Pumping Data Upstream
- Tensor Parallelism: Slicing the Silicon
- The 2026 Serving-Engine Churn Tax: vLLM 0.29, SGLang 0.5.19, and TensorRT-LLM Losing TensorRT
- Snapdragon X2 on Linux: The 80-TOPS NPU Is Not the Story — 228 GB/s Is
- Step-5-Preview: Sparse Reads Are Not a Compressed KV Cache
- Uncheatable Eval: Scoring LLMs by How Few Bits They Need
- UNISON and the Paused Fleet: Why Agent KV Traffic Breaks Every Cache Policy You Know
- Understand AI Visually: Start Here
- vLLM vs SGLang: Architecture, Overhead, and When to Pick Which
- My Journey: Obsession, Failure, and AI as an Anchor
- KV Cache Memory Growth Explorer
- LLM Inference Calculator & Break-Even Analyzer
- LLM Inference Economics: Costs & Cloud Break-Even
inference54 articles
- Nemotron 3.5 Transcribe: From Recorded Speech to Dataset Audits
- Agent-Fleet Token Economics Is Cache Economics: Decomposing the 60x Dollar Spread
- Your Agent Fleet Is a Storage Array: The KV-Cache Tiering Math Behind Agentic Inference
- Random Attention vs Score-Based KV Eviction on Reasoning Traces
- Beam 1.0: Local RoPE, Global NoPE and Scalable Softmax
- BOOST: Concurrent Host and HBM Access for LLM Decode
- The Cache-Read Price War of September 2026 That Wasn't Three-Sided
- Inside Cloudflare Clef: Prefill-Only Qwen and a Joint Schema Head
- Clef Omni: Audio, Video, Sparse Experts and Schema Decisions
- CXL-SSDs For LLM Prefix Caching: Byte-Addressability Buys Nothing Without Chunk Awareness
- DeepSeek-V4.1-Flash KV-Cache Compression: What the Reported 890 Bytes per Token Includes
- Disaggregated Quantization: Splitting the Prefill and Decode Price System Across Two Formats
- Inside EmbeddingGemma 2: Soft Tokens, PLE, Hybrid Attention, and MRL
- Fibo 2.0: Gist Compression, Dual Text Injection and Clean-Latent Editing
- FlashLoop, or: Looping a Transformer Quadruples the Bill and Then Throws Most of It Away
- Inside GLM-5.3-Flash: KDA, Sparse MLA, mHC, and 8-of-288 MoE
- HBF as the Third KV Tier: 24x Sessions or 5x Worse Latency - the Medium Is Fine, the Placement Policy Decides
- HBM Capacity and KV-Cache Economics
- Inside Kimi K3: Delta Attention, Depth Attention, and LatentMoE
- Kimi K3.1 Pro: Sparse Reads, Not a Smaller History
- Inside Kolibri 1: Hybrid RNoPE Attention, Sparse MoE, and FP8 Serving
- KV-Cache Glossary: every term in the serving stack
- Who Pays for the KV Cache? Metering Rules Are a Pricing Decision in Disguise
- The KV Cache Explained Visually
- Risk-Controlled KV Eviction: The Reliability Contract and Full-KV Fallback
- KITE: Scale the Model, Freeze the KV Bill — the Prefill Invariant arXiv:2609.27294 Actually Proves (and the Three Things It Does Not)
- KV-Cache Tensor Decomposition: Full-Rank Head and Layer Modes in Two Models
- KVSET: LRU Stack Distance for LLM Prefix-Cache Sizing
- LFM2.5-24B-A2B: Biased Routing, Short Convolutions and Ten KV Caches
- Ling-3.1-flash: Grouped Routing and Two Different MLA Caches
- Liquid d1-3B: Branching Decisions Without Decoding
- Mellum2.1: Sparse Experts, Local Attention and Selective YaRN
- Serving Qwen3.8 on MI350P: Kernels, Failures, and Measurements
- Muse Spark Open: Norm-Preserving Gates Are Not a Norm Guarantee
- Nemotron 3.5 Bolt E4B: Block-Parallel Speculation with DFlash
- Nemotron 3.5 Bolt: Heterogeneous LatentMoE and the 24 GB Budget
- Nemotron 3.5 Super VL: Where Visual Token Compression Happens
- NVIDIA Groq 3 LPX: The SRAM Decode Engine and the Arithmetic NVIDIA Won't Do
- Greedy Decoding Is Not Precision-Invariant: BF16-vs-FP16 Flips, Margin Gating, and the Kernel-Order Axis
- LLM Prefix Caching: Where LRU Holds Up and Where It Falls Short
- PTQ Configuration Pricing: Predicting the Quantization Price Before You Build the Model
- Quantization Can Change Retrieval Results While Classification Accuracy Holds
- Qwen-Image 2.1: Block-Causal Attention, Prefix KV Cache, and Turbo
- DSpine for Qwen3.8-27B: Causal Features Through Depth
- Inside Qwen3.8-Flash-Next: Gated DeltaNet, Sparse Attention, and N-gram Memory
- Ripple-65B: Embedding Value Residuals and Hybrid Attention
- Step-5-Preview: Sparse Reads Are Not a Compressed KV Cache
- UNISON and the Paused Fleet: Why Agent KV Traffic Breaks Every Cache Policy You Know
- AMD's MI455X 34× Throughput Claim: What the Disclosure Establishes
- AMD MI300X/MI350/MI400: More Memory, But Does the Math Win? — A Critical Silicon View of AMD Instinct vs NVIDIA
- MLPerf Inference 2026: What a Result Says About Hardware and Software
- KV Cache Memory Growth Explorer
- LLM Inference Calculator & Break-Even Analyzer
- LLM Inference Economics: Costs & Cloud Break-Even
machine-learning43 articles
- Qwen3-Next-80B-A3B: Architecture and Serving Limits
- TypeSafe Jev: What the Public Evidence Supports
- Agent-Fleet Token Economics Is Cache Economics: Decomposing the 60x Dollar Spread
- Your Agent Fleet Is a Storage Array: The KV-Cache Tiering Math Behind Agentic Inference
- Granite Speech 5.0 TurboCTC: Pyramidal Conformer Serving
- Parakeet TDT vs Encoder–Decoder ASR: Production Trade-offs
- Random Attention vs Score-Based KV Eviction on Reasoning Traces
- BOOST: Concurrent Host and HBM Access for LLM Decode
- The Cache-Read Price War of September 2026 That Wasn't Three-Sided
- CXL-SSDs For LLM Prefix Caching: Byte-Addressability Buys Nothing Without Chunk Awareness
- Self-hosting DeepSeek V4 Pro: the actual math
- DeepSeek-V4.1-Flash KV-Cache Compression: What the Reported 890 Bytes per Token Includes
- Disaggregated Quantization: Splitting the Prefill and Decode Price System Across Two Formats
- FlashLoop, or: Looping a Transformer Quadruples the Bill and Then Throws Most of It Away
- Gumbel Watermarking Finally Shipped — and the Compliance Asymmetry Nobody Priced
- HBF as the Third KV Tier: 24x Sessions or 5x Worse Latency - the Medium Is Fine, the Placement Policy Decides
- HBM Capacity and KV-Cache Economics
- KernelOPT's Agentic Kernel Search: The Verification Cascade Works, The Speedup Fades With Difficulty
- KREX: Shared-GPU Kernel Benchmarking Without Corrupting the Agent's Search
- The KV Cache: Bit-Exact Memory Math, GQA vs MQA vs MLA, and PagedAttention
- KV-Cache Glossary: every term in the serving stack
- Who Pays for the KV Cache? Metering Rules Are a Pricing Decision in Disguise
- Risk-Controlled KV Eviction: The Reliability Contract and Full-KV Fallback
- KITE: Scale the Model, Freeze the KV Bill — the Prefill Invariant arXiv:2609.27294 Actually Proves (and the Three Things It Does Not)
- KV-Cache Tensor Decomposition: Full-Rank Head and Layer Modes in Two Models
- KVSET: LRU Stack Distance for LLM Prefix-Cache Sizing
- Leaderboard Margins vs Hidden Model Selection: Which Gains Survive k Secret Variants?
- llama.cpp / Ollama vs vLLM: Which One, When, and Why
- LLM Inference Math: From Theory to Hardware
- LLM VRAM Requirements: A Mathematical Deep Dive
- LoRA and QLoRA Fine-Tuning: The Actual Memory Math
- NVIDIA Groq 3 LPX: The SRAM Decode Engine and the Arithmetic NVIDIA Won't Do
- NVIDIA Agrees to Buy Hugging Face: The Dependency Math Behind the $12.93B Headline
- Greedy Decoding Is Not Precision-Invariant: BF16-vs-FP16 Flips, Margin Gating, and the Kernel-Order Axis
- LLM Prefix Caching: Where LRU Holds Up and Where It Falls Short
- PTQ Configuration Pricing: Predicting the Quantization Price Before You Build the Model
- LLM Quantization: Bit Layouts, Block Scales, and the VRAM Math
- Quantization Can Change Retrieval Results While Classification Accuracy Holds
- RAG Retrieval Math: Embedding Memory, Vector Index Size, and the Latency Budget Nobody Calculates
- The 2026 Serving-Engine Churn Tax: vLLM 0.29, SGLang 0.5.19, and TensorRT-LLM Losing TensorRT
- Uncheatable Eval: Scoring LLMs by How Few Bits They Need
- UNISON and the Paused Fleet: Why Agent KV Traffic Breaks Every Cache Policy You Know
- vLLM vs SGLang: Architecture, Overhead, and When to Pick Which
llm36 articles
- Qwen3-Next-80B-A3B: Architecture and Serving Limits
- Agent-Fleet Token Economics Is Cache Economics: Decomposing the 60x Dollar Spread
- Beam 1.0: Local RoPE, Global NoPE and Scalable Softmax
- The Cache-Read Price War of September 2026 That Wasn't Three-Sided
- Inside Cloudflare Clef: Prefill-Only Qwen and a Joint Schema Head
- CXL-SSDs For LLM Prefix Caching: Byte-Addressability Buys Nothing Without Chunk Awareness
- DeepSeek-V4.1-Flash KV-Cache Compression: What the Reported 890 Bytes per Token Includes
- Disaggregated Quantization: Splitting the Prefill and Decode Price System Across Two Formats
- FlashLoop, or: Looping a Transformer Quadruples the Bill and Then Throws Most of It Away
- Inside GLM-5.3-Flash: KDA, Sparse MLA, mHC, and 8-of-288 MoE
- Gumbel Watermarking Finally Shipped — and the Compliance Asymmetry Nobody Priced
- How LLMs Actually Work — An Animated Walkthrough
- KernelOPT's Agentic Kernel Search: The Verification Cascade Works, The Speedup Fades With Difficulty
- Inside Kimi K3: Delta Attention, Depth Attention, and LatentMoE
- Inside Kolibri 1: Hybrid RNoPE Attention, Sparse MoE, and FP8 Serving
- KREX: Shared-GPU Kernel Benchmarking Without Corrupting the Agent's Search
- Who Pays for the KV Cache? Metering Rules Are a Pricing Decision in Disguise
- Risk-Controlled KV Eviction: The Reliability Contract and Full-KV Fallback
- KITE: Scale the Model, Freeze the KV Bill — the Prefill Invariant arXiv:2609.27294 Actually Proves (and the Three Things It Does Not)
- KV-Cache Tensor Decomposition: Full-Rank Head and Layer Modes in Two Models
- KVSET: LRU Stack Distance for LLM Prefix-Cache Sizing
- Leaderboard Margins vs Hidden Model Selection: Which Gains Survive k Secret Variants?
- LFM2.5-24B-A2B: Biased Routing, Short Convolutions and Ten KV Caches
- Ling-3.1-flash: Grouped Routing and Two Different MLA Caches
- Liquid d1-3B: Branching Decisions Without Decoding
- Muse Spark Open: Norm-Preserving Gates Are Not a Norm Guarantee
- Nemotron 3.5 Bolt E4B: Block-Parallel Speculation with DFlash
- Nemotron 3.5 Bolt: Heterogeneous LatentMoE and the 24 GB Budget
- Greedy Decoding Is Not Precision-Invariant: BF16-vs-FP16 Flips, Margin Gating, and the Kernel-Order Axis
- LLM Prefix Caching: Where LRU Holds Up and Where It Falls Short
- PTQ Configuration Pricing: Predicting the Quantization Price Before You Build the Model
- Quantization Can Change Retrieval Results While Classification Accuracy Holds
- Inside Qwen3.8-Flash-Next: Gated DeltaNet, Sparse Attention, and N-gram Memory
- Step-5-Preview: Sparse Reads Are Not a Compressed KV Cache
- Uncheatable Eval: Scoring LLMs by How Few Bits They Need
- Scale-Up vs Scale-Out: Network Bandwidth and Collective Math
gpu34 articles
- Your Agent Fleet Is a Storage Array: The KV-Cache Tiering Math Behind Agentic Inference
- Random Attention vs Score-Based KV Eviction on Reasoning Traces
- BOOST: Concurrent Host and HBM Access for LLM Decode
- Self-hosting DeepSeek V4 Pro: the actual math
- GPU Buying Guide for LLM Inference (2026)
- Selecting the Right GPU for Qwen3 Inference
- HBF as the Third KV Tier: 24x Sessions or 5x Worse Latency - the Medium Is Fine, the Placement Policy Decides
- HBM Capacity and KV-Cache Economics
- KernelOPT's Agentic Kernel Search: The Verification Cascade Works, The Speedup Fades With Difficulty
- KREX: Shared-GPU Kernel Benchmarking Without Corrupting the Agent's Search
- The KV Cache: Bit-Exact Memory Math, GQA vs MQA vs MLA, and PagedAttention
- KV-Cache Glossary: every term in the serving stack
- llama.cpp / Ollama vs vLLM: Which One, When, and Why
- LLM Inference Math: From Theory to Hardware
- LLM VRAM Requirements: A Mathematical Deep Dive
- LoRA and QLoRA Fine-Tuning: The Actual Memory Math
- Serving Qwen3.8 on MI350P: Kernels, Failures, and Measurements
- NVIDIA Groq 3 LPX: The SRAM Decode Engine and the Arithmetic NVIDIA Won't Do
- NVIDIA Agrees to Buy Hugging Face: The Dependency Math Behind the $12.93B Headline
- LLM Quantization: Bit Layouts, Block Scales, and the VRAM Math
- RAG Retrieval Math: Embedding Memory, Vector Index Size, and the Latency Budget Nobody Calculates
- Expert Parallelism: The Router's Dilemma
- Pipeline Parallelism: Pumping Data Upstream
- Tensor Parallelism: Slicing the Silicon
- The 2026 Serving-Engine Churn Tax: vLLM 0.29, SGLang 0.5.19, and TensorRT-LLM Losing TensorRT
- UNISON and the Paused Fleet: Why Agent KV Traffic Breaks Every Cache Policy You Know
- vLLM vs SGLang: Architecture, Overhead, and When to Pick Which
- NVIDIA GPU Containers on Ubuntu 24.04
- AMD MI300X/MI350/MI400: More Memory, But Does the Math Win? — A Critical Silicon View of AMD Instinct vs NVIDIA
- NVIDIA B200 vs GB200: What MLPerf Training Actually Shows
- NVIDIA NVLink: Generations, Domains and Verification
- KV Cache Memory Growth Explorer
- LLM Inference Calculator & Break-Even Analyzer
- LLM Inference Economics: Costs & Cloud Break-Even
architecture26 articles
- Beam 1.0: Local RoPE, Global NoPE and Scalable Softmax
- Inside Cloudflare Clef: Prefill-Only Qwen and a Joint Schema Head
- Clef Omni: Audio, Video, Sparse Experts and Schema Decisions
- Inside EmbeddingGemma 2: Soft Tokens, PLE, Hybrid Attention, and MRL
- Fibo 2.0: Gist Compression, Dual Text Injection and Clean-Latent Editing
- Inside GLM-5.3-Flash: KDA, Sparse MLA, mHC, and 8-of-288 MoE
- Inside Kimi K3: Delta Attention, Depth Attention, and LatentMoE
- Kimi K3.1 Pro: Sparse Reads, Not a Smaller History
- Inside Kolibri 1: Hybrid RNoPE Attention, Sparse MoE, and FP8 Serving
- KITE: Scale the Model, Freeze the KV Bill — the Prefill Invariant arXiv:2609.27294 Actually Proves (and the Three Things It Does Not)
- LFM2.5-24B-A2B: Biased Routing, Short Convolutions and Ten KV Caches
- Ling-3.1-flash: Grouped Routing and Two Different MLA Caches
- Liquid d1-3B: Branching Decisions Without Decoding
- Mellum2.1: Sparse Experts, Local Attention and Selective YaRN
- Muse Spark Open: Norm-Preserving Gates Are Not a Norm Guarantee
- Nemotron 3.5 Bolt E4B: Block-Parallel Speculation with DFlash
- Nemotron 3.5 Bolt: Heterogeneous LatentMoE and the 24 GB Budget
- Nemotron 3.5 Embed: adding images without replacing the text index
- Nemotron 3.5 Super VL: Where Visual Token Compression Happens
- OlmoEarth v1.3: Attention Pooling, Matryoshka Distillation, and Searchlight
- pplx-embed-v2: 9B documents, 0.6B queries
- Qwen-Image 2.1: Block-Causal Attention, Prefix KV Cache, and Turbo
- DSpine for Qwen3.8-27B: Causal Features Through Depth
- Inside Qwen3.8-Flash-Next: Gated DeltaNet, Sparse Attention, and N-gram Memory
- Ripple-65B: Embedding Value Residuals and Hybrid Attention
- Step-5-Preview: Sparse Reads Are Not a Compressed KV Cache
kv-cache21 articles
- Random Attention vs Score-Based KV Eviction on Reasoning Traces
- Beam 1.0: Local RoPE, Global NoPE and Scalable Softmax
- BOOST: Concurrent Host and HBM Access for LLM Decode
- DeepSeek-V4.1-Flash KV-Cache Compression: What the Reported 890 Bytes per Token Includes
- FlashLoop, or: Looping a Transformer Quadruples the Bill and Then Throws Most of It Away
- HBF as the Third KV Tier: 24x Sessions or 5x Worse Latency - the Medium Is Fine, the Placement Policy Decides
- Kimi K3.1 Pro: Sparse Reads, Not a Smaller History
- The KV Cache Explained Visually
- Risk-Controlled KV Eviction: The Reliability Contract and Full-KV Fallback
- KITE: Scale the Model, Freeze the KV Bill — the Prefill Invariant arXiv:2609.27294 Actually Proves (and the Three Things It Does Not)
- KV-Cache Tensor Decomposition: Full-Rank Head and Layer Modes in Two Models
- KVSET: LRU Stack Distance for LLM Prefix-Cache Sizing
- LFM2.5-24B-A2B: Biased Routing, Short Convolutions and Ten KV Caches
- Ling-3.1-flash: Grouped Routing and Two Different MLA Caches
- Mellum2.1: Sparse Experts, Local Attention and Selective YaRN
- Nemotron 3.5 Super VL: Where Visual Token Compression Happens
- Qwen-Image 2.1: Block-Causal Attention, Prefix KV Cache, and Turbo
- DSpine for Qwen3.8-27B: Causal Features Through Depth
- Ripple-65B: Embedding Value Residuals and Hybrid Attention
- Step-5-Preview: Sparse Reads Are Not a Compressed KV Cache
- Understand AI Visually: Start Here
attention18 articles
- Random Attention vs Score-Based KV Eviction on Reasoning Traces
- Attention Explained Visually
- Beam 1.0: Local RoPE, Global NoPE and Scalable Softmax
- Inside Cloudflare Clef: Prefill-Only Qwen and a Joint Schema Head
- Clef Omni: Audio, Video, Sparse Experts and Schema Decisions
- Inside EmbeddingGemma 2: Soft Tokens, PLE, Hybrid Attention, and MRL
- Fibo 2.0: Gist Compression, Dual Text Injection and Clean-Latent Editing
- How LLMs Actually Work — An Animated Walkthrough
- Inside Kimi K3: Delta Attention, Depth Attention, and LatentMoE
- Kimi K3.1 Pro: Sparse Reads, Not a Smaller History
- Inside Kolibri 1: Hybrid RNoPE Attention, Sparse MoE, and FP8 Serving
- LFM2.5-24B-A2B: Biased Routing, Short Convolutions and Ten KV Caches
- Liquid d1-3B: Branching Decisions Without Decoding
- Mellum2.1: Sparse Experts, Local Attention and Selective YaRN
- Muse Spark Open: Norm-Preserving Gates Are Not a Norm Guarantee
- Qwen-Image 2.1: Block-Causal Attention, Prefix KV Cache, and Turbo
- Ripple-65B: Embedding Value Residuals and Hybrid Attention
- Understand AI Visually: Start Here
hardware18 articles
- BOOST: Concurrent Host and HBM Access for LLM Decode
- CXL-SSDs For LLM Prefix Caching: Byte-Addressability Buys Nothing Without Chunk Awareness
- CXMT at 10% of DRAM Revenue: What the Figures Show
- GPU Buying Guide for LLM Inference (2026)
- Selecting the Right GPU for Qwen3 Inference
- HBF as the Third KV Tier: 24x Sessions or 5x Worse Latency - the Medium Is Fine, the Placement Policy Decides
- HBM Capacity and KV-Cache Economics
- LLM Inference Math: From Theory to Hardware
- NVIDIA Groq 3 LPX: The SRAM Decode Engine and the Arithmetic NVIDIA Won't Do
- LLM Quantization: Bit Layouts, Block Scales, and the VRAM Math
- Expert Parallelism: The Router's Dilemma
- Pipeline Parallelism: Pumping Data Upstream
- Tensor Parallelism: Slicing the Silicon
- Snapdragon X2 on Linux: The 80-TOPS NPU Is Not the Story — 228 GB/s Is
- AMD's MI455X 34× Throughput Claim: What the Disclosure Establishes
- AMD MI300X/MI350/MI400: More Memory, But Does the Math Win? — A Critical Silicon View of AMD Instinct vs NVIDIA
- NVIDIA NVLink: Generations, Domains and Verification
- Scale-Up vs Scale-Out: Network Bandwidth and Collective Math
deep-learning16 articles
- Your Agent Fleet Is a Storage Array: The KV-Cache Tiering Math Behind Agentic Inference
- Selecting the Right GPU for Qwen3 Inference
- How LLMs Actually Work — An Animated Walkthrough
- The KV Cache: Bit-Exact Memory Math, GQA vs MQA vs MLA, and PagedAttention
- KV-Cache Glossary: every term in the serving stack
- LLM Inference Math: From Theory to Hardware
- LLM VRAM Requirements: A Mathematical Deep Dive
- LoRA and QLoRA Fine-Tuning: The Actual Memory Math
- LLM Quantization: Bit Layouts, Block Scales, and the VRAM Math
- Expert Parallelism: The Router's Dilemma
- Pipeline Parallelism: Pumping Data Upstream
- Tensor Parallelism: Slicing the Silicon
- UNISON and the Paused Fleet: Why Agent KV Traffic Breaks Every Cache Policy You Know
- KV Cache Memory Growth Explorer
- LLM Inference Calculator & Break-Even Analyzer
- LLM Inference Economics: Costs & Cloud Break-Even
gpu-memory15 articles
- Agent-Fleet Token Economics Is Cache Economics: Decomposing the 60x Dollar Spread
- Your Agent Fleet Is a Storage Array: The KV-Cache Tiering Math Behind Agentic Inference
- Random Attention vs Score-Based KV Eviction on Reasoning Traces
- BOOST: Concurrent Host and HBM Access for LLM Decode
- HBF as the Third KV Tier: 24x Sessions or 5x Worse Latency - the Medium Is Fine, the Placement Policy Decides
- HBM Capacity and KV-Cache Economics
- The KV Cache: Bit-Exact Memory Math, GQA vs MQA vs MLA, and PagedAttention
- KV-Cache Glossary: every term in the serving stack
- Who Pays for the KV Cache? Metering Rules Are a Pricing Decision in Disguise
- KITE: Scale the Model, Freeze the KV Bill — the Prefill Invariant arXiv:2609.27294 Actually Proves (and the Three Things It Does Not)
- KV-Cache Tensor Decomposition: Full-Rank Head and Layer Modes in Two Models
- KVSET: LRU Stack Distance for LLM Prefix-Cache Sizing
- LoRA and QLoRA Fine-Tuning: The Actual Memory Math
- RAG Retrieval Math: Embedding Memory, Vector Index Size, and the Latency Budget Nobody Calculates
- UNISON and the Paused Fleet: Why Agent KV Traffic Breaks Every Cache Policy You Know
moe15 articles
- Qwen3-Next-80B-A3B: Architecture and Serving Limits
- Beam 1.0: Local RoPE, Global NoPE and Scalable Softmax
- Clef Omni: Audio, Video, Sparse Experts and Schema Decisions
- Self-hosting DeepSeek V4 Pro: the actual math
- Inside GLM-5.3-Flash: KDA, Sparse MLA, mHC, and 8-of-288 MoE
- Inside Kimi K3: Delta Attention, Depth Attention, and LatentMoE
- Kimi K3.1 Pro: Sparse Reads, Not a Smaller History
- Inside Kolibri 1: Hybrid RNoPE Attention, Sparse MoE, and FP8 Serving
- LFM2.5-24B-A2B: Biased Routing, Short Convolutions and Ten KV Caches
- Ling-3.1-flash: Grouped Routing and Two Different MLA Caches
- Mellum2.1: Sparse Experts, Local Attention and Selective YaRN
- Muse Spark Open: Norm-Preserving Gates Are Not a Norm Guarantee
- Nemotron 3.5 Bolt: Heterogeneous LatentMoE and the 24 GB Budget
- Inside Qwen3.8-Flash-Next: Gated DeltaNet, Sparse Attention, and N-gram Memory
- Expert Parallelism: The Router's Dilemma
guide13 articles
- Granite Speech 5.0 TurboCTC: Pyramidal Conformer Serving
- Parakeet TDT vs Encoder–Decoder ASR: Production Trade-offs
- Attention Explained Visually
- Selecting the Right GPU for Qwen3 Inference
- How LLMs Actually Work — An Animated Walkthrough
- The KV Cache Explained Visually
- NVFP4 vs INT4 Explained Visually
- Expert Parallelism: The Router's Dilemma
- Pipeline Parallelism: Pumping Data Upstream
- Tensor Parallelism: Slicing the Silicon
- Understand AI Visually: Start Here
- NVIDIA GPU Containers on Ubuntu 24.04
- NVIDIA Error 802 on Ubuntu: Diagnose Before Trying nokaslr
quantization12 articles
- Disaggregated Quantization: Splitting the Prefill and Decode Price System Across Two Formats
- Inside Kolibri 1: Hybrid RNoPE Attention, Sparse MoE, and FP8 Serving
- Serving Qwen3.8 on MI350P: Kernels, Failures, and Measurements
- Nemotron 3.5 Bolt E4B: Block-Parallel Speculation with DFlash
- Nemotron 3.5 Bolt: Heterogeneous LatentMoE and the 24 GB Budget
- Nemotron 3.5 Embed: adding images without replacing the text index
- OlmoEarth v1.3: Attention Pooling, Matryoshka Distillation, and Searchlight
- PTQ Configuration Pricing: Predicting the Quantization Price Before You Build the Model
- LLM Quantization: Bit Layouts, Block Scales, and the VRAM Math
- Quantization Can Change Retrieval Results While Classification Accuracy Holds
- NVFP4 vs INT4 Explained Visually
- Understand AI Visually: Start Here
nvidia11 articles
- Qwen3-Next-80B-A3B: Architecture and Serving Limits
- Serving Qwen3.8 on MI350P: Kernels, Failures, and Measurements
- NVIDIA Groq 3 LPX: The SRAM Decode Engine and the Arithmetic NVIDIA Won't Do
- NVIDIA Agrees to Buy Hugging Face: The Dependency Math Behind the $12.93B Headline
- NVIDIA GPU Containers on Ubuntu 24.04
- NVIDIA Error 802 on Ubuntu: Diagnose Before Trying nokaslr
- AMD MI300X/MI350/MI400: More Memory, But Does the Math Win? — A Critical Silicon View of AMD Instinct vs NVIDIA
- NVIDIA B200 vs GB200: What MLPerf Training Actually Shows
- MLPerf Inference 2026: What a Result Says About Hardware and Software
- NVIDIA NVLink: Generations, Domains and Verification
- Scale-Up vs Scale-Out: Network Bandwidth and Collective Math
vllm8 articles
- llama.cpp / Ollama vs vLLM: Which One, When, and Why
- Serving Qwen3.8 on MI350P: Kernels, Failures, and Measurements
- Nemotron 3.5 Bolt E4B: Block-Parallel Speculation with DFlash
- NVIDIA Agrees to Buy Hugging Face: The Dependency Math Behind the $12.93B Headline
- The 2026 Serving-Engine Churn Tax: vLLM 0.29, SGLang 0.5.19, and TensorRT-LLM Losing TensorRT
- vLLM vs SGLang: Architecture, Overhead, and When to Pick Which
- KV Cache Memory Growth Explorer
- LLM Inference Calculator & Break-Even Analyzer
datacenter7 articles
- About Florian Zimmermeister
- Parakeet TDT vs Encoder–Decoder ASR: Production Trade-offs
- Self-hosting DeepSeek V4 Pro: the actual math
- Selecting the Right GPU for Qwen3 Inference
- LLM Inference Math: From Theory to Hardware
- LLM VRAM Requirements: A Mathematical Deep Dive
- NVIDIA Groq 3 LPX: The SRAM Decode Engine and the Arithmetic NVIDIA Won't Do
multimodal7 articles
- Clef Omni: Audio, Video, Sparse Experts and Schema Decisions
- Inside EmbeddingGemma 2: Soft Tokens, PLE, Hybrid Attention, and MRL
- Fibo 2.0: Gist Compression, Dual Text Injection and Clean-Latent Editing
- Nemotron 3.5 Embed: adding images without replacing the text index
- Nemotron 3.5 Super VL: Where Visual Token Compression Happens
- OlmoEarth v1.3: Attention Pooling, Matryoshka Distillation, and Searchlight
- Qwen-Image 2.1: Block-Causal Attention, Prefix KV Cache, and Turbo
qwen7 articles
- Qwen3-Next-80B-A3B: Architecture and Serving Limits
- Inside Cloudflare Clef: Prefill-Only Qwen and a Joint Schema Head
- Clef Omni: Audio, Video, Sparse Experts and Schema Decisions
- Fibo 2.0: Gist Compression, Dual Text Injection and Clean-Latent Editing
- Qwen-Image 2.1: Block-Causal Attention, Prefix KV Cache, and Turbo
- DSpine for Qwen3.8-27B: Causal Features Through Depth
- Inside Qwen3.8-Flash-Next: Gated DeltaNet, Sparse Attention, and N-gram Memory
compliance5 articles
economics5 articles
- Agent-Fleet Token Economics Is Cache Economics: Decomposing the 60x Dollar Spread
- The Cache-Read Price War of September 2026 That Wasn't Three-Sided
- Who Pays for the KV Cache? Metering Rules Are a Pricing Decision in Disguise
- KITE: Scale the Model, Freeze the KV Bill — the Prefill Invariant arXiv:2609.27294 Actually Proves (and the Three Things It Does Not)
- LLM Inference Economics: Costs & Cloud Break-Even
ai-act4 articles
amd4 articles
- Serving Qwen3.8 on MI350P: Kernels, Failures, and Measurements
- AMD's MI455X 34× Throughput Claim: What the Disclosure Establishes
- AMD MI300X/MI350/MI400: More Memory, But Does the Math Win? — A Critical Silicon View of AMD Instinct vs NVIDIA
- MLPerf Inference 2026: What a Result Says About Hardware and Software