Skip to content
Friendly disclaimer: flozi00 TechHub is a solo side-project next to a full-time job — personal learning notes, no official statements. Verify critical steps yourself.
← Home

Find your topic. Follow your curiosity.

Choose a model family or a technical topic to see the relevant articles together. For a guided introduction, follow a reading path below.

Learning & fundamentals

9 articles

Understand AI step by step: visual explanations, attention, KV cache and quantization.

Back to topics ↑

AI models

36 articles

Model families, architectures and practical applications, from Nemotron and Qwen to speech, images and embeddings.

Architecture, use cases and measurements are grouped by model family.

NVIDIA Nemotron

Qwen

Beam

Cloudflare Clef

DeepSeek

EmbeddingGemma

Fibo

GLM

IBM Granite Speech

JetBrains Mellum

Kimi

Kolibri

Ling

Liquid AI

Muse

NVIDIA Parakeet

OlmoEarth

Perplexity pplx-embed

Ripple

StepFun

TypeSafe Jev

Back to topics ↑

Inference & optimization

60 articles

Serving, memory, quantization, GPU kernels and costs for operating AI systems.

KV cache & memory

Serving & deployment

Quantization & fine-tuning

GPU kernels & scaling

Costs & capacity

Back to topics ↑

Embeddings & RAG

6 articles

Embedding models, vector search, retrieval quality and memory budgets.

Back to topics ↑

Hardware & systems

17 articles

Compare GPUs, memory and networking, and set up Linux, drivers and containers.

Back to topics ↑

Benchmarks & evaluation

7 articles

Assess measurements, model evaluations and datasets, and interpret results.

Back to topics ↑

AI regulation & transparency

5 articles

EU AI Act, data protection, disclosure and the technical limits of watermarking.

Back to topics ↑

Calculators & tools

3 articles

Calculate VRAM, KV-cache growth and inference costs interactively.

Back to topics ↑

Learn step by step

Start with a question. Each reading path puts the foundations, practical tools and deeper analyses in a useful order.

New to AI? See how it works.

Start with attention, KV cache and quantization. Continue with an individual visual explanation in every technical article, from model architectures to GPUs and Linux.

  1. Visual Tech Learning: From AI Basics to Systems

    Choose your starting point; no machine-learning background needed.

  2. Attention Explained Visually: Queries, Keys and Values

    Select a token and calculate how it mixes information from the context.

  3. KV Cache Explained Visually: Prefill, Decode and Memory

    Watch which computations are reused as a model generates text.

  4. NVFP4 vs INT4: A Visual Guide to 4-Bit Quantization

    Move a number to compare INT4 and NVFP4 rounding and scales.

  5. How LLMs Actually Work — An Animated Walkthrough

    Connect the pieces: tokens, embeddings, Transformer blocks and sampling.

How much VRAM does my LLM need?

Start with the memory budget, then calculate weights, context and runtime overhead. Fine-tuning has a separate budget.

  1. LLM VRAM Requirements: Weights, Quantization & KV Cache

    Understand the inference memory budget and its assumptions.

  2. LLM VRAM Calculator: GPU Memory & Inference Estimates

    Calculate model fit for a chosen GPU, precision and context.

  3. LLM Quantization: FP8, INT4, GGUF & VRAM Requirements

    Separate weight compression from quality and runtime constraints.

  4. KV Cache Explained: Memory Formula, GQA, MQA & MLA

    Work out the memory consumed by context and concurrent requests.

  5. LoRA and QLoRA Fine-Tuning: The Actual Memory Math

    Plan adapters, optimizer states and activations for fine-tuning.

Which GPU and interconnect fit my workload?

Use model fit and bandwidth for inference planning. Read training benchmarks as system results and distinguish them from purchasing evidence.

  1. GPU Buying Guide for LLM Inference (2026)

    Compare specifications, quotes and workload requirements.

  2. GPU for Qwen3 Inference: VRAM, Bandwidth & Model Fit

    Apply roofline math to concrete Qwen3 and GPU pairings.

  3. NVIDIA B200 vs GB200: Differences & MLPerf Benchmarks

    Compare published B200 and GB200 training submissions fairly.

  4. NVIDIA NVLink: Bandwidth, NVSwitch & GPU Topology

    Understand what an NVLink domain connects.

  5. Scale-Up vs Scale-Out: NVLink and All-Reduce

    Calculate communication within and between GPU servers.

  6. AMD Instinct vs NVIDIA: Memory and Serving Limits

    Compare AMD and NVIDIA memory specifications and their limits.

How do I choose and operate an LLM serving engine?

Keep local deployment, engine selection, release maintenance and cost calculations separate. Each guide answers a different operational question.

  1. llama.cpp vs Ollama vs vLLM: Local LLM Serving Compared

    Choose between local runtimes and GPU serving.

  2. vLLM vs SGLang: Serving, Caching & When to Use Each

    Compare vLLM and SGLang for your request patterns.

  3. LLM Serving Engines 2026: Upgrades and Version Pinning

    Budget for upgrades, version pinning and rollback.

  4. LLM Inference Performance: Roofline Math & GPU Bandwidth

    Understand the formulas behind decode and prefill limits.

  5. MI350P vLLM Optimization: Kernels and Measurements

    Audit an inference campaign and follow GDN, QSA and MoE kernel changes.

  6. LLM Inference Economics: Costs & Cloud Break-Even

    Compare self-hosting costs with an API under explicit assumptions.

How do I size and manage a KV cache?

Move from per-token memory to working sets and prefix reuse. Sizing, eviction and offload solve different problems.

  1. KV Cache Explained Visually: Prefill, Decode and Memory

    See prefill, decode and cache reuse one step at a time.

  2. KV Cache Explained: Memory Formula, GQA, MQA & MLA

    Learn the per-token formula and attention-layout differences.

  3. KV Cache Memory Growth Explorer

    Explore how growing context consumes free GPU memory.

  4. KV-Cache Glossary: GQA, MLA, TTFT and Eviction

    Look up serving terms and their precise definitions.

  5. KVSET: Size an LLM Prefix Cache with LRU Traces

    Derive a cache-capacity curve from an LRU trace.

  6. LLM Prefix Caching: LRU Eviction and Its Limits

    Evaluate LRU eviction for different workloads and capacities.

  7. Agent Inference: KV-Cache Traffic and Memory Tiers

    Calculate agent working sets and memory-tier traffic.

What changes between model architectures?

Learn the basic token-to-output flow, then compare released language and multimodal checkpoints. Architecture guides describe mechanisms and serving limits.

  1. Attention Explained Visually: Queries, Keys and Values

    Calculate one attention head before exploring whole architectures.

  2. How LLMs Actually Work — An Animated Walkthrough

    Follow embeddings, attention and sampling in an animated walkthrough.

  3. Qwen-Image 2.1: Attention, KV Cache and Turbo Math

    Inspect block-causal diffusion attention, prefix caching and the Turbo schedule.

  4. Fibo 2.0 Architecture: Gist Tokens and Editing Costs

    Trace adaptive gist tokens, dual text injection and clean-latent editing costs.

  5. OlmoEarth v1.3: pooler, Matryoshka student, Searchlight

    Trace spatial attention pooling, nested distillation and seam-aware embedding serving.

  6. Qwen3-Next-80B-A3B: Architecture and Serving Limits

    Inspect Qwen3-Next hybrid attention and sparse experts.

  7. LFM2.5-24B-A2B: Routing and Cache Architecture

    Separate biased expert selection from mixture weights and derive hybrid LFM2.5 cache memory.

  8. Mellum2.1: MoE, Sliding Attention and Selective YaRN

    Derive Mellum2.1 local/global attention, selective YaRN and hybrid cache memory.

  9. Beam 1.0: NoPE, Softmax Scaling and KV Memory

    Trace local RoPE and global NoPE, derive SSMax entropy effects and distinguish KV payload from allocation.

  10. Ripple-65B: Value Residuals and Hybrid KV Cache

    Trace embedding-seeded value residuals, PeriNorm and the difference between windowed reads and compact cache storage.

  11. Kolibri 1 Architecture: Hybrid Attention, MoE and FP8

    Read Kolibri 1 architecture and its serving caveats.

  12. Kimi K3 Architecture: KDA, MLA and LatentMoE

    Read Kimi K3 recurrence, MLA and LatentMoE routing.

  13. Kimi K3.1 Pro: DSA Indexer and KV-Cache Limits

    Inspect K3.1 NoPE indexing, sparse MLA reads and the distinction between read width and retained history.

  14. Ling-3.1-flash: KDA, MLA and Actual Cache Layouts

    Derive Ling grouped routing and the difference between expanded and absorbed MLA caches.

  15. Step-5-Preview: Sparse GQA, Indexer and KV Memory

    Trace Step-5 block indexing and separate sparse K/V reads from retained cache capacity.

  16. Nemotron 3.5 Super VL: Visual Tokens and Video Pruning

    Trace Super VL visual patches, paired frames and change-based pruning; separate token savings from vision work.

  17. Nemotron 3.5 Bolt: LatentMoE, IQ2 and Memory Budgets

    Inspect Bolt heterogeneous LatentMoE, packed IQ2 weights and the separate runtime memory budget.

  18. Nemotron E4B DFlash: Block Speculation and Serving Math

    Derive E4B DFlash block proposals, exact verification and the latency/concurrency break-even point.

  19. Qwen3.8 DSpine: Causal Drafting, Refinement and Cache

    Trace adjacent causal draft writes, serial candidate refinement and the dense-cache serving boundary.

  20. Muse Spark Open: Residual Gates and Normalized Experts

    Distinguish Muse attention, routing and residual gates, and derive the limits of norm-preserving mixing.

  21. Inside GLM-5.3-Flash: KDA, Sparse MLA, mHC, and 8-of-288 MoE

    Inspect GLM-5.3-Flash attention, residuals and cache shapes.

  22. Cloudflare Clef: Qwen Backbones and Schema Scoring

    Understand Clef decision scoring and its prefill-only design.

  23. Liquid d1-3B: Shared-State Branch Inference

    Derive shared-state branch inference and exact split-attention merging in d1-3B.

  24. Clef Omni: Audio, Video, MoE and Schema Scoring

    Follow Clef Omni audio/video fusion, sparse experts and schema scoring.

  25. Granite Speech 5.0 TurboCTC Architecture

    Trace pyramidal Conformer shapes, block attention and encoder-only CTC decoding.

  26. Nemotron 3.5 Transcribe: Architecture and Dataset Auditing

    Connect promptable speech architecture to applications and evidence-based audio–transcript audits.

How do I compare embeddings for retrieval?

Start with storage and latency budgets, compare single-vector and late-interaction approaches, then examine quantization limits.

  1. RAG Memory Math: Embeddings, HNSW and Rerankers

    Budget embedding weights, vector-index storage and retrieval latency.

  2. EmbeddingGemma 2 Architecture: Soft Tokens, PLE and MRL

    Inspect an embedding encoder, pooling and production limits.

  3. Nemotron 3.5 Embed: compatible multimodal retrieval

    Trace frozen text-space compatibility, visual alignment, distillation and serving dimensions.

  4. pplx-embed-v2: 9B documents, 0.6B queries

    Compare late interaction and asymmetric document/query encoding.

  5. 4-Bit Quantization: Why Retrieval Rankings Change

    Distinguish retrieval-ranking changes from losses in relevance.

How do I get NVIDIA GPUs working on Ubuntu?

Separate installation and container verification from CUDA error diagnosis. A reported workaround is not a universal fix.

  1. NVIDIA GPU Containers on Ubuntu 24.04

    Install the host driver and verify GPU access inside Docker.

  2. NVIDIA Error 802 on Ubuntu: Diagnose Before Trying nokaslr

    Diagnose CUDA error 802 before considering nokaslr.

Browse all articles by date →