Tag: ai-ml
All the articles with the tag "ai-ml".
-
AI/ML - How Speculative Decoding Speeds Up LLM Generation
A practical explanation of speculative decoding: how a small draft model proposes tokens, a larger model verifies them in parallel, and rejection sampling preserves the target model's output distribution.
-
AI/ML - How the LLM KV Cache Works
How the LLM key-value cache speeds up token-by-token generation, what it stores, why it consumes GPU memory, and how paged allocation helps serving systems.
-
AI/ML - How BPE Tokenization Works in Large Language Models
How Byte Pair Encoding (BPE) tokenization works in large language models: the training algorithm, encoding and decoding, tiktoken and sentencepiece implementations, vocabulary construction, and why subword tokenization strikes the right balance between characters and words.
-
AI/ML - How the Transformer Attention Mechanism Works
How the Transformer attention mechanism works: intuition behind queries/keys/values, scaled dot-product attention, multi-head attention, positional encoding, and a source code walkthrough from PyTorch and the original paper.