( Writing )

Blogs.

Paper summaries on attention, KV caches, pruning, and how transformers carry state, written to explain the idea first and the equations second.

Posts
4
On VLG
2
Series
papers_we_read ↗

( 01 — Paper summaries )

Latest

01 Recirculation Inference-time methods / Recurrence / State tracking Sep 2026 A training-free tweak that leaks a sliver of a deep-layer activation back into a shallow layer at every step, turning a feedforward transformer into something closer to a recurrent state tracker. 02 Keyless Attention: Value-Space Routing and Value-Only Caching KV cache / Efficient attention / Architecture Sep 2026 Delete the key projection, route attention through the values, and cache values only: an exact 50% KV-cache cut that matches standard attention on 4 of 5 models. 03 Why do LLMs attend to the first token? Attention sinks / Long context / Interpretability Oct 2025 Attention sinks look like wasted compute, but they give heads an off-switch against over-mixing: a learned way to keep token representations distinct in deep, long-context models. 04 LLM-Pruner: On the Structural Pruning of Large Language Models Pruning / Compression / LoRA Jun 2025 A task-agnostic structural pruner that finds coupled structures, scores them with gradient information, and recovers quality with LoRA: about 3 hours and 50K samples.

( 02 — Elsewhere )

Published blogposts

↗ Don’t Look Up (Every Token): Escaping Quadratic Complexity via Patterns and Algorithms ICLR 2026 Blogpost Track / with Tanvi Sharma & Vansh Agrawal ICLR 2026