( Research output )

Publications.

Efficient attention, long-context inference, and diffusion language models. Every entry links to the paper and code, with copy-ready BibTeX.

Accepted
7
Oral
1
Profile
Scholar ↗

( 01 — Papers & blogposts )

01

Dropping the Anchor: Statistical Context Summarization for Distributed Systems via Pulsar Attention

Aryan Sood, Shantanu Acharya, Gaurav Kumar Nayak

Findings of EMNLP 2026

Developed Pulsar Attention for distributed long-context inference, replacing static anchor duplication with content-aware statistical summaries and reporting up to 4.7% accuracy gain over dense attention, 6.1% over Star Attention, and 3.3x lower Phase-1 FLOPs. Also accepted at the AdaptFM Workshop on Resource-Adaptive Foundation Model Inference, ICML 2026.

Interactive explainer

Star Attention prepends a full copy of block 1 to every GPU; Pulsar sends a 64-token sink plus Max-IDF chunks (rare tokens) from earlier blocks, then discards that prefix and merges the query globally.
★ Highlight EMNLP 2026 Findings 2026 · conference
02

Are You Sure You're Sure? Two Confounds in a Sycophancy Benchmark

Atharv Gupta*, Akshat Jindal*, Lavanya Nigam†, Aryan Sood†

* equal contribution (first authors) · † equal contribution (second authors)

First Workshop on Reliable Evaluation for Language Models (JUDGe), NeurIPS 2026 · Oral

Audits SycEval and shows two template confounds: naming a target answer raises the follow rate by 14.1–49.5 pp and, once controlled, reverses the reported timing effect on three of five model conditions; relocating the output-format instruction flips its effect across models. Closes with three checks benchmark authors can run on their own templates.

Interactive explainer

SycEval's preemptive template names an answer and its in-context one doesn't, so timing and naming move together. Adding the missing named in-context arm separates them. Effects in percentage points, from the paper.
★ Oral NeurIPS 2026 2026 · workshop
03

More Than a Quick Glance: Overcoming the Greedy Bias in KV-Cache Compression

Aryan Sood, Tanvi Sharma, Vansh Agrawal

ICBINB Workshop (I Can't Believe It's Not Better), ICLR 2026

Introduced LASER-KV, a KV-cache compression method achieving up to 10% higher accuracy at 128k context over greedy pruning baselines via block-wise accumulation and Exact-LSH sampling.

Interactive explainer

Bars show attention scores. Greedy top-k keeps the spikes and drops low-attention supporting facts; LASER-KV protects sink and local tokens (divisor n) and splits the recall budget between exact attention and LSH.
ICLR 2026 2026 · workshop
04

Lost in Interpolation: Why Predictive Feedback Fails in Diffusion Language Models

Lavanya Nigam, Ishaan Bansal, Aryan Sood, Vidit Aggarwal, Gaurav Kumar Nayak

Non-Autoregressive Language Models Workshop, COLM 2026

Introduced spherical soft-masking for diffusion LLMs, replacing Euclidean LERP feedback with SLERP over Fréchet means to mitigate embedding norm collapse during MDLM fine-tuning.

Interactive explainer

The mask and predicted-token embeddings sit ~73° apart on a hypersphere. LERP feedback cuts through the sphere and shrinks the norm; Spherical Soft-Masking takes a Fréchet mean and SLERPs along the surface.
COLM 2026 2026 · workshop
05

BRIEF: Bounded-Context Regeneration of Long Documents with Verifiable Structural Topology

Aryan Sood, Soham Bhattacharjee, Pratinav Seth

Third Workshop on Long-Context Foundation Models (LCFM), NeurIPS 2026

Regenerates long legal and financial documents one clause at a time from short structural briefs (number path, required terms, cross-references), never showing the generator source prose. Keeps cross-reference fidelity near 0.98 with a near-constant ~86-token maximum prompt, and improves verifiable section and cross-reference topology over one-pass generation on CUAD and EDGAR EX-10.

Interactive explainer

BRIEF writes each clause from a short brief (number path, terms, cross-references), so every call stays around 86 tokens while one-pass context grows with the document. Numbers from the paper's synthetic length sweep.
NeurIPS 2026 2026 · workshop
06

Underscoring the Problem: Why Softpick Fails at Initialization

Aryan Sood, Jaikaran Singh, Ishaan Bansal

Third Workshop on Long-Context Foundation Models (LCFM), NeurIPS 2026

Reframes softpick's failure at scale as a normalization problem by splitting its denominator into forward and backward roles. Traces the failure to initialization (rows with zero positive mass and gradient norms above 1012) and identifies a stop-gradient operator that trains from scratch and, at 230M parameters, matches softpick on quantization with fewer dead heads and better passkey retrieval.

Interactive explainer

Softpick rectifies scores so irrelevant keys get exactly 0. When every score in a row is ≤ 0, D+ = 0 and the whole row is dead; early causal rows see few keys, so this happens at initialisation.
NeurIPS 2026 2026 · workshop
07

Don't Look Up (Every Token): Escaping Quadratic Complexity via Patterns and Algorithms

Aryan Sood, Tanvi Sharma, Vansh Agrawal

ICLR 2026 Blogpost Track

Surveyed sparsity-based efficiency methods for transformers, organizing them into a unified taxonomy distinguishing static structural patterns from dynamic, attention-driven approaches.

Interactive explainer

Causal attention masks for a 20-token sequence. Static patterns fix the mask in advance; dynamic ones pick keys from the content, so the mask changes with the input.
Blog
ICLR 2026 2026 · blog