Publications
Research outputs across efficient attention, long-context inference, and diffusion language models.
Dropping the Anchor: Statistical Summarization for Distributed Systems via Pulsar Attention
- Developed Pulsar Attention for distributed long-context inference, replacing static anchor duplication with content-aware summaries and reporting up to 4.7% accuracy gain over dense attention, 6.1% over Star Attention, and 3.3x lower Phase-1 FLOPs.
@inproceedings{sood2026dropping,
title={Dropping the Anchor: Statistical Context Summarization for Distributed Systems via Pulsar Attention},
author={Aryan Sood and Shantanu Acharya},
booktitle={AdaptFM: Resource-Adaptive Foundation Model Inference},
year={2026},
url={https://openreview.net/forum?id=uCthChFfbe}
}
More Than a Quick Glance: Overcoming the Greedy Bias in KV-Cache Compression
- Introduced LASER-KV, a KV-cache compression method achieving up to 10% higher accuracy at 128k context over greedy pruning baselines via block-wise accumulation and Exact-LSH sampling.
@article{DBLP:journals/corr/abs-2602-02199,
publtype={informal},
author={Aryan Sood and Tanvi Sharma and Vansh Agrawal},
title={More Than a Quick Glance: Overcoming the Greedy Bias in KV-Cache Compression},
year={2026},
month={February},
journal={CoRR},
volume={abs/2602.02199},
url={https://doi.org/10.48550/arXiv.2602.02199}
}
Lost in Interpolation: Why Predictive Feedback Fails in Diffusion Language Models
- Introduced spherical soft-masking for diffusion LLMs, replacing Euclidean LERP feedback with SLERP over Frechet means to mitigate embedding norm collapse during MDLM fine-tuning.
@article{sood2026lostinterpolation,
title={Lost in Interpolation: Why Predictive Feedback Fails in Diffusion Language Models},
author={Sood, Aryan and Bansal, Ishaan and Nigam, Lavanya and Aggarwal, Vidit},
journal={Workshop paper under review},
year={2026}
}
Don't Look Up (Every Token): Escaping Quadratic Complexity via Patterns and Algorithms
- Surveyed sparsity-based efficiency methods for transformers, organizing them into a unified taxonomy distinguishing static structural patterns from dynamic, attention-driven approaches.
@inproceedings{sood2026dontlookup,
author={Sood, Aryan and Sharma, Tanvi and Agrawal, Vansh},
title={Don't Look Up (Every Token): Escaping Quadratic Complexity via Geometric Patterns and Algorithms},
booktitle={ICLR Blogposts 2026},
year={2026},
url={https://iclr-blogposts.github.io/2026/blog/2026/sparsity/}
}