KV Cache from scratch in nanoVLM — 38% generation speedup

AI Impact Summary

The implementation of KV Caching in nanoVLM demonstrates a significant optimization technique for autoregressive language model generation. By caching previously computed keys and values, the model avoids redundant computations, resulting in a 38% speedup in generation. This approach is particularly effective for long sequences where the quadratic memory/compute complexity of standard attention mechanisms becomes a bottleneck, and the implementation leverages a per-layer dictionary for efficient caching and updates.