nanoVLM: PyTorch Vision-Language training with google/siglip-base-patch16-224 and HuggingFaceTB/SmolLM2-135M

AI Impact Summary

nanoVLM provides a minimal PyTorch toolkit to train a Vision-Language Model by aligning a SigLIP-based vision encoder with a SmolLM2-135M language backbone via a Modality Projection module. The repo supports quick-start training on Colab, uses Hugging Face load_dataset for data, and logs with Weights & Biases, making it accessible for experimentation and education. While attractive for rapid prototyping, this setup relies on pre-trained backbones, a relatively small ~1.7M-sample dataset, and Colab hardware, which will limit scalability and reproducibility without migrating to dedicated GPUs, distributed training, and a robust data pipeline.