Smolagents adds Vision Language Model (VLM) support

AI Impact Summary

Smolagents now supports Vision Language Models (VLMs), unlocking the ability for agents to natively process visual information from web pages. This expands agent capabilities beyond text-based interactions, enabling tasks like autonomous web browsing and understanding visual content. The implementation utilizes a callback mechanism to dynamically add image observations to the agent's memory during each step, allowing agents to react to visual changes in their environment.