L2D: Learning to Drive opens world's largest open-source self-driving dataset via LeRobot

AI Impact Summary

Yaak and LeRobot release L2D, the world's largest open-source self-driving dataset, comprising 90+ TB of multimodal data from 60 EVs across 30 German cities, captured with six RGB cameras plus full vehicle state. The dataset includes expert and student driving policies, natural language instructions, and future waypoints generated via OpenStreetMap, enabling end-to-end spatial intelligence models conditioned on language. This enables researchers to train and benchmark vision-language and LLM-conditioned control pipelines using the LeRobot training pipeline, and positions L2D alongside established datasets like Waymo and NuScenes as a primary open benchmark. The release implies a need for substantial storage and compute for ingestion and experimentation, with upcoming R5+ introducing natural language reasoning for sub-optimality in student policies.