Minions: Reducing Cloud Costs with On-Device LLMs

AI Impact Summary

This announcement details the development of 'Minions,' a new approach to leveraging small language models (LLMs) on-device to reduce cloud compute costs. By shifting workloads to consumer devices and employing a collaborative protocol between small and large models, the team achieved significant cost savings while maintaining high performance. This represents a shift in strategy towards more efficient AI utilization, particularly for tasks involving long contexts and complex reasoning, and highlights the potential for consumer hardware to play a larger role in AI processing.