Training Efficiency Experiment: REPA for Text-to-Image Model PRX-1.2B

AI Impact Summary

This document details an experimental logbook exploring training efficiency techniques for text-to-image models, specifically focusing on the PRX-1.2B model. The core approach involves aligning representations by adding a loss that directly supervises intermediate features using a frozen vision encoder. This technique, termed REPA, aims to accelerate convergence and improve training efficiency by leveraging a powerful representation space, mirroring the approach used in foundation models. The experiment uses a baseline configuration with fixed hyperparameters and a small dataset to provide a stable reference for evaluating the impact of these training interventions.