Variance reduction for policy gradient with action-dependent factorized baselines

AI Impact Summary

This change introduces a variance-reduction technique for policy gradient methods using action-dependent factorized baselines. It implies integrating baselines that factorize across actions, which can lower gradient estimator variance and improve sample efficiency, potentially speeding up convergence in RL training. Engineers should plan for integrating this approach into existing RL pipelines and verify compatibility with current optimization routines and libraries.