Enable nonlinear computation in deep linear networks

AI Impact Summary

This change enables nonlinear computation within deep linear networks, allowing activations or other nonlinear components to be integrated into architectures previously limited to linear mappings. By increasing expressiveness, models can better fit nonlinear data patterns, potentially boosting accuracy on tasks with complex relationships. However, introducing nonlinearity alters optimization dynamics, which can affect convergence and training stability, and will require updated regularization, initialization, and evaluation strategies. Teams should map where nonlinearities should be added in existing templates, adjust training pipelines to monitor gradient norms and convergence, and revise hyperparameters to accommodate nonlinearity.