RL Fine-Tuning Setup: Policy and Reference Models

Using the SFT model as initial policy, keeping a frozen reference model, and the actor-critic framework.