Writings

• Nov 2025
  Off-Policy Variance Reduction through Baseline Placement

  Is a single reward baseline the right structure for off-policy RL in LLMs? A theory of multiple baseline placements, their joint variance optimum, and the variance cost of estimating the baselines.