We revisit DAgger for multi-turn language-model agents, combining student–teacher rollouts with teacher-provided supervision to address covariate shift while retaining dense learning signals.
On SWE-bench Verified, this approach improves over the strongest post-training baseline by 3.9 points for 4B models and 3.6 points for 8B models.
Exploration-Driven Optimization (EDO) incorporates exploration incentives into iterative DPO and GRPO, preserving solution diversity for inference-time reasoning.
EDO improves reasoning on in-distribution and out-of-distribution benchmarks while maintaining model entropy and stabilizing reinforcement learning against over-optimization.
We organize the fast-growing landscape of RL with diffusion and flow policies into a unified taxonomy, clarifying the design choices behind existing algorithms.
We release a modular, JAX-based toolkit with JIT-compiled training and standardized benchmarks, making algorithms easy to compose, compare, and select across generative-modeling and robotics tasks.
We cast online diffusion RL through a measure-matching lens, generalizing the reweighting scheme from softmax to general monotonic functions.
By constructing a virtual target policy and minimizing divergence via reweighted flow matching, GeMPO enables flexible weight design and justifies negative reweighting that actively repels the policy from suboptimal actions.
We derive spectral representations from a decomposition of the transition operator, yielding an effective abstraction of the system dynamics with a clear theoretical characterization for downstream policy optimization.
The framework covers latent-variable and energy-based dynamics, extends to POMDPs, and matches or exceeds strong baselines on 20+ DeepMind Control Suite tasks.
Kimi K2 is a 1T-parameter (32B active) mixture-of-experts model trained with the MuonClip optimizer for stable large-scale pre-training, achieving strong agentic and coding results (e.g., 65.8 on SWE-Bench Verified).
As part of the Kimi Team during my internship at Moonshot.AI, I worked on strengthening the model’s interactive tool-use capability across agent scaffolding, supervised fine-tuning, and reinforcement learning.
We Introduce pathwise KL to estimate and control the divergences between two diffusion policies;
Leveraging pathwise KL, we propose an actor-critic framework with two-time-scale temporal difference learning to efficiently optimize diffusion policies with behavior regularization.
We leverages the flexibility of diffusion models and extract spectral representations (Diff-SR) that capture the dynamics structure.
Diff-SR is able to represent the value function of any policy sufficiently, paving the way for efficient planning and exploration for downstream RL optimization.
We identify two pain points in offline-to-online reinforcement learning: 1) value overestimation causes fluctuations during learning, and 2) the primacy bias hinders the policy from further improvement.
With the proposed Continual Policy Revitalization, we can fine-tune pret-rained policies efficiently and stably.
We explore the idea of rehearsal for offline reinforcement learning, which generates diverse while eligible dynamics models using extremely limited amount of data and optimizes an contextual policy with the generated models.
By recognizing the context, the policy is able to generalize to the environment at hand during the online stage.
We identify failure modes of existing return-conditioned decision-making systems, and suggest to use advantages as the property token for conditional generation.
Real-world RL applications are with data limitations, such as limited tasks and limited behavior diversity.
We develop GENTLE, a simple yet effective task representation learning method to extract generalizable and accurate task representations from offline contextual datasets.
