← Papers & projects

Open-source engineering · 2026 · Upstream work

Open-Source Systems Engineering

RL infrastructure, long-context training, inference optimization, and reliable training pipelines

50%
Lower vLLM TP2 conversion peak
5.31×
Peak controlled Ray transfer speedup
FP8
Attention calibration correctness

I contribute to agentic post-training and ML systems, with the core work grouped below: RL infrastructure, long-context training, inference optimization, optimizer correctness, and training-data pipelines. Smaller runtime and integration fixes follow separately. Each result links to its implementation and validation scope.

RL infrastructure

  • NeMo RL #3943: kept teacher top-k payloads in the Ray object store and handed references to compatible student workers. Controlled 8×B200 transfer measurements achieved 4.44–5.31× speedup across 0.25–19.7 GB payloads, with driver RSS deltas of roughly 0.1–0.3 MB, matching checksums, and zero object spilling. These measure the transfer bottleneck, not end-to-end training speedup.
  • TRL #7294: corrected async GRPO and distillation checkpoint-resume bookkeeping when stale rollouts are dropped. Resume cursors advance past trained or dropped prompts without skipping unresolved gaps; dropped prompts do not count as training examples. 26 focused CPU tests passed, with one GPU-only skip, on the current head; model initialization and Trainer checkpoint machinery were mocked. Independent GPU verification of both trainers applies to an earlier head, not the current rebase.
  • Sanitized non-finite rollout log probabilities in NeMo RL #2962.
  • Validated the authoritative actor FSDP strategy in verl #7597.

Long-context training and model integration

  • Building Bridge-local Qwen4-Exp text-decoder support in Megatron Bridge #6315, including QSA, per-layer embeddings, gated residuals, and HF conversion. CPU unit checks passed; GPU integration and end-to-end training parity have not yet been established.
  • Fused GatedDeltaNet Q/K normalization for long-context SFT, exercised on 16×B200 in Megatron-LM #5396. The current GDN/GDN2 replay additionally passed forward/backward parity and deterministic-mode checks on all eight B200 ranks.
  • Added selective Mamba recompute, validated on 8×B200, in Megatron-LM #5463.
  • Reused packed chunkwise context-parallel metadata across GDN/KDA layers in Megatron-LM #7881. This targets eager packed training; GPU parity and throughput are pending.
  • Made packed QSA block scoring document-local in mcore-bridge #211. Selected IDs matched the masked reference; an 8K synthetic selector case was 2.70× faster, not a measured full-model speedup. Related work preserves BF16/FP16 gated-residual rounding in #212 and bounds PLE backward’s extra workspace through chunked reductions in #213.

Inference memory and correctness

  • vLLM #54699: preallocated aligned BF16 MoE weights and converted them in place using single-expert scratch storage. Exact-shape TP2 peak allocation fell from 7.88 to 3.94 GiB. Earlier validation loaded a real 120B checkpoint and passed generation plus 3/3 sleep–wake cycles. The latest follow-up zeroes raw padding on every conversion and reconstructs kernel views from restored tensor shapes, covering cross-process IPC and repeated reload/refit paths. These follow-ups have targeted SM90 and SM100 coverage; the real 120B lifecycle was not rerun on the latest head.
  • FlashInfer #4984: corrected global K/V calibration for ragged prefill with 16-bit queries and FP8 KV, covering scale combinations, causal modes, soft capping, output/LSE accuracy, and buffer aliasing. This patch is now included upstream. This is wrapper calibration work, not a new FA3 kernel; Hopper execution is outside the reported validation.
  • SGLang #39765: fixed Mamba cache-publication ordering before the overlap scheduler prepares the next dependent batch. This prevents stale KV-slot mappings when cache insertion replaces duplicate slots. 10 focused CPU tests passed on the rebased head; an earlier SM90 generation smoke was not rerun on that head. No inference throughput improvement is claimed.

Optimizer correctness

  • Stabilized Newton–Schulz updates for small-norm Muon inputs in NVIDIA NeMo Emerging Optimizers #230.
  • Addressed GDN/Muon gradient clipping, optimizer routing, and parameter-group isolation in Megatron-LM #5395, #5400, and #5431.
  • Preserved native Adam step counters through checkpoint restoration in Megatron-LM #7864. The CPU/DCP round-trip checks cover the next optimizer update; full CUDA training validation remains pending.

Training-data pipelines

  • Preserved source shards when shuffling streaming datasets in Hugging Face Datasets #8670. With 19 source shards, all four DataLoader workers can receive work instead of only one. Regression coverage includes deterministic epochs and checkpoint restoration; no multi-host training result is claimed.

Further runtime and integration work

  • SGLang #31621 / #38063: hardened skipped-tensor weight checks and added MXFP4 cold-JIT startup diagnostics when the optional FlashInfer cache is absent. The startup change is diagnostic, not a measured speedup.
  • Fixed recompute propagation and hybrid Mamba + attention + MoE runtime paths in vLLM/Vime #337.

  • Added order-preserving packing, NCCL warmup, and Muon correctness fixes across ModelScope ms-swift #9598, #9602, #9599, and #9591.

  • Hermes #100693: resolved local schema references before nested tool-argument coercion, preserving fail-safe behavior for external or cyclic references; 22 targeted tests passed.
  • Hermes #102549: added ownership-checked cleanup for idle pooled SSH backends and serialized stop/reconnect handling; 14 targeted tests passed.
  • Co-authored NeMo Gym #2726 for pickle-safe cross-process HTTP failures, allowing worker errors to cross process boundaries without breaking failure recovery.

Updated October 8, 2026. Validation scopes and source revisions are recorded in the linked PRs; targeted tests do not imply full-suite or deployment coverage.