ICML 2026: Papers That Matter
Our favorite papers from Seoul across Nebius’ focus areas.
The International Conference on Machine Learning (ICML) is the oldest of the major AI conferences, with roots going back to 1980, prehistory in a field that now reinvents itself every few months.
This year’s edition runs July 6 to 11 in Seoul, against a backdrop of scaling pressure and sharpening debates over how AI gets built and adopted. ICML hasn’t been spared the field’s exponential growth either: 2026 drew a record 23,918 submissions, nearly double the 12,107 of a year earlier.
The scale brought the coordination strain you’d expect, and review drew its share of controversy: to keep LLM-written reviews in check, the conference asked reviewers to disclose any AI use and watermarked submission PDFs with hidden prompts, then desk-rejected 497 papers whose authors were caught submitting machine-generated reviews, about 2% of submissions.
But the program that opened at Seoul’s COEX center was a strong one, drawn from a 26.6% acceptance rate and headlined by AMI Labs’ Pascale Fung’s keynote Towards AI Agents in the Real World.
Beyond being a Silver sponsor, Nebius contributed two posters to ICML 2026: SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale, on building better data for coding agents, and LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding, on accelerating LLM inference.
Nebius builds around a few focused bets, so we asked the people closest to that work to read through the program and flag the papers they found most interesting. Our panel has experts from the R&D team, key verticals, and Tavily, the web access layer for AI agents that Nebius acquired earlier this year. These are their picks, a reflection of where our attention goes rather than a ranking of the conference’s best, much like our earlier ICLR list.
Sorting through 6,352 accepted papers is no small task. The result spans five categories: Agentic AI, AI Systems (inference and infrastructure), Physical AI, Healthcare & Life Sciences, and a new section: Sustainable AI. The winning authors in each category receive Nebius cloud credits.
Winners in the Agentic AI category will also receive a grant of Tavily credits, giving them the tools to test Retrieval-Augmented Generation, real-time knowledge retrieval, and autonomous research agents.
Agentic AI
Agentic AI sits at the core of Nebius' research. We’ve built a pipeline that turns real GitHub repositories into agent-ready tasks — powering both the SWE-rebench leaderboard, our live ranking of coding agents on fresh real-world GitHub tasks, and SWE-rebench V2, the dataset that teaches agents to code. Tavily, our recent acquisition, equips AI agents with web search and retrieval built specifically for LLMs. It’s rapidly becoming a staple of AI research, with more than 200 peer-reviewed publications already citing Tavily.
ICML 2026 is pushing in the same direction, and then some: this year’s standouts are less about flashy demos and more about measuring agents honestly, with long-horizon benchmarks and hard questions about where agentic RL breaks down. Reinforcement learning, multi-agent setups, and tool use run through nearly every top paper, and deep research agents emerge as a category of their own.
We chose these together with Tavily, whose CEO Rotem Weiss joined the panel as a judge. The winning papers in this category will receive Tavily credits on top of the Nebius credits awarded across every category.
Reinforcement Learning with Evolving Rubrics for Deep Research
Reinforcement learning with verifiable rewards works when answers are short and checkable. Deep research is neither: the agent searches the web and writes a long, well-attributed report with no single answer to check against, so open agents have been trained on short-form QA instead, a poor stand-in. RLER (Reinforcement Learning with Evolving Rubrics) fixes that by letting the grading rubrics co-evolve with the policy during training, folding in freshly searched evidence as it goes. The result, DR Tulu-8B, is the first fully open model trained end-to-end for long-form deep research: across four benchmarks it beats the strongest open competitor, Alibaba’s Tongyi DeepResearch, by 15.6% on average, edges out OpenAI Deep Research, and runs 1,000x cheaper per query.
Characterizing Agents in Production
How do teams actually build agents once real users depend on them? There has been almost no public data, since that knowledge stays inside the companies running the systems. This study is the first systematic look, drawing on a survey of 86 practitioners and 20 case-study interviews across 26 domains, and the picture cuts against the research zeitgeist: 68% of deployed agents run at most 10 steps before a human steps in, 70% just prompt off-the-shelf models rather than tune weights, and 74% lean mainly on human evaluation. Reliability is the top concern practitioners report, and they engineer it through system design rather than better models.
Understanding Reasoning Collapse in LLM Agent Reinforcement Learning
Entropy is the metric the field watches to judge whether RL training is healthy, and it can lie. This paper shows multi-turn agents can hold entropy steady while sliding into"template collaps”: the reasoning still looks varied but is nearly identical for every input, because the model has stopped listening to the question. Their fix, RAGEN-2, catches this with mutual-information probes that test whether a reasoning trace can be matched back to its prompt, then trains only on the prompts where rewards still vary (SNR-Aware Filtering). Across planning, math, web navigation, and code execution, mutual information tracks final performance far better than entropy.
LiveNewsBench: Evaluating LLM Web Search Capabilities with Freshly Curated News
Static QA benchmarks can’t tell whether a search agent actually found the answer or just remembered it. LiveNewsBench gets around that by generating question-answer pairs from freshly published news, so answers sit beyond any model’s training data and take multi-hop searches and page visits to reach. Without internet access, frontier models top out at 21.5%, versus over 70% on FreshQA, a good sign the questions resist memorization. Turn search on and the field spreads out fast, from 11% for a small open model to 90% for GPT-5.2's official web search API.
Stream RAG: Instant and Accurate Spoken Dialogue Systems with Streaming Tool Usage
Voice assistants are moving from the old three-stage pipeline of speech recognition, language model, and speech synthesis to single models that listen and talk directly. Without factual grounding those models hallucinate, and bolting web search on adds enough latency to break the conversation. Stream RAG predicts search queries while the user is still talking, with a post-training recipe that teaches the model when to call tools and how to speak the retrieved text back. On AudioCRAG, a spoken version of CRAG built for this paper, accuracy jumps by over 20 points, and on SLUE-SQA it beats pipeline systems while cutting latency by up to 57%, with the same trick working for typed input too.
AI Systems
Inference and infrastructure are the defining research bets at Nebius, from serving stacks that squeeze more out of every GPU to the training systems underneath them. It is also the single largest slice of ICML 2026, and one of the fastest-moving.
The papers attack efficiency from every angle: quantization down to four bits, sparse and pruned inference, smarter GPU serving, and learning at test time, with a growing share aimed squarely at reasoning models and the cost of letting them think. One even asks whether we can trust what these models look at while they reason, and streams that evidence in real time.
ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training
Quantization has a blind spot: reasoning. Shrinking a model’s weights to four bits cuts the cost of running it but wrecks chain-of-thought accuracy in ways existing methods can’t repair. ReQAT traces the damage to one spot, the digits and operators where the model has to commit to an exact answer and a little rounding can derail the whole chain, then drills training on exactly those places. The payoff is four-bit (FP4) models that beat the same models fine-tuned at full sixteen-bit precision (BF16) while running up to 3.9x faster on DGX Spark.
DELTA4: Sparse Matrix-Vector Multiplication for Low Sparsity
Unstructured pruning has long promised memory savings it rarely delivers: prune an LLM to 50% sparsity and standard sparse kernels run slower than the dense math they replace. The reason is historical, since GPU sparse matrix-vector libraries were tuned for the 99%-sparse matrices of scientific computing, not the 30-90% range of pruned models. DELTA4 co-designs a compact storage format and matching kernel that fit the hardware’s execution model, with no tensor cores or format-specific precomputation. At 50% sparsity it is the first to deliver both a 1.5x memory cut and a 1.2 to 1.5x speedup, so a pruned Llama2-7B runs about 1.5x faster than the full model.
Learning to Discover at Test Time
Test-time scaling usually means taking a finished model and sampling from it many times, hoping one attempt lands. But for discovery problems like an open math question or a coding contest, only the single best solution counts. Instead of leaving the model frozen, TTT-Discover keeps training it on the one problem in front of it: it makes attempts, learns from the ones that scored well, and uses that to try again, all at test time. Built on the open-weight gpt-oss-120b with public code, it sets new state-of-the-art marks on Erdős' minimum overlap problem, past AtCoder contests, and single-cell denoising, and its kernels run up to 2x faster than prior art in a GPUMode competition.
DuetServe: Harmonizing Prefill and Decode for LLM Serving via Adaptive GPU Multiplexing
Serving an LLM is really two jobs. Prefill reads the whole prompt in one heavy burst; decode then writes the reply one token at a time, mostly waiting on memory. Run both on one GPU and they trip over each other, so the reply comes out in stutters; split them across separate GPUs and the stutter goes away, but now each GPU group holds a full copy of the model and the prompt’s KV cache has to cross the network between them. DuetServe moves that split inside a single GPU, running both jobs together until a lightweight predictor sees them about to collide, then carving up the GPU’s streaming multiprocessors on the fly so prefill stops trampling decode. On Qwen3-8B and Qwen3-14B across three realistic workloads, it lifts total throughput by up to 1.3x while keeping token latency low.
Real-Time Visual Attribution Streaming in Thinking Model
When a multimodal model reasons over an image for thousands of tokens, is it actually looking at the parts that support its answer? The trustworthy way to check is causal: mask a region, rerun the model, and see whether the answer changes, which is far too slow to run routinely. Attention maps are instant instead, but a model can stare at a region that never actually sways its answer. This paper trains a shortcut, a lightweight module that predicts those causal effects straight from attention features, and across five benchmarks and four thinking models it matches exhaustive causal methods on faithfulness while running in real time, so users watch the supporting regions light up as the model reasons rather than after.
Physical AI
Physical AI is a key vertical at Nebius, and in robotics the scarce resource is not compute but data: demonstrations, human feedback, and real-robot trials all cost far more than tokens. Our picks reflect that.
Rather than ever-bigger robot models, they target the machinery that makes robot learning practical: pretraining policies so reinforcement learning can actually improve them, learning rewards from ten human labels instead of thousands, and keeping generated plans within what a physical robot can execute.
The other thread is honest measurement: two of the picks are benchmarks that give young subfields, memory-augmented policies and code-writing manipulation agents, their first rigorous test. And every method here proves itself on real hardware, from tabletop arms to a Unitree quadruped, because in this field the demo that survives contact with a physical robot is the result that counts.
RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies
Robot policies keep gaining memory modules, but the field has no shared way to test whether they actually remember. Long-horizon manipulation means recalling how many times an action was repeated or where an occluded object went, yet memory-augmented vision-language-action models get evaluated in narrow, ad hoc setups. RoboMME standardizes it with 16 manipulation tasks spanning temporal, spatial, object, and procedural memory. Across 14 VLA variants the best reaches just 44.5% success, far below a symbolic-memory oracle at 84% and humans at 90.5%.
Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning
Most robot policies are built in two steps: imitate human demonstrations, then improve with reinforcement learning. This paper shows the first step can quietly sabotage the second, because a policy that imitates too faithfully locks onto one way of doing each task, and RL can only improve behaviors the policy is still willing to try. Posterior behavioral cloning (PostBC) fixes the pretraining target rather than the RL, staying precise where demonstrations are plentiful and keeping its options open where they are scarce. It trains with ordinary supervised learning and yields faster RL finetuning on the Robomimic and LIBERO benchmarks and a real WidowX arm, so most robot foundation models could get cheaper finetuning just from a better starting checkpoint.
CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation
Most robot manipulation trains vision-language-action models end-to-end on ever-larger datasets, but a quieter line skips the data hunger: let an LLM write code that composes perception and control primitives. Nobody had rigorously measured how well that works. CaP-X builds the testbed, pairing CaP-Gym, an interactive environment for program-driven control, with CaP-Bench, which puts 12 frontier models through hundreds of tasks. Performance leans heavily on human-crafted abstractions, but multi-turn feedback, visual differencing, and skill synthesis close the gap, giving a training-free agent (CaP-Agent0) that matches expert human code on 4 of 7 core tasks and an RL variant that lifts real cubes at 84% success.
Video-Based Optimal Transport for Feedback-Efficient Offline Preference-Based Reinforcement Learning
Preference-based RL replaces hand-tuned reward functions with human judgments, but every judgment costs money and offline methods usually need thousands of them. VOTP goes after that bill directly. From as few as ten labeled preference pairs, it uses optimal transport in the embedding space of video foundation models to pseudo-label large pools of unlabeled pairs. The learned rewards drive offline RL that beats state-of-the-art baselines on D4RL locomotion and Meta-World manipulation under tight budgets, holds up against visual distractors, and reaches 80% success on a real-robot lifting task from those ten labels.
Model-Based Diffusion Sampling for Predictive Control in Offline Decision Making
Diffusion planners dream up diverse, task-aligned trajectories from offline data, but a real robot often can’t execute them, because nothing forces the plans to respect system dynamics. MPDiffuser splits the work between two specialists: a diffusion planner proposes behaviors and a separate dynamics model nudges them toward feasibility, the two alternating denoising steps while a lightweight ranker picks the best candidate. Because the dynamics model trains on its own, it can absorb data the planner never saw. It averages an 85.1 normalized score on D4RL, ahead of prior diffusion planners, holds up on constrained DSRL tasks, and runs on a Unitree Go2 quadruped.
Physical AI is a key vertical at Nebius. ICLR paints robotics as shifting to generalist systems grounded in the real world, with policies trained on large, diverse datasets, fused with VLA models, and fine-tuned through interaction. The center of gravity is moving toward embodied foundation models that can plan, adapt, and reuse skills across tasks. Sim-to-real transfer, data efficiency, and safety are core constraints.
Healthcare & Life Sciences
Nebius just wrapped the second AI Discovery Awards in London, an initiative to back AI-native startups in the life sciences space, which this year drew 647 applications from five continents.
Generative models for proteins and molecules are everywhere, with flow matching and diffusion treating structures as objects to be designed rather than merely predicted, while synthesis planning turns chemistry into an optimization problem. Alongside them runs a strong neuroscience and clinical thread, from decoding large-scale neuronal recordings to Bayesian models built for the messy, longitudinal reality of patient data.
dnaHNet: A Scalable and Hierarchical Foundation Model for Genomic Sequence Learning
Genomic foundation models face a tokenization dilemma: fixed-vocabulary tokenizers split apart meaningful units like codons, while nucleotide-level models stay true to the biology but pay huge compute over long contexts. dnaHNet skips the trade-off. Built on the H-Net architecture, it works directly on raw nucleotides and learns where to segment them, recursively compressing DNA into latent tokens that mirror the genome’s nested structure. Trained on prokaryotic genomes from 10M to 1B parameters, it sets a new state of the art, beating architectures like StripedHyena2 with more than 3x inference speedup over Transformers, and with no supervision it still discovers biological structure and predicts protein variant fitness zero-shot.
From Feasible to Practical: Pareto-Optimal Synthesis Plannin
Most computer-aided synthesis planning calls retrosynthesis done the moment one feasible route appears, optimizing for convergence or shortest paths. That is not how chemists work, since a usable route has to balance cost, yield, toxicity, and sustainability at once. MORetro* reframes it as multi-objective search and returns a Pareto front of routes instead of a single winner, and because it builds on multi-objective A* search it comes with a guarantee: for a fixed single-step model, it recovers the true Pareto front under the paper’s assumptions. Across retrosynthesis benchmarks it surfaces diverse, high-quality routes that single-objective planners miss, closer to how real industrial decisions get made.
SleepLM: Natural-Language Intelligence for Human Sleep
Sleep is dense, continuous physiology, but the models that read it live in closed label spaces of fixed stages and events, so they can score a night without ever describing or answering questions about it, and even frontier models stumble on zero-shot sleep-stage and unseen-concept tasks. SleepLM is a family of sleep-language foundation models that bridge natural language and multimodal polysomnography, trained with a Reconstructive Contrastive Captioner objective fusing contrastive alignment, captioning, and signal reconstruction. Getting there meant building the first large-scale sleep-text dataset, over 100,000 hours from more than 10,000 people. Beyond beating state-of-the-art on zero- and few-shot learning and cross-modal retrieval, it can localize events from a text query and generalize zero-shot to unseen tasks.
Autoregressive Boltzmann Generators
Sampling molecular systems at thermodynamic equilibrium is one of the oldest hard problems in statistical physics, and Boltzmann generators tackle it by pairing a generative model with exact likelihoods and an importance-sampling correction. The catch is they have leaned on normalizing flows, which are either boxed in by invertibility constraints or stuck with expensive likelihoods. ArBG drops flows entirely: it is a diffeomorphism-free autoregressive model that works directly on atoms in native Cartesian coordinates, giving exact likelihoods with no Jacobian determinants or ODE integration, and it opens the door to inference-time interventions and LLM-style scaling. The gains show up most on larger peptides like the 10-residue Chignolin, where the authors' 132M-parameter Robin model cuts zero-shot Energy-W2 error on 8-residue systems by roughly 60% over the previous state of the art.
Time-Conditioned Foreseeing: An EHR-Specific Foundation Model for Irregular Dynamics and Calendrical Time
EHRs look like text, so the field has mostly treated them as text, borrowing NLP architectures wholesale and paying for it, since the same lab value on day 3 and day 14 can mean very different things. This work rebuilds pretraining around what makes EHRs distinct: it spends resolution on clinically significant value ranges rather than normal ones, encodes both absolute timestamps and relative intervals, and its Time-Conditioned Foreseeing objective extends next-token prediction to forecast events across several horizons, the way clinicians plan treatment. The result beats existing foundation models across eleven downstream tasks, up to a 54% AUPRC gain, while generating realistic, temporally consistent patient trajectories.
Sustainable AI
As a cloud provider building AI-native data centers, Nebius has a direct stake in making machine learning cheaper to run and lighter on the planet. Sustainable AI is still the smallest and youngest track at ICML 2026, but a telling one.
The work splits in two: AI applied to sustainability, with foundation models for climate, weather, and drought forecasting, and sustainability applied to AI, with papers accounting for the energy and carbon cost of training and serving large models. It is an early field, but one whose questions grow more urgent with every jump in scale, like this year’s near-doubling of submissions.
Towards Resource-Efficient LLMs: End-to-End Energy Accounting of Distillation Pipelines
Distillation is sold as the cheap path to an efficient LLM, but the accounting usually stops at the student. Teacher-side work like synthetic data generation, logit caching, and evaluation can rival or exceed student training and rarely shows up on the bill. This paper tracks GPU power stage by stage across whole distillation pipelines, comparing logit-based distillation and synthetic-data fine-tuning while distilling a 32B OLMo-2 teacher into 1B-13B students. At the 1B scale, distillation burns roughly 2.4x more end-to-end energy than plain fine-tuning for slight quality gains, unless the teacher’s artifacts get reused.
Hugging Carbon: Quantifying the Training Carbon Emissions of AI Models at Scale
AI’s carbon problem is widely acknowledged but rarely counted, since unlike manufacturing or agriculture the industry has no accounting method that scales past one model at a time. Hugging Carbon treats the Hugging Face platform as a public corpus for that, estimating aggregate training emissions from energy, compute, and model metadata, with a tiered scheme for incomplete disclosures. It also introduces ATCI, an emissions-per-compute metric for training efficiency. By its estimate, training the platform’s most popular open-source models, those with over 5,000 downloads, has already produced roughly 60,000 metric tons of carbon.
Learning General Causal Structures with Hidden Dynamic Process for Climate Analysis
Climate records are shaped by drivers that are not measured directly, like pressure and atmospheric processes. Causal representation learning recovers latent factors but ignores causal links among the observed variables, and causal discovery does the reverse. CaDRe does both at once: a time-series generative model with structural constraints that identifies the hidden dynamic process and the causal graph over observations together, with identifiability guarantees that hold even in nonparametric settings. On the CESM2, Weather, and ERSST datasets it beats time-series forecasting baselines and recovers causal graphs that line up with real wind circulation patterns.
How we picked the papers
We ranked all 6,352 accepted ICML 2026 papers within thematic categories. The final output is one CSV per category containing every paper ranked by a composite quality score. The methodology has three independent components: (1) LLM-based thematic classification to assign each paper to a category, (2) a reviewer-score-based composite to rank papers within each category, (3) domain experts selecting the final papers from the top of each algorithmically ranked list.
Each paper was classified into exactly one of 5 focused categories — Agentic AI, AI Systems, Physical AI, Health Care & Life Sciences, or Sustainable AI — using Qwen3.5-397B-A17B from TokenFactory with structured JSON output at temperature 0.
The model received a system prompt defining each category with specific scope boundaries. Classification rules instructed the model to prioritize the paper’s main contribution over secondary applications.
Papers with"datasets and benchmark” as primary areas were overridden into a separate category and excluded, since their contribution type is qualitatively different from research papers.
Papers within each category were ranked by local_top_overall, a weighted linear combination of five reviewer-derived signals standardized to zero mean and unit variance within the category (not globally). Category-local z-scoring ensures that a paper is measured against its thematic peers — a top Physical AI paper surfaces even if the subfield’s raw scores run lower than, say, Sustainable AI.
The formula is:
local_top_overall = 0.40 × z (contribution_mean) + 0.35 × z (rating_mean) + 0.10 × z (confidence_mean) + 0.10 × z (soundness_mean) − 0.05 × z (rating_std)
where all z-scores are computed over the papers in that category only.
Weight rationale. Contribution (perceived novelty and significance) carries the highest weight (40%), reflecting the goal of surfacing impactful work rather than merely well-executed work. Mean reviewer rating is second (35%) as a general quality signal. Soundness and reviewer confidence serve as minor quality gates (10% each). Rating standard deviation applies a small penalty for disagreement (5%) — kept low because within a focused thematic category, some spread in opinions is natural. The reviewer scores used here were taken from the public ICLR 2026 review data on OpenReview.
The ranking did the heavy lifting, our experts shaped the final cut. After the composite ranking surfaced the top-30 candidates in each category, domain experts at Nebius went through these papers and made the final selection. Our list reflects two signals: quantitative ranking as a starting point, expert judgment informed by Nebius focus areas as the deciding vote.
