Skip to content

Conversation

@EssamWisam
Copy link

@EssamWisam EssamWisam commented Dec 20, 2025

We have identified that a number of KernelBench problems:

  • Can be solved by a kernel returning a constant output (constant fill)
  • Can be solved by a kernel eliminating one of the operations completely

This results in astronomical speedups for some of these problems which does not reflect the agent's ability to perform genuine optimizations and rather reflects their ability to exploit flaws in the given program. Correspondingly, one agent performing genuine optimizations while remaining logically equivalent to the program code would likely underperform an agent that rather focuses on exploiting these flaws. Not to mention that whether the agent exploits the flaw could depend on "luck".

In proposing fixes to each of the problems we tried to look for the most minimal change that would fix the problem. All the fixes to the redundant operations flaw are non-breaking (correctness-wise).

The PR includes:

The changelog includes a list of todos which are problem renames. Delayed applying these until we approve the fixes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant