Question：Why use torch.cuda.Event instead of torch.profiler for performance measurement?

[study-never-alone]

Hi~ I've noticed that using torch.cuda.Event for measuring kernel performance can lead to significant timing variance across multiple runs, affecting stability. Given that torch.profiler offers more stable and robust measurements of pure kernel execution time, was there a specific reason for choosing cuda.Event in the benchmark's design?

[yuxuan-z19]

Agree. Given that tools like torch.profiler, nsys, or ncu provide more stable and precise kernel-level timing — as also adopted in sakana.ai's leaderboard — I'm wondering if there's a specific reason behind choosing cuda.Event here?
Switching to system-level profilers could make the benchmarking results more robust and reliable. Would love to hear your thoughts!

[wqd008]

I also have this question. But there was a blog (https://www.speechmatics.com/company/articles-and-news/timing-operations-in-pytorch) shows the usage of torch.cuda.event for accurate measurement. I wonder which one should I use: torch.cuda.event or torch.profiler ? And what is the difference?

[yuxuan-z19]

For CUDA kernel profiling, you might find this implementation more practical and professional: robust_kbench’s cuda_profile.py