Added Benchmark Evaluation Framework with CORE Benchmark Suite#402

Closed

Jasmine-Yuting-Zhang wants to merge 17 commits intomainfrom

Commits on Oct 27, 2025

Added NanoChat evaluation modules under benchmarks/language_models.
Jasmine-Yuting-Zhang
committed
Added evaluation script for running NanoChat benchmark on HuggingFace models.
Jasmine-Yuting-Zhang
committed
Added missing explanation of --max_per_task default value.
Jasmine-Yuting-Zhang
committed

Commits on Oct 28, 2025

Moved NanoChat benchmark from benchmarks/language_models to plato/benchmarks/language_model.
Jasmine-Yuting-Zhang
committed
Cleaned up unused code from nanochat.
Jasmine-Yuting-Zhang
committed
Added abstract eval_model() to TestingStrategy.
Jasmine-Yuting-Zhang
committed
Added eval_model() to DefaultTestingStrategy.
Jasmine-Yuting-Zhang
committed
Added benchmark result save/load utilities.
Jasmine-Yuting-Zhang
committed
Implemented benchmark evaluation pipeline with multiprocessing.
Jasmine-Yuting-Zhang
committed
Added registry for benchmark.
Jasmine-Yuting-Zhang
committed
Added base class for evaluating trained models.
Jasmine-Yuting-Zhang
committed
Added CORE benchmark implementation for language models.
Jasmine-Yuting-Zhang
committed
Added helper functions for CORE benchmark implementation.
Jasmine-Yuting-Zhang
committed
Added benchmark evaluation support in fedavg.py.
Jasmine-Yuting-Zhang
committed
Added benchmark configuration support in config.py.
Jasmine-Yuting-Zhang
committed
Added support for split learning benchmark evaluation.
Jasmine-Yuting-Zhang
committed
Reformatted code using Ruff.
Jasmine-Yuting-Zhang
committed