sharkdp/hyperfine

Add an option to save a benchmark run for later comparisons

Open

#607 opened on Jan 28, 2023

View on GitHub
 (6 comments) (11 reactions) (0 assignees)Rust (479 forks)batch import
feature-requesthelp wanted

Repository metrics

Stars
 (28,110 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

There are times when I want to have some baseline saved that I compare later changes to. It would be convenient if I had a way to save a benchmarking run and then load that to later runs. Something like

$ hyperfine --save-benchmark baseline 'sleep 1'
$ hyperfine --load-benchmark baseline 'sleep 0.5'
{{ display things as though `hyperfine 'sleep 1' 'sleep 0.5'` had been run }}

If you're interested in supporting this then I can work on adding it as a feature

Contributor guide

Research direction
Explore the existing hyperfine codebase to understand how runs are currently stored in memory and summary statistics are generated. Design a serialization format (e.g., JSON) to save benchmark results to a file and load them back. Implement new CLI flags save benchmark and load benchmark, and write tests.
Tech stack
rust
Domain
cli
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
RustGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
75