Optimize by Benchmarking Variants
Your AI runs repeated tests on code variants to find the fastest or most efficient implementation. It measures latency, throughput, or cost under constraints, promoting only correct and reproducible improvements.
affaan-m on GitHub
Curated by PromptsEdge from a public repo · MIT license. All credit goes to the author.
What is this skill?
Use this skill to convert "make it 20x faster" or "try 50 recursive optimizations" into a bounded measured loop that can actually improve a system.
Required Baseline
Do not optimize until these exist:
- the operation being optimized;
- the correctness gate that must stay green;
- the metric: wall time, p95 latency, rows/sec, cost/run, memory, error rate;
- the current baseline;
- the search budget: max variants, max time, max spend, max data impact.
If the user asks for an unrealistic target, keep the ambition but make the loop bounded and measurable.
Loop
- Measure the baseline.
- Identify bottlenecks from evidence.
- Generate variants that test one hypothesis each.
- Run variants with the same input shape.
- Reject variants that fail correctness, safety, or reproducibility.
- Promote the fastest safe variant.
- Codify the winning path in a script, command, test, config, or doc.
- Rerun the baseline and winner to confirm the delta.
Variant Table
Track variants like this:
Variant | Hypothesis | Command | Time | Correct? | Notes
baseline | current path | npm run job | 120s | yes | stable
batch-500 | fewer round trips | npm run job -- --batch 500 | 42s | yes | winner
parallel-8 | more workers | npm run job -- --workers 8 | 31s | no | rate limited
Recursive Search
For recursive or hyperparameter work:
- persist every run to a ledger;
- compare against the prior accepted winner, not only the previous run;
- keep a holdout or replay check;
- stop when improvement is within noise, correctness fails, cost exceeds the budget, or the search starts changing more variables than it can explain.
Use phrases like "best measured safe variant" instead of "global optimum" unless the search space was actually exhaustive.
Promotion Gate
A variant cannot become the new default until:
- correctness tests pass;
- the performance delta is repeated or explained;
- rollback is obvious;
- the change is encoded in source control or a durable runbook;
- the final summary includes exact commands and measurements.
Install this skill
- 1
Get the skill — it’s free
Use the Get this skill panel. Unlocked skills stay in My skills. - 2
Download or clone the files
Download the zip, or clone the repo and copy theskills/benchmark-optimization-loopfolder. - 3
Put it where your agent looks for skills
For Claude Code, use your personal skills folder (every project) or a project’s own folder:~/.claude/skills/benchmark-optimization-loop/SKILL.md # all projects .claude/skills/benchmark-optimization-loop/SKILL.md # this project only
- 4
Just ask
No command needed. The agent reads the skill’s description and loads it on its own when your request matches.
SKILL.md frontmatter
What your agent reads to decide when to load this skill.
--- name: benchmark-optimization-loop description: Use when the user asks to make something faster, try many variants, run recursive optimization, benchmark latency/throughput/cost, or choose the best implementation by repeated measured tests. ---
Files
Open any Markdown file to read it here.
Related skills
View all →- Windows Desktop E2E TestingFreeYour AI writes and runs end-to-end tests for Windows native desktop apps using pywinauto and Windows UI Automation. It supports WPF, WinForms, Win32/MFC, and Qt apps, including CI/CD integration and testability improvements.
- Vue 3 and Nuxt PatternsFreeGet guidance on Vue 3 Composition API, Pinia state management, Vue Router, and Nuxt SSR patterns. It covers component design, reactivity, routing, testing, and project structure best practices.
- Configure and Optimize ViteFreeYour AI configures Vite projects including plugins, environment variables, proxy setup, and build optimization. It handles dev server, SSR, library mode, and troubleshooting common Vite issues.
- Convert UI to Vue 3FreeTurn batches of UI screenshots or design exports into Vue 3 components. It organizes pages and modules, maps elements to Vant, Element Plus, or Ant Design Vue, and wires up routing automatically.