ci(benchmarks): Add benchmark CI isolation flags and document runner adoption gate#42164
ci(benchmarks): Add benchmark CI isolation flags and document runner adoption gate#42164
Conversation
Agent-Logs-Url: https://github.com/MetaMask/metamask-extension/sessions/f6962b91-3f48-45ea-9ca8-3d3859439aad Co-authored-by: MajorLift <34228073+MajorLift@users.noreply.github.com>
Builds ready [10839f0] [reused from 3c0306f]
⚡ Performance Benchmarks (Total: 🟢 6 pass · 🟡 9 warn · 🔴 0 fail)
Bundle size diffs
|
|
CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes. |
9ed3753 to
4e25deb
Compare
Builds ready [4e25deb]
⚡ Performance Benchmarks (Total: 🟢 7 pass · 🟡 8 warn · 🔴 0 fail)
Bundle size diffs
|
Builds ready [10d4770]
⚡ Performance Benchmarks (Total: 🟢 6 pass · 🟡 8 warn · 🔴 0 fail)
Bundle size diffs [🚨 Warning! Bundle size has increased!]
|
…use \`Awaited<typeof import(...)>\`
Builds ready [201352a]
⚡ Performance Benchmarks (Total: 🟢 7 pass · 🟡 8 warn · 🔴 0 fail)
Bundle size diffs [🚨 Warning! Bundle size has increased!]
|
|
Builds ready [7b99e7e]
⚡ Performance Benchmarks (Total: 🟢 7 pass · 🟡 8 warn · 🔴 0 fail)
Bundle size diffs [🚨 Warning! Bundle size has increased!]
|



Description
Power-user startup benchmarks remain noisier than standard startup on the same code path after addressing heap debt (#42163), render-completion races (#42162), and network mock gaps (#42161). The remaining variance sources are GPU/compositor scheduling and Chrome background-throttling behavior, both of which are controllable via launch flags at low risk.
This PR applies the browser isolation controls scoped to CI/Codespaces environments and explicitly defers any larger-runner rollout behind a measurable gate.
Chrome benchmark isolation (
test/e2e/webdriver/chrome.js)Adds the following flags when
CI || CODESPACES, leaving local interactive runs unchanged:--disable-gpu— removes GPU scheduling jitter from compositor timing--disable-renderer-backgrounding— prevents the renderer process from being throttled when the window is not foregrounded--disable-backgrounding-occluded-windows— prevents throttling due to window occlusion--disable-background-timer-throttling— preventssetTimeout/setIntervalfrom being clamped to 1 Hz in background contextsTargeted regression coverage (
test/e2e/benchmarks/chrome-driver.test.ts)Asserts that all four isolation flags are present in
ChromeDriver.buildoutput whenCI=trueand absent whenCIis unset.Runner adoption decision documented, not prematurely adopted (
.github/workflows/run-benchmarks.yml)benchmarksandbenchmarks-page-loadremain onubuntu-latest. Inline workflow comments record the adoption gate: a 15-runstartupPowerUserHometrial must demonstrate ≥2pp CV improvement overubuntu-latestbefore switching to an 8-core runner. GitHub Actions cost delta and current image/Chrome pin strategy are documented indocs/quality-gates/benchmark-ci-constraints.md.Changelog
CHANGELOG entry: null
Related issues
Fixes:
Manual testing steps
yarn jest test/e2e/benchmarks/chrome-driver.test.tsScreenshots/Recordings
Before
N/A — CI-only change.
After
N/A — CI-only change.
Pre-merge author checklist
Pre-merge reviewer checklist