Repository navigation
Show trace view aria snapshot in browser test failure terminal output #11616
Description
- follow-up to feat(browser): show trace steps timeline on terminal on test failure #11594
- follow-up to feat(browser): capture aria snapshot in trace view #11492
- related Trace View: Failure-Only Trace Recording #10310 (
'failure-only'trace mode) - related Test-end browser trace snapshots are taken after
afterEachcleanup #11615 (test-end snapshot taken afterafterEachcleanup)
Clear and concise description of the problem
#11592 prints the trace timeline under a failing browser test's error, which shows the steps that led to the failure. It doesn't show what was on the page when the test failed. For that, there is the failure screenshot, but in CI the log only shows its path, and a coding agent has to interpret pixels to read it.
#11492 records an aria snapshot of the page with each trace entry. It is a text rendering of the accessibility tree, which is compact, searchable, and easy to compare between runs, but it is only visible in the trace viewer.
One kind of failure already prints this. When a locator can't find an element, the Cannot find element with locator error includes an aria tree of the container, depending on browser.locators.errorFormat. Other failures, such as an expect.element assertion that times out on an element that exists, or a plain expect, show no page state at all.
Suggested solution
When a test with trace data fails, print the aria snapshot from the end of the test under the trace timeline:
Trace:
step locator start duration location
click getByRole('button') +32ms 78ms test/basic.test.ts:14:36
toMatchTextContent [ERROR] getByRole('button') +117ms 3.00s test/basic.test.ts:18:20
Page at test end:
- button "Click Count: 2"
- The snapshot comes from the entry that records the test's end (
vitest:onAfterRetryTask). With Trace View: Failure-Only Trace Recording #10310's'failure-only'mode, that entry holds the only snapshot of a failed test, so the same rule works in both modes. - It is skipped when the error already contains an aria tree, as element-not-found errors do unless
browser.locators.errorFormatis'html'. The browser marks the error with a flag wherever it adds a tree, and the reporter checks the flag. A check onerror.namewould only recognize today's not-found error, while the flag also covers any error that includes a tree later. - The tree is printed in full, like the aria tree in element-not-found errors. Only their HTML part is capped (
DEBUG_PRINT_LIMIT, 7000 characters by default). - There is no new option. It prints whenever the test-end entry has an aria snapshot, so
browser.traceView.ariaSnapshot: falseturns it off.
Today the test-end snapshot is taken after afterEach cleanup, so a test that cleans up in afterEach would print an empty page. That is a separate bug (#11615), which affects the trace viewer's last step the same way. vitest-browser-react and vitest-browser-vue clean up in beforeEach, so it doesn't apply to them.
Additional context
- A separate follow-up can include the aria tree in
expect.elementand action errors, the same way element-not-found errors include it (Trace View: Failure-Only Trace Recording #10310 describes this as extendingbrowser.locators.errorFormatto all locator errors). That works without trace view and captures the page at the moment of failure. It is independent of this proposal: once those errors carry a tree, the flag makes the timeline skip its snapshot for them, and plainexpectfailures and thrown errors still get the test-end snapshot. - Another follow-up can write a failure context file next to the failure screenshot and print its path, similar to Playwright's
error-context.md. It would hold what is too big for the terminal: the whole timeline without the step cap, and each step's aria snapshot as a history of the page. In'failure-only'mode steps carry no snapshots, so the file would have the full timeline and the test-end tree. The file name should include the attempt, so retries don't overwrite each other the way failure screenshots do (Failure screenshot of a retried test always shows the last attempt #11614). - When a test fails on every retry, the timeline currently prints the last attempt's trace under each attempt's error (Failure screenshot of a retried test always shows the last attempt #11614). The snapshot would repeat the same way and should follow the same fix.
- If whole-page trees turn out too long in practice, one limit should cover both this output and the aria tree in element-not-found errors.
Activity
[-]Show the page's aria snapshot in browser test failure output[/-][+]Show the page's aria snapshot in browser test failure terminal output[/+][-]Show the page's aria snapshot in browser test failure terminal output[/-][+]Show the page's trace aria snapshot in browser test failure terminal output[/+][-]Show the page's trace aria snapshot in browser test failure terminal output[/-][+]Show trace view aria snapshot in browser test failure terminal output[/+]