Accessibility inspect returns far-offscreen elements, blowing the budget on long pages #178
Description
What happened
Inspecting a Safari window showing Proton VPN's WireGuard config page returned
~25k tokens and was truncated ("truncated": true, "omitted": 964), dropping
elements the agent needed. The page has a 240-country server list below the
form.
The output includes a large number of elements that are nowhere near the
visible window (window is 1298x953):
{"id":"elem_52", "label":"Downloads", "bounds":[337,-739,949,51]}
{"id":"elem_60", "label":"Download Android", "bounds":[384,-379,85,30]}
{"id":"elem_88", "label":"form", "bounds":[337,96,949,7117]}
{"id":"elem_520","label":"Eritrea Eritrea 1 city", "bounds":[337,2733,949,41]}
Elements at y=-739 and y=2733 on a 953pt-tall window can't be acted on
without scrolling first, but they consume the element budget ahead of elements
that are actually visible.
scope doesn't help here
The documented workaround is to scope to a container, but the only container
that holds the form (elem_88, the form group) also contains the whole
country list — its bounds are 7117pt tall. Scoping returned the same
truncation.
Suggestion
Add a way to limit inspection to what's actionable:
- a
visible_onlyflag that drops elements whose bounds don't intersect the
window's visible rect, or - a
max_depth, or - spend the element budget on intersecting elements first, so truncation drops
offscreen content rather than visible content.
For an agent driving a page, offscreen elements are close to useless: any
action on them is refused until scrolled into view anyway.
Workaround used
mode: pixels + point clicks, which is cheap but gives up element targeting
and can't do text insertion.
Environment
Desktop Tools 0.16.1, macOS 27, Safari 27
Activity
native2: inspect spends its budget on what's on screen first (#178)
native2: inspect spends its budget on what's on screen first (#178)
native2: inspect spends its budget on what's on screen first (#178)
native2: inspect spends its budget on what's on screen first (#178)
native2: inspect spends its budget on what's on screen first (#178)
native2: inspect spends its budget on what's on screen first (#178)
native2: inspect spends its budget on what's on screen first (#178)