Map agent intent to desktop coordinates.

Compare normalized vision bounds (0–1000) against Cocoa logical points, Wayland fractional surfaces, and OS accessibility trees without dispatch surprises.

Inspect laboratory
AGENT SPACE [0..1000] Norm(630, 380) SCALE x2 LOGICAL DISPLAY 806.4 × 304.0 pt Physical: 1612 × 608 px ACCESSIBILITY BOUNDING RECT CHECK AXUIElement(Button) AT-SPI Role: push VERIFIED Deterministic transform confirms target rect contains hit coordinate before synthetic event dispatch.
01 / COORDINATE LABORATORY

Transform normalized inputs to hardware pixels.

Agent vision models emit point coordinates in normalized units [0..1000]. Execute the multi-stage pipeline below to calculate precise OS dispatch targets.

Normalized Space [0..1000] Cocoa AXUIElement Wayland Fractional Scaling AT-SPI Tree Inspection Retina Scale Factor 2.0x Compositor Hit Test

Grounding Stepper

COMPUTED DISPATCH TARGET

Hardware Coordinate Mapping

806.4 × 304.0 pt
1612 × 608 physical pixels on macOS (2.0× Retina). Grounding source: Hybrid verification.
Accessibility API
AXUIElement
Compositor Unit
Cocoa Point
Status
Calculated

Interactive Target Pad

Norm(630, 380)
02 / OS SPECIFICATION & SEMANTICS

Synthesize intent across platform layers.

A coordinate without OS semantic backing leads to failed clicks on detached surfaces, popup layers, or unscaled window viewports.

Visual grounding maps SCREENSHOT_PX into LOGICAL_PTS with ACCESSIBILITY_TREE validation.

Multi-modal agent runtimes must not blindly inject CGEvents or XTest events into raw screen coordinates. Every model output must pass geometry checks against the active desktop window hierarchy.

INTERACTIVE CAUSALITY

Transform pipeline verification

Capture frame at scale, normalize bounding box, query accessibility node, then dispatch event.
[1] RAW_FRAME: 2560 × 1600 px
[2] AGENT_POINT: (630, 380) / 1000
[3] DISPATCH_PT: (806.4, 304.0)
[4] AX_MATCH: AXButton("Submit") OK
Zero synthetic misclicks recorded when coordinate projection satisfies both accessibility frame bounding checks and compositor focus states.

On macOS, Quartz window server coordinates operate in logical points where top-left is (0,0). High-DPI Retina displays apply a backing store scale factor (typically 2.0x). Synthetic input via CGEventCreateMouseEvent uses logical points, while accessibility queries via AXUIElementCopyAttributeValue report element frame origins in screen coordinates.

Wayland intentionally restricts global coordinate discovery and synthetic event injection across arbitrary client surfaces. Automation agents must interface with Compositor protocols (such as wlr-virtual-pointer or XDG Desktop Portals) alongside AT-SPI D-Bus trees to discover fractional scale surfaces.

Under X11, global coordinates are directly addressable via the root window, and synthetic events are dispatched through XTestFakeMotionEvent. However, multi-monitor xrandr configurations with mixed DPI scaling require manual coordinate offsetting to prevent pointer clipping.

Export mapping specifications for agent pipelines.

Download current transform parameters as a structured plain-text specification ready for integration with OS agent test runners.

Configure custom resolution