MCP server

#Capture

Captures are saved into the Agent folder in the format you pick and come back with path, width and height in pixels, scale, and an inline preview (with preview_width and preview_height) unless include_image is false. Whether Sightline's own windows, such as pins, appear in screen and area captures follows the Hide Sightline in captures and recordings setting.

#capture_screen

Takes a screenshot of a whole display, optionally with marks drawn on it.

The second display as a lossy WebP, without a preview, to keep the agent's context small:

{
  "tool": "capture_screen",
  "arguments": {
    "display": 1, "format": "webp", "name": "second-display",
    "include_image": false
  }
}

Returns path, width, height, scale and a preview.

#capture_area

Takes a screenshot of a rectangle given in points.

The top left 1280 by 800 points, with a numbered callout drawn in at once. Annotation positions are pixels, so on a Retina display x 400 is 200 points in:

{
  "tool": "capture_area",
  "arguments": {
    "x": 0,
    "y": 25,
    "width": 1280,
    "height": 800,
    "name": "menu-bar-area",
    "annotations": [
      {
        "type": "number", "x": 400, "y": 120, "label": "Click here first",
        "number_size": "large"
      }
    ]
  }
}

Returns path, width, height, scale and a preview.

#capture_window

Takes a screenshot of one window, picked by app name, title or id, without its shadow.

The Safari window whose title contains Checkout, with the failing field boxed and a note:

{
  "tool": "capture_window",
  "arguments": {
    "app": "Safari",
    "title": "Checkout",
    "name": "checkout-error",
    "annotations": [
      {
        "type": "rectangle", "x": 312, "y": 540, "width": 620,
        "height": 88, "color": "#FF453A", "stroke_width": 6
      },
      {
        "type": "text", "x": 960, "y": 560,
        "text": "Card number rejected\nbut format is valid", "size": 30,
        "text_background": "solid"
      }
    ]
  }
}

A window by id, as list_windows returned it:

{"tool": "capture_window", "arguments": {"id": 4211, "format": "jpeg"}}

Returns path, width, height, scale and a preview.

#capture_scrolling

Scrolls a window, an area or a region inside a window and stitches the frames into one tall screenshot.

The Mac App Store version doesn't list this tool, because the sandbox doesn't let Sightline scroll other apps; called by name from the CLI it returns the App Store build cannot scroll windows for you; .... Capture the visible part with capture_window, or ask the user to take a scrolling capture with ⇧⌘6 and find it with search_captures. Builds outside the store also need Accessibility permission.

Scroll only the message list of Mail, found in a capture_window image first:

{
  "tool": "capture_scrolling",
  "arguments": {
    "window": {"app": "Mail"},
    "region": {"x": 0, "y": 104, "width": 760, "height": 1300},
    "name": "inbox-full"
  }
}

Returns path, width, height, scale and a preview.

#list_windows

Lists the windows on screen, front to back.

No parameters.

{"tool": "list_windows", "arguments": {}}

Returns An array of windows, each with id, app, title and the frame x, y, width, height in points.

#list_displays

Lists the connected displays.

No parameters.

{"tool": "list_displays", "arguments": {}}

Returns An array of displays, each with index (what display takes), name, the frame x, y, width, height in points, scale and is_main.

#Read the screen

#capture_text

Reads the text in an area, a window, a display or a saved image with on-device text recognition, and decodes QR codes and barcodes.

Read a terminal window:

{"tool": "capture_text", "arguments": {"window": {"app": "Terminal"}}}

Read an image the agent captured earlier, for example to quote an error message exactly:

{
  "tool": "capture_text",
  "arguments": {"path": "/Users/you/Pictures/Screenshots/Agent/checkout-error.png"}
}

Returns text, codes (each with payload, symbology and a pixel box), the image width and height, and scale for a capture. hidden_lines counts lines left out as credentials.

#find_text

Finds where a piece of text is and returns its pixel boxes, ready for a mark, a blur or a scrolling region.

Where does Total appear in the open invoice screenshot:

{"tool": "find_text", "arguments": {"text": "Total", "editor": "front"}}

Find a token in a saved capture before blurring it:

{
  "tool": "find_text",
  "arguments": {
    "text": "sk_live",
    "path": "/Users/you/Pictures/Screenshots/Agent/env-vars.png"
  }
}

Returns matches, each with text (the whole line) and x, y, width, height; codes whose payload contains the text; the image width and height.

#pick_color

Reads the color of one point on screen.

The color at 612, 344 points on the second display:

{"tool": "pick_color", "arguments": {"x": 612, "y": 344, "display": 1}}

Returns hex (#RRGGBB in sRGB), red, green, blue from 0 to 255, and the x and y read, rounded down to whole points.

#compare_with_design

Compares a design image with a window, an area or a display, the same check as Show Differences on a pin.

The running iOS Simulator against the exported design:

{
  "tool": "compare_with_design",
  "arguments": {
    "design": "/Users/you/Pictures/Screenshots/Agent/signup-design.png",
    "window": {"app": "Simulator"}
  }
}

Strict, without lining up, for a pixel exact check:

{
  "tool": "compare_with_design",
  "arguments": {
    "design": "/Users/you/Pictures/Screenshots/Agent/signup-design.png",
    "window": {"app": "Simulator"},
    "align": false,
    "include_image": false
  }
}

Returns A marked capture with path, width, height and scale; shift with dx and dy; and up to 20 differences, each with number, a pixel box and hint: shifted (with dx, dy), only_in_design, only_on_screen or looks_different.

#Edit and annotate

annotate changes a file and returns it. open_in_editor puts the same changes into the Sightline editor, where the user can adjust them before saving. Both, and beautify, take editor instead of a path to change a window the user already has open. remove_background writes a cutout of a screenshot to a new file.

#annotate

Draws marks onto a screenshot or recording, and crops, rotates, flips, resizes, styles or exports it. Recording options are under Editing a recording.

If the file is open in a Sightline editor, also one the user opened, the change lands there live as one undo step. A capture the user saved with its edit is changed on its unedited original, with the user's marks kept.

Explain a bug on a capture: an arrow, a boxed note in the handwriting font, a pixelated email address, and a crop to the interesting part, written to a new file:

{
  "tool": "annotate",
  "arguments": {
    "path": "/Users/you/Pictures/Screenshots/Agent/checkout-error.png",
    "name": "checkout-error-marked",
    "annotations": [
      {
        "type": "arrow", "from": [1180, 300], "to": [940, 560],
        "stroke_width": 6, "shadow": true
      },
      {
        "type": "rectangle", "x": 1190, "y": 180, "width": 520,
        "height": 130, "fill": true, "color": "#FFD60A",
        "text": "Rejects valid Visa numbers", "font": "handwriting",
        "size": 34
      },
      {
        "type": "blur", "x": 120, "y": 96, "width": 380, "height": 44,
        "blur_style": "pixelate"
      }
    ],
    "crop": {"x": 80, "y": 60, "width": 1700, "height": 760}
  }
}

A step by step guide: three numbered badges with labels and a spotlight on the first step, then a small JPEG for a chat:

{
  "tool": "annotate",
  "arguments": {
    "path": "/Users/you/Pictures/Screenshots/Agent/settings-window.png",
    "annotations": [
      {
        "type": "spotlight", "x": 40, "y": 220, "width": 560,
        "height": 120, "dim": 0.6
      },
      {"type": "number", "x": 70, "y": 280, "label": "Open General"},
      {"type": "number", "x": 70, "y": 520, "label": "Turn on Sync"},
      {"type": "number", "x": 70, "y": 760, "label": "Click Done"}
    ],
    "resize": {"width": 1200},
    "export": {"format": "jpeg", "quality": "medium"}
  }
}

Circle something loosely and magnify a tiny label:

{
  "tool": "annotate",
  "arguments": {
    "path": "/Users/you/Pictures/Screenshots/Agent/toolbar.png",
    "annotations": [
      {
        "type": "freehand",
        "points": [
          [400, 92],
          [470, 70],
          [560, 78],
          [600, 110],
          [560, 142],
          [460, 146],
          [404, 120],
          [410, 96]
        ],
        "color": "#30D158",
        "stroke_width": 5
      },
      {
        "type": "magnify", "x": 760, "y": 40, "width": 220, "height": 220,
        "zoom": 3
      }
    ]
  }
}

Returns path, width and height of the result and a preview. By default a screenshot is updated in place and the same path comes back.

#Annotation object

Each entry of annotations is one mark. Every field except type is optional.

#open_in_editor

Opens a screenshot or recording in the Sightline editor, brings Sightline to the front and adds marks, a look, cuts or zooms the user can still change.

If the file is already open, no second window opens: the open window comes to the front and the change lands there as one undo step. With paths, several screenshots open together in one window, or with editor are added to that window. To save every picture of a window as files, use export_pictures.

Hand a capture to the user with suggestions they can move or delete:

{
  "tool": "open_in_editor",
  "arguments": {
    "path": "/Users/you/Pictures/Screenshots/Agent/onboarding.png",
    "annotations": [
      {
        "type": "highlight", "x": 210, "y": 404, "width": 520,
        "height": 48
      },
      {
        "type": "text", "x": 760, "y": 410, "text": "Shorter title?",
        "color": "#0A84FF", "size": 28
      }
    ],
    "beautify": {"look": "soft", "padding": 48}
  }
}

Open what the user just copied:

{"tool": "open_in_editor", "arguments": {"from_clipboard": true}}

Returns The path that was opened, or from_clipboard, width and height. For a capture saved with its edit, restored_edit: true plus the restored crop size, resize, and for a recording its duration, pieces, sound, zooms, crop, pointer, keys and camera in the same form as the arguments. With paths: paths and editors, one editor id per picture in the same order. With all_pictures: editors in column order and resized, those that changed.

Add two screenshots to the window the user has open, then halve every picture in it:

{
  "tool": "open_in_editor",
  "arguments": {
    "editor": "front",
    "paths": [
      "/Users/you/Pictures/Screenshots/Agent/step-2.png",
      "/Users/you/Pictures/Screenshots/Agent/step-3.png"
    ]
  }
}

{
  "tool": "open_in_editor",
  "arguments": {"editor": "front", "all_pictures": true, "resize": {"scale": 0.5}}
}

#remove_background

Lifts the main subject of a screenshot off its background with on-device Vision, like Remove Background on a thumbnail, a pin or in History, and saves it as a new PNG with a transparent background. Needs macOS 27.

The source stays as it is, and the cutout becomes this agent's last screenshot, so the next tool can leave out path. It works best on a picture with a clear subject, such as a person, an object, an icon or a window on a desktop; without one it returns no subject found to lift off the background. To show the result to the user, pass it to open_in_editor.

Cut the app icon out of a capture and put it on a soft background:

{
  "tool": "remove_background",
  "arguments": {
    "path": "/Users/you/Pictures/Screenshots/Agent/app-icon.png",
    "name": "app-icon-cutout"
  }
}

{"tool": "beautify", "arguments": {"look": "soft", "name": "app-icon-card"}}

Returns path, width, height, scale and a preview, like a capture.

#Recordings

Recording tools need macOS 15. Recordings are MP4 in the codec of Sightline's Recording settings. A recording you made is never overwritten; annotate writes its result to a new file in the Agent folder.

#record_screen

Records a window, an area or a display for up to 60 seconds and notes where the pointer moves and clicks. Recording starts right away, so ask the user to get ready first.

A drawn pointer is not in the file itself, so pass the result through annotate before handing the file on. The microphone and the camera are never recorded, whatever the user's settings, so an agent never captures anyone's voice or face, and keys are never noted for agent recordings. The agent recording's pointer track goes with the Agent folder clean-up.

Record the Simulator for 20 seconds with its sound, the pointer left out for later, at 1080p:

{
  "tool": "record_screen",
  "arguments": {
    "window": {"app": "Simulator"},
    "seconds": 20,
    "include_audio": true,
    "pointer": "drawn",
    "size": "1080p"
  }
}

A small 720p clip of one screen area for a quick bug report:

{
  "tool": "record_screen",
  "arguments": {
    "area": {"x": 200, "y": 120, "width": 1280, "height": 800},
    "seconds": 8,
    "size": "720p",
    "quality": "small"
  }
}

Returns path, duration, width and height of the file, scale, and the pointer used (drawn falls back to video when the track can't be noted); pointer_note when the track couldn't be saved.

#last_recording

Returns the newest recording, for requests like "I just recorded this, cut the start".

{"tool": "last_recording", "arguments": {}}

Only what this agent recorded:

{"tool": "last_recording", "arguments": {"source": "mine"}}

Returns path, made_by (user or agent), duration, width, height and modified (ISO 8601).

#inspect_recording

Looks at a recording without changing it: when it is still or moving, what text frames show, where the user clicked, which keys they pressed and how their camera shows.

Find the loading stretch in the content area, read the text at three moments and look at one frame:

{
  "tool": "inspect_recording",
  "arguments": {
    "path": "/Users/you/Pictures/Screenshots/Agent/signup-flow.mp4",
    "region": {"x": 0, "y": 120, "width": 1920, "height": 900},
    "text_at_seconds": [1.5, 6, 11],
    "frame_at_second": 6
  }
}

Returns duration, width and height (after the crop), source_width and source_height (the uncropped video), crop (x, y, width, height of the crop in the user's saved edit, or null), frame_rate and segments (each start, end, moving); text with the lines and pixel boxes per second; has_pointer_track and up to 200 clicks (second, x, y, button, count, released_second, plus released_x, released_y for a drag); has_key_track and up to 200 keys (shortcuts such as ⇧⌘5 and typed words, each with second); clicks_left_out and keys_left_out count the rest; has_camera, true only for a recording the user made with the camera, and then camera with parts (in the form the camera argument takes), shadow and hidden_while_recording, the stretches the user hid the camera while recording, which have no picture; frame for frame_at_second.

#Editing a recording

Pass the path of an MP4 to annotate or open_in_editor and these options apply. Marks and blurs are drawn into every frame, or between their start_second and end_second. rotate, flip, resize, beautify and magnify marks are for screenshots only.

A full edit in one call: cut the first 2 seconds, play the waiting part at 8x, lower a noisy stretch, zoom on every click plus one slow zoom that follows the pointer, ripple highlights, a following blur on an email field, and a 960 pixel GIF:

{
  "tool": "annotate",
  "arguments": {
    "path": "/Users/you/Pictures/Screenshots/Agent/signup-flow.mp4",
    "pieces": [
      {"start_second": 2, "end_second": 7.5},
      {"start_second": 7.5, "end_second": 14, "speed": 8},
      {"start_second": 14, "end_second": 26}
    ],
    "sound": [{"start_second": 14, "end_second": 18, "volume": 0.4}],
    "zoom_on_clicks": true,
    "zooms": [
      {
        "start_second": 19, "end_second": 24, "x": 640, "y": 300,
        "width": 960, "height": 540, "transition_seconds": 0.8,
        "follow": "pointer"
      }
    ],
    "pointer": {"click_highlight": "ripple", "highlight_color": "#FFD60A"},
    "annotations": [
      {
        "type": "blur", "x": 820, "y": 412, "width": 420, "height": 48,
        "second": 15, "start_second": 14, "end_second": 26
      }
    ],
    "export": {"format": "gif", "width": 960, "fps": 15}
  }
}

Crop a recording to a vertical clip around the form, in video pixels, and make it a GIF:

{
  "tool": "annotate",
  "arguments": {
    "path": "/Users/you/Pictures/Screenshots/Agent/signup-flow.mp4",
    "crop": {"x": 560, "y": 0, "width": 800, "height": 1080},
    "annotations": [],
    "export": {"format": "gif", "width": 480, "fps": 12}
  }
}

Take the crop away again with "crop": false, or use "crop": {"aspect": "9:16"} for the largest centered 9:16 crop.

Keys for a recording the user made with Show keystrokes on, as last_recording returned it: words while typing a commit message, only shortcuts afterwards, nothing during a password, labels at the top in the light look:

{
  "tool": "annotate",
  "arguments": {
    "path": "/Users/you/Pictures/Screenshots/Sightline Recording 2026-10-24 10.12.04.mp4",
    "keys": {
      "parts": [
        {"start_second": 0, "end_second": 12, "mode": "words"},
        {"start_second": 12, "end_second": 15, "mode": "hidden"},
        {"start_second": 15, "end_second": 40, "mode": "shortcuts"}
      ],
      "position": "topCenter",
      "size": "large",
      "look": "light"
    },
    "annotations": []
  }
}

The camera of a recording the user made with it: a large circle in the middle for the intro, hidden while a password is typed, then a small rounded rectangle in the bottom right corner without a shadow:

{
  "tool": "annotate",
  "arguments": {
    "path": "/Users/you/Pictures/Screenshots/Sightline Recording 2026-11-06 14.20.51.mp4",
    "camera": {
      "parts": [
        {
          "start_second": 0, "end_second": 5, "x": 0.5, "y": 0.5,
          "height": 0.6, "shape": "circle"
        },
        {"start_second": 5, "end_second": 9, "hidden": true},
        {
          "start_second": 9, "end_second": 45, "x": 0.88, "y": 0.84,
          "height": 0.22, "shape": "rounded_rectangle"
        }
      ],
      "shadow": false
    },
    "annotations": []
  }
}

A drawn pointer at double size that hides while it rests, and a smaller MOV for a slide deck:

{
  "tool": "annotate",
  "arguments": {
    "path": "/Users/you/Pictures/Screenshots/Agent/demo-drawn.mp4",
    "annotations": [],
    "pointer": {
      "shown": true, "size": 2, "hide_when_idle": true,
      "click_highlight": "ring"
    },
    "export": {"format": "mov", "size": "1080p", "quality": "medium"}
  }
}

Returns path, duration, width, height, has_sound, frames for a GIF, and under following any blur or zoom whose content was lost. When the recording is open in the recording editor, the change goes there as one undo step and note says the file was left for the user to save. No inline image for a recording.

#Open editors

#list_editors

Lists the Sightline editor windows the user has open, front window first, so the agent knows what the user is looking at, also a picture with no file such as a copied capture. A window with several pictures lists each as its own editor: window is shared by the pictures of one window, shown marks the one it shows, and unsaved says it has changes not saved yet. front means the picture the front window shows.

{"tool": "list_editors", "arguments": {}}

Only the list:

{"tool": "list_editors", "arguments": {"include_image": false}}

Returns editors, each with id (stays the same while the picture is open), window, shown, unsaved, kind (screenshot or recording), title, width, height, path or null, for a recording crop or null, front, and editing, false while the window is in view mode. When the image is scaled down, preview_width and preview_height: multiply positions in it by width / preview_width.

#export_pictures

Writes every picture of an editor window as files, like Export All in the editor's File menu: each picture as it looks now, with its marks, crop, Style look and size, named after the picture, into a new folder inside the Agent folder. The pictures and their saved files are not changed.

Every picture of the front window as small WebP files for a pull request:

{
  "tool": "export_pictures",
  "arguments": {"editor": "front", "format": "webp", "quality": "small"}
}

Returns folder and paths, the files written in the window's column order; a picture that couldn't be written is named under failed. Credentials are blurred in the files when credential hiding is on.

#Style

#beautify

Puts a screenshot on a background with padding, rounded corners and a shadow, like the editor's Style panel.

The tool keeps the name beautify; in the app the panel is called Style. Without look, the fields change the look the image already has, else Soft. With editor and all_pictures, every picture of that window gets the look, like Apply Style to All Pictures.

A social card: a custom two color gradient, square, roomy padding, to a new file:

{
  "tool": "beautify",
  "arguments": {
    "path": "/Users/you/Pictures/Screenshots/Agent/dashboard.png",
    "background": "#0FB5A8,#0A7F9A", "aspect": "1:1", "padding": 120,
    "corners": 18, "shadow": 70, "name": "dashboard-square"
  }
}

Set the user's own look on the screenshot they have open:

{"tool": "beautify", "arguments": {"editor": "front", "look": "default"}}

Returns path, width, height and a preview. In the editor and on a saved edit the look stays adjustable; on a plain file it is drawn into the pixels, so a second call wraps it again. With all_pictures also editors, the window's pictures in column order.

#list_looks

Lists the built-in looks, the user's presets and the user's default look.

No parameters.

{"tool": "list_looks", "arguments": {}}

Returns looks, presets (each with background, background_opacity, padding, corners, shadow, aspect, has_background_image and reachable), and default or null.

#save_look

Saves a look as a preset the user can pick in the Style panel.

A house style for documentation:

{
  "tool": "save_look",
  "arguments": {
    "name": "Docs", "look": "paper", "padding": 40, "corners": 10,
    "shadow": 30, "aspect": "16:9"
  }
}

Returns The saved name and replaced.

#Captures and history

#last_screenshot

Returns this agent's own last capture, the one other tools use when path is left out. For the user's screenshot, call list_editors.

{"tool": "last_screenshot", "arguments": {"include_image": false}}

Returns The same fields as a capture, plus open_editors and a hint when the user has editors open, and agent_editor while an editor an agent opened is open.

#search_captures

Searches the user's screenshots and recordings like Sightline's History, by file name and by the text Sightline read in each screenshot on the Mac.

It searches the save folder and the Agent folder, the same index as History. Screenshots the user only copied, which History keeps for 7 days, aren't included. With credential hiding on, text near anything that looks like a credential doesn't match at all.

The invoice screenshots the user starred:

{
  "tool": "search_captures",
  "arguments": {
    "query": "invoice total", "kind": "screenshot", "favorites_only": true,
    "limit": 5
  }
}

The ten newest recordings:

{
  "tool": "search_captures",
  "arguments": {"kind": "recording", "limit": 10}
}

Returns results, newest first, each with path, kind, date (ISO 8601), favorite and, when the text matched, a snippet; total counts every match; reading_text says how many screenshots are still being read.

#Clipboard and replay

#copy_to_clipboard

Puts a screenshot on the user's clipboard at the resolution set in Sightline, or a recording as a file that pastes into Finder, Mail and chat apps. With editor it copies a screenshot the user has open as it looks there now, like Copy in the editor, or every picture of the window, like Copy All. The picture is not returned to the agent.

The marked capture from before:

{
  "tool": "copy_to_clipboard",
  "arguments": {
    "path": "/Users/you/Pictures/Screenshots/Agent/checkout-error-marked.png"
  }
}

All pictures of the window the user has open, ready to paste into an issue:

{"tool": "copy_to_clipboard", "arguments": {"editor": "front", "all_pictures": true}}

Returns The path that was copied, or with editor the editors copied.

#save_replay

Saves the last seconds of the display while Replay is on in Settings > Recording. Needs macOS 27.

{"tool": "save_replay", "arguments": {"seconds": 10}}

Returns path, duration, width and height, usable like a record_screen result.