> ## Documentation Index
> Fetch the complete documentation index at: https://docs.rebind.gg/llms.txt
> Use this file to discover all available pages before exploring further.

# rb.screen

> Read pixel colors, find colors and images on screen, capture displays, and list them.

`rb.screen` reads what is on the screen: the color of one pixel, the first match
for a color or an image in a region, or a display as a PNG. A script reaches for it to react
to what a game or app shows, such as a health bar changing color or a button
appearing.

**Needs:** the `screen` or [`screen:target`](https://docs.rebind.gg/reference/sdk/modeline#scoped-permissions)
[permission](https://docs.rebind.gg/reference/sdk/modeline.md#permissions) for pixel reads in a script
that declares permissions; a coroutine for `capture`, `find_image`, and a
`search_color` region over 200,000 physical pixels, so call them inside `rb.run`
or `rb.async` (see [Where code runs](https://docs.rebind.gg/reference/sdk/globals.md#where-code-runs)).

This script prints the color under the cursor on F6, then looks for red in a 400
by 300 box at the top left of the primary display:

```luau theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
-- rebind: min_sdk=3.9.0 name=example version=1.0.0 permissions=screen
rb.bind("F6", rb.async(function()
  print("under the cursor: " .. rb.screen.pixel_hex())
  local hit = rb.screen.search_color({ 0, 0, 400, 300 }, "ff0000", 0.1)
  if hit then
    print("red at " .. hit.x .. ", " .. hit.y)
  else
    print("no red in the box")
  end
end))
```

| Signature | Coroutine only |
| - | - |
| `rb.screen.pixel_hex(x: number?, y: number?) -> string` | no |
| `rb.screen.search_color(region: { number }, color: string, tolerance: number?) -> PixelMatch?` | over 200,000 physical pixels |
| `rb.screen.list() -> { MonitorEntry }` | no |
| `rb.screen.capture(options: CaptureOptions?) -> CaptureResult` | yes |
| `rb.screen.find_image(needle: string, options: FindImageOptions?) -> ImageMatch?` | yes |
| `rb.screen.work_area(x: number?, y: number?) -> WindowPos` | no |

Coordinates are logical screen pixels on the virtual desktop; displays left of or
above the primary have negative coordinates. Outside a coroutine, `capture` raises
`Screen.Capture: call inside rb.run() or an rb.async handler; it yields instead of blocking the tick`,
and `find_image` raises the same message starting `Screen.FindImage:`.

## rb.screen.pixel\_hex

Returns the color of the pixel at `x`, `y`, or under the cursor when they are
omitted, as `"rrggbb"`, lowercase.

## rb.screen.search\_color

* `region`: `{ x1, y1, x2, y2 }`; `x2` must be greater than `x1` and `y2`
  greater than `y1`.
* `color`: six hex digits, `"rrggbb"`, any case, without `#`; anything else
  raises `SearchForColor: invalid color '<color>' (expected 6-char hex like "ff0000")`.
  For a value from `rb.ui.color`, pass `cfg.color:sub(2)`.
* `tolerance`: 0 (exact, default) to 1, per channel as a fraction of 255.
* Returns `{ x, y }` of the first match, scanning left to right and top to
  bottom, or `nil`. A failed read raises.

A region up to 200,000 physical pixels is scanned where it is called. A larger
one raises outside a coroutine:
`Screen.SearchForColor: region of <n> physical pixels exceeds the 200000 inline limit; call inside rb.run()`.

## rb.screen.list

One `{ index, x, y, width, height, primary }` per display, `index` starting at 1.
Returns an empty table when displays cannot be listed.

## rb.screen.capture

Options:

* `display`: index from `rb.screen.list()`. Default: the display under the
  cursor, else the primary. Out of range raises
  `Screen.Capture: display <n> out of range (1..<count>)`.
* `region`: `{ x, y, w, h }` in that display's own coordinates, clamped to the
  display. A zero width or height raises.
* `max_edge`: downscale so the longer side is at most this many pixels; `0` or
  omitted keeps native resolution.

Returns `{ image, width, height, display, cursor }`: `image` is a base64 PNG,
`width` and `height` are its pixel size after any downscale, `display` is the
display's `{ index, x, y, width, height, primary }`, and `cursor` is the cursor
position read with the frame. On a high-density display the image has more
pixels than the logical region.

## rb.screen.find\_image

Captures like `capture` and scans for a needle image.

* `needle`: path to a PNG in the script directory or its data folder. Pixels with alpha below 128
  match anything.
* `options`: `display` and `region` as in `capture`; `tolerance` as in
  `search_color`; `scales`, a list of 1 to 8 ratios between 0.1 and 10 to
  resize the needle by, tried in order (default `{ 1 }`).
* Returns `{ x, y, width, height }` of the first match in screen coordinates,
  or `nil`.

The needle is compared at native pixel scale. A needle cut from a 2x display
finds its 1x rendering with `scales = { 1, 0.5 }`. Each ratio costs one more
scan, and resizing softens edges, so raise `tolerance` with it.

This bind looks for `ok-button.png` on F6 and clicks the center of the first match:

```luau theme={"theme":{"light":"github-light-default","dark":"github-dark-default"}}
-- rebind: min_sdk=3.9.0 name=example version=1.0.0
rb.bind("F6", rb.async(function()
  local hit = rb.screen.find_image("ok-button.png", { tolerance = 0.05 })
  if hit then
    rb.hid.move_to(hit.x + hit.width / 2, hit.y + hit.height / 2)
    rb.hid.press("Mouse1")
  end
end))
```

## rb.screen.work\_area

The usable desktop of the display containing the point, or the cursor when
omitted: the display minus the taskbar. Windows only; elsewhere it raises.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.