@screenpipe/js
method | description | compatibility | example |
---|---|---|---|
openApplication(name) | launches an application | macOS only | pipe.operator.openApplication("Chrome") |
openUrl(url, browser?) | opens a url in a browser | macOS only | pipe.operator.openUrl("github.com") |
getByRole(role, options) | finds elements by accessibility role | macOS only | pipe.operator.getByRole("button", {app: "Chrome"}) |
getById(id, options) | gets element by id | macOS only | pipe.operator.getById("element-123", {app: "Chrome"}) |
.click() | clicks an element | macOS only | pipe.operator.getById(id).click() |
.fill(text) | enters text in a field | macOS only | pipe.operator.getById(id).fill("hello") |
.scroll(direction, amount) | scrolls an element | macOS only | pipe.operator.getById(id).scroll("down", 300) |
pixel.type(text) | types text | all platforms | pipe.operator.pixel.type("hello world") |
pixel.press(key) | presses a keyboard key | all platforms | pipe.operator.pixel.press("enter") |
pixel.moveMouse(x, y) | moves mouse cursor to position | all platforms | pipe.operator.pixel.moveMouse(100, 200) |
pixel.click(button) | clicks mouse button | all platforms | pipe.operator.pixel.click("left") |
openApplication
, getByRole
, click
, etc.getByRole()
, you’ll need to specify the accessibility role. here are common ones:
"button"
- clickable buttons"textfield"
- text input fields"searchfield"
- search input fields"checkbox"
- checkbox elements"radiobutton"
- radio button elements"combobox"
- dropdown menus"link"
- hyperlinks"image"
- images"statictext"
- text labels"scrollarea"
- scrollable containersactivateApp: true
option to ensure the target app is in focus