Type text into a UI element

screenpipe documentation

Terminator: Introduction

Terminator: Getting Started

typescript sdk reference

python sdk reference

Welcome to screenpipe

Getting Started

Operator

Architecture

Pipes

Contributing

Query Screenpipe for content based on various filters.

## Use Cases

- Search for specific text across all applications
- Find content from a specific application or window
- Get screenshots from a particular time period
- Retrieve all visits to a specific website

## Examples

### Basic search for recent browser activity:
```js
const githubActivity = await pipe.queryScreenpipe({
  browserUrl: "github.com",
  contentType: "ocr",
  limit: 20,
  includeFrames: true
});
```

### Search for specific text with date filters:
```js
const searchResults = await pipe.queryScreenpipe({
  q: "authentication",
  browserUrl: "auth0.com",
  appName: "Chrome",
  contentType: "ocr",
  startTime: new Date(Date.now() - 7 * 24 * 60 * 60 * 1000).toISOString(),
  endTime: new Date().toISOString(),
  limit: 50
});
```


Search screen and audio content with various filters

Keyword-based search

Find UI elements in applications by role to enable desktop automation.

## Use Cases

- Find buttons, text fields, and other UI elements for automation
- Build workflows that interact with desktop applications
- Create AI-powered desktop automation agents

## Examples

### Find all buttons in Chrome:
```js
const buttons = await pipe.operator
  .getByRole("button", {
    app: "Chrome",
    activateApp: true
  })
  .all(5);
```

### Click a button:
```js
const button = await pipe.operator
  .getByRole("button", { app: "Chrome" })
  .first();
  
await pipe.operator
  .getById(button.id, { app: "Chrome" })
  .click();
```


Find UI elements in applications for automation

Open an application

Open a URL in a browser

Click on a UI element

Post experimentaloperatorscroll

Post experimentaloperatorpress key

Post experimentaloperatorget text

Post experimentaloperatorlist interactable elements

Post experimentaloperatorclick by index

Post experimentaloperatortype by index

Post experimentaloperatorpress key by index

Post experimentalinput control

Get audiolist

Get visionlist

Post tags 

Delete tags 

Get pipesinfo

Get pipeslist

Post pipesdownload

Post pipesdownload private

Post pipesenable

Post pipesdisable

Post pipesupdate

Post pipesupdate version

Post pipesdelete

Post pipespurge

Get frames

Get health

Post raw sql

Post add

Post experimentalframesmerge

Get experimentalvalidatemedia

Get pipesbuild status

Post v1embeddings

Get speakersunnamed

Post speakersupdate

Get speakerssearch

Post speakersdelete

Post speakershallucination

Post speakersmerge

Get speakerssimilar

Post audiostart

Post audiostop

Post audiodevicestart

Post audiodevicestop

Context Retrieval

Computer Automation

API Reference

Speaker Management

Audio Control

Type text into a UI element

Body

Response