# give AI memory of your screen Source: https://docs.screenpi.pe/ai-memory Give your AI assistant memory of everything on your screen. Use screenpipe with Claude, Cursor, Ollama, and other LLMs. screenpipe acts as a memory layer for AI — it gives LLMs context about what you've been doing on your computer. ## why AI needs memory LLMs are stateless — they don't know what you were working on 5 minutes ago. screenpipe bridges this by: * capturing everything on your screen 24/7 * extracting text via OCR * making it searchable via REST API on `localhost:3030` ## connect to AI ### MCP (recommended) screenpipe has a built-in MCP server that works with Claude Desktop, Cursor, and other MCP-compatible tools: ```json theme={null} { "mcpServers": { "screenpipe": { "command": "npx", "args": ["-y", "screenpipe-mcp"] } } } ``` see [MCP server setup](/mcp-server) for details. ### pipes (scheduled agents) [pipes](/pipes) are AI agents that run on a schedule and act on your screen data automatically — like syncing to Obsidian, tracking time in Toggl, or sending daily summaries. ### direct API any tool that can make HTTP requests can query screenpipe: ```bash theme={null} # get recent screen activity curl "http://localhost:3030/search?content_type=ocr&limit=20" # search for specific content curl "http://localhost:3030/search?q=meeting+notes&app_name=Slack&limit=10" ``` ## use cases | use case | how | | ---------------------------- | -------------------------- | | "what was I working on?" | search by time range | | "summarize today's meetings" | query audio transcriptions | | "find that code snippet" | search OCR text | | "auto-track my time" | toggl-sync pipe | | "sync activity to notes" | obsidian-sync pipe | ## privacy-first * all data stays on your device * use local LLMs (Ollama, LMStudio) for complete privacy * filter what gets captured with `--ignored-windows` and `--included-windows` * no data sent to cloud unless you explicitly choose cloud providers ## next steps * [set up MCP server](/mcp-server) — connect to Claude, Cursor * [set up pipes](/pipes) — scheduled AI agents * [API reference](/cli-reference) — search parameters and API endpoints * [join our discord](https://discord.gg/screenpipe) — get help from the community ## get screenpipe screenpipe gives your AI assistants memory of everything on your screen. lifetime plans available — no subscriptions needed. [download screenpipe →](https://screenpi.pe/onboarding) # Get raw audio pipeline metrics snapshot Source: https://docs.screenpi.pe/api-reference/audio-control/get-raw-audio-pipeline-metrics-snapshot openapi.yaml get /audio/metrics # List available audio devices Source: https://docs.screenpi.pe/api-reference/audio-control/list-available-audio-devices openapi.yaml get /audio/list # Start audio processing Source: https://docs.screenpi.pe/api-reference/audio-control/start-audio-processing openapi.yaml post /audio/start # Start recording a specific audio device Source: https://docs.screenpi.pe/api-reference/audio-control/start-recording-a-specific-audio-device openapi.yaml post /audio/device/start # Stop audio processing Source: https://docs.screenpi.pe/api-reference/audio-control/stop-audio-processing openapi.yaml post /audio/stop # Stop recording a specific audio device Source: https://docs.screenpi.pe/api-reference/audio-control/stop-recording-a-specific-audio-device openapi.yaml post /audio/device/stop # Download and import data from other devices Source: https://docs.screenpi.pe/api-reference/cloud-sync/download-and-import-data-from-other-devices openapi.yaml post /sync/download # Get cloud sync status Source: https://docs.screenpi.pe/api-reference/cloud-sync/get-cloud-sync-status openapi.yaml get /sync/status # Initialize cloud sync with credentials Source: https://docs.screenpi.pe/api-reference/cloud-sync/initialize-cloud-sync-with-credentials openapi.yaml post /sync/init # Lock sync for exclusive access Source: https://docs.screenpi.pe/api-reference/cloud-sync/lock-sync-for-exclusive-access openapi.yaml post /sync/lock # Trigger a sync operation Source: https://docs.screenpi.pe/api-reference/cloud-sync/trigger-a-sync-operation openapi.yaml post /sync/trigger # Add content (frames or transcriptions) to database Source: https://docs.screenpi.pe/api-reference/content-management/add-content-frames-or-transcriptions-to-database openapi.yaml post /add # Add tags to content Source: https://docs.screenpi.pe/api-reference/content-management/add-tags-to-content openapi.yaml post /tags/{content_type}/{id} # Execute raw SQL query Source: https://docs.screenpi.pe/api-reference/content-management/execute-raw-sql-query openapi.yaml post /raw_sql # Remove tags from content Source: https://docs.screenpi.pe/api-reference/content-management/remove-tags-from-content openapi.yaml delete /tags/{content_type}/{id} # Keyword-based search with optional grouping Source: https://docs.screenpi.pe/api-reference/context-retrieval/keyword-based-search-with-optional-grouping openapi.yaml get /search/keyword # Search screen and audio content with various filters Source: https://docs.screenpi.pe/api-reference/context-retrieval/search-screen-and-audio-content-with-various-filters openapi.yaml get /search Query Screenpipe for content based on various filters. ## Use Cases - Search for specific text across all applications - Find content from a specific application or window - Get screenshots from a particular time period - Retrieve all visits to a specific website ## Examples ### Basic search for recent browser activity: ```js const githubActivity = await pipe.queryScreenpipe({ browserUrl: "github.com", contentType: "ocr", limit: 20, includeFrames: true }); ``` ### Search for specific text with date filters: ```js const searchResults = await pipe.queryScreenpipe({ q: "authentication", browserUrl: "auth0.com", appName: "Chrome", contentType: "ocr", startTime: new Date(Date.now() - 7 * 24 * 60 * 60 * 1000).toISOString(), endTime: new Date().toISOString(), limit: 50 }); ``` # Merge multiple frames into a video Source: https://docs.screenpi.pe/api-reference/experimental/merge-multiple-frames-into-a-video openapi.yaml post /experimental/frames/merge # Validate a media file Source: https://docs.screenpi.pe/api-reference/experimental/validate-a-media-file openapi.yaml get /experimental/validate/media # Find next frame with valid video file on disk Source: https://docs.screenpi.pe/api-reference/frames/find-next-frame-with-valid-video-file-on-disk openapi.yaml get /frames/next-valid Allows the frontend to skip directly to a valid frame instead of trying each one when frames fail to load. # Get frame image data Source: https://docs.screenpi.pe/api-reference/frames/get-frame-image-data openapi.yaml get /frames/{frame_id} Returns the JPEG image for a given frame. Supports optional PII redaction. # Get frame metadata (timestamp) for deep link navigation Source: https://docs.screenpi.pe/api-reference/frames/get-frame-metadata-timestamp-for-deep-link-navigation openapi.yaml get /frames/{frame_id}/metadata # Get OCR text positions for a frame Source: https://docs.screenpi.pe/api-reference/frames/get-ocr-text-positions-for-a-frame openapi.yaml get /frames/{frame_id}/ocr Returns OCR text positions with bounding boxes for text selection overlay on screenshots. # Delete a pipe Source: https://docs.screenpi.pe/api-reference/pipes/delete-a-pipe openapi.yaml delete /pipes/{id} # Enable or disable a pipe Source: https://docs.screenpi.pe/api-reference/pipes/enable-or-disable-a-pipe openapi.yaml post /pipes/{id}/enable # Get pipe execution history from DB Source: https://docs.screenpi.pe/api-reference/pipes/get-pipe-execution-history-from-db openapi.yaml get /pipes/{id}/executions # Get recent pipe run logs (in-memory) Source: https://docs.screenpi.pe/api-reference/pipes/get-recent-pipe-run-logs-in-memory openapi.yaml get /pipes/{id}/logs # Get single pipe detail Source: https://docs.screenpi.pe/api-reference/pipes/get-single-pipe-detail openapi.yaml get /pipes/{id} # Install a pipe from URL or local path Source: https://docs.screenpi.pe/api-reference/pipes/install-a-pipe-from-url-or-local-path openapi.yaml post /pipes/install # List all pipes with status Source: https://docs.screenpi.pe/api-reference/pipes/list-all-pipes-with-status openapi.yaml get /pipes Re-scans disk so pipes installed externally (e.g. via CLI) are picked up. # Stop a running pipe Source: https://docs.screenpi.pe/api-reference/pipes/stop-a-running-pipe openapi.yaml post /pipes/{id}/stop # Trigger a manual pipe run Source: https://docs.screenpi.pe/api-reference/pipes/trigger-a-manual-pipe-run openapi.yaml post /pipes/{id}/run # Update pipe config fields Source: https://docs.screenpi.pe/api-reference/pipes/update-pipe-config-fields openapi.yaml post /pipes/{id}/config # Delete a speaker and associated audio chunks Source: https://docs.screenpi.pe/api-reference/speaker-management/delete-a-speaker-and-associated-audio-chunks openapi.yaml post /speakers/delete # Get speakers similar to a given speaker Source: https://docs.screenpi.pe/api-reference/speaker-management/get-speakers-similar-to-a-given-speaker openapi.yaml get /speakers/similar # Get unnamed speakers Source: https://docs.screenpi.pe/api-reference/speaker-management/get-unnamed-speakers openapi.yaml get /speakers/unnamed # Mark a speaker as hallucination Source: https://docs.screenpi.pe/api-reference/speaker-management/mark-a-speaker-as-hallucination openapi.yaml post /speakers/hallucination # Merge two speakers into one Source: https://docs.screenpi.pe/api-reference/speaker-management/merge-two-speakers-into-one openapi.yaml post /speakers/merge # Reassign an audio chunk to a different speaker Source: https://docs.screenpi.pe/api-reference/speaker-management/reassign-an-audio-chunk-to-a-different-speaker openapi.yaml post /speakers/reassign # Search speakers by name Source: https://docs.screenpi.pe/api-reference/speaker-management/search-speakers-by-name openapi.yaml get /speakers/search # Undo a speaker reassignment Source: https://docs.screenpi.pe/api-reference/speaker-management/undo-a-speaker-reassignment openapi.yaml post /speakers/undo-reassign # Update speaker name or metadata Source: https://docs.screenpi.pe/api-reference/speaker-management/update-speaker-name-or-metadata openapi.yaml post /speakers/update # Stream frames via SSE Source: https://docs.screenpi.pe/api-reference/streaming/stream-frames-via-sse openapi.yaml get /stream/frames Server-Sent Events stream for real-time frame data. Not fully representable in OpenAPI. # WebSocket for health monitoring Source: https://docs.screenpi.pe/api-reference/streaming/websocket-for-health-monitoring openapi.yaml get /ws/health # WebSocket for real-time events Source: https://docs.screenpi.pe/api-reference/streaming/websocket-for-real-time-events openapi.yaml get /ws/events WebSocket endpoint for streaming OCR, audio, and UI events in real-time. # WebSocket for video export Source: https://docs.screenpi.pe/api-reference/streaming/websocket-for-video-export openapi.yaml get /frames/export WebSocket endpoint for exporting frames as video. # Health check with pipeline metrics Source: https://docs.screenpi.pe/api-reference/system/health-check-with-pipeline-metrics openapi.yaml get /health # Get UI event statistics grouped by app and event type Source: https://docs.screenpi.pe/api-reference/ui-events/get-ui-event-statistics-grouped-by-app-and-event-type openapi.yaml get /ui-events/stats # Search UI events (clicks, keystrokes, clipboard, etc.) Source: https://docs.screenpi.pe/api-reference/ui-events/search-ui-events-clicks-keystrokes-clipboard-etc openapi.yaml get /ui-events # Get raw vision pipeline metrics snapshot Source: https://docs.screenpi.pe/api-reference/vision-control/get-raw-vision-pipeline-metrics-snapshot openapi.yaml get /vision/metrics # Get vision system status and permissions Source: https://docs.screenpi.pe/api-reference/vision-control/get-vision-system-status-and-permissions openapi.yaml get /vision/status # List available monitors Source: https://docs.screenpi.pe/api-reference/vision-control/list-available-monitors openapi.yaml get /vision/list # Apple Intelligence — on-device daily summaries & auto reminders Source: https://docs.screenpi.pe/apple-intelligence screenpipe uses Apple Intelligence (on-device Foundation Models) to generate daily summaries and automatically extract action items into Apple Reminders. screenpipe uses Apple's on-device Foundation Models to analyze your recorded activity — no data leaves your Mac. requires macOS 26+ with Apple Intelligence enabled on Apple Silicon. ## daily summaries screenpipe automatically generates a summary of your day at **6 PM** (when plugged in). the summary appears in your **timeline** view and includes: * **one-liner** — casual summary of your day * **key moments** — up to 5 important events from audio transcriptions * **action items** — things mentioned in conversations, ranked by urgency * **people mentioned** — names and context from your conversations you can also click the Apple Intelligence icon in the timeline to generate a summary manually at any time. ### how it works 1. fetches your audio transcriptions for the day 2. sends them to Apple's on-device Foundation Models (never leaves your Mac) 3. returns a structured summary with key moments, action items, and people 4. caches the result locally so it loads instantly next time ### enable/disable open **settings** → **connections** → toggle **Apple Intelligence** on or off. ## auto reminders screenpipe can scan your activity every 30 minutes and automatically create action items in **Apple Reminders**. ### setup 1. open **settings** → **connections** → find the **Apple Intelligence** card 2. click **"Connect Apple Reminders"** (grants Reminders permission) 3. toggle on **auto-scanning** once enabled, screenpipe will: * scan your screen & audio every 30 minutes * extract action items using Apple Intelligence * create reminders in a **"Screenpipe"** list in Apple Reminders * auto-deduplicate so you don't get repeat reminders ### options * **audio only** — skip screen data, only analyze audio transcriptions (on by default) * **custom prompt** — add instructions like "focus on coding tasks" or "ignore social media" * **scan now** — manually trigger a scan anytime ## requirements * **macOS 26+** (Tahoe) with Apple Intelligence enabled * **Apple Silicon** (M1 or later) * screenpipe running on localhost:3030 if Apple Intelligence shows as "not available", check that it's enabled in **System Settings → Apple Intelligence & Siri** and that your Mac meets the requirements. ## vs Apple's built-in screen time | feature | screenpipe | Apple Screen Time | | ------------------- | --------------------------------- | ----------------- | | full text search | ✅ OCR on every frame | ❌ app usage only | | audio transcription | ✅ local Whisper | ❌ | | AI summaries | ✅ daily summary with action items | ❌ | | auto reminders | ✅ action items → Apple Reminders | ❌ | | API access | ✅ full REST API | ❌ | | cross-platform | ✅ macOS, Windows, Linux | ❌ Apple only | questions? [join our discord](https://discord.gg/screenpipe). # architecture Source: https://docs.screenpi.pe/architecture How screenpipe uses event-driven capture, accessibility tree extraction, and OCR fallback to build a searchable local memory. ## overview screenpipe is a Rust application that captures your screen and audio using an event-driven architecture, processes them locally, and stores everything in a SQLite database. instead of recording every second, it listens for meaningful OS events and captures only when something actually changes — pairing each screenshot with accessibility tree data for maximum quality at minimal cost. ```mermaid theme={null} graph LR subgraph trigger["event triggers"] E1[app switch] E2[click / scroll] E3[typing pause] E4[idle timer] end subgraph capture["paired capture"] SS[screenshot] A11Y[accessibility tree] OCR[OCR fallback] end subgraph audio["audio"] MIC[microphone] SYS[system audio] STT[speech-to-text] end subgraph store["storage"] DB[(SQLite)] FS[JPEG files] end subgraph serve["API · localhost:3030"] REST[REST API] MCP[MCP server] end E1 & E2 & E3 & E4 --> SS SS --> A11Y A11Y -->|empty?| OCR A11Y --> DB OCR --> DB SS --> FS MIC & SYS --> STT --> DB DB --> REST DB --> MCP FS --> REST REST --> P[pipes / AI agents] MCP --> AI[Claude · Cursor · etc.] ``` ## data flow ```mermaid theme={null} sequenceDiagram participant OS as OS Events participant Capture participant A11Y as Accessibility participant OCR as OCR (fallback) participant Audio participant SQLite participant API participant AI OS->>Capture: meaningful event (click, app switch, typing pause...) Capture->>Capture: screenshot Capture->>A11Y: walk accessibility tree alt accessibility data available A11Y->>SQLite: structured text + metadata else accessibility empty (remote desktop, games) A11Y->>OCR: fallback OCR->>SQLite: extracted text + metadata end Capture->>SQLite: JPEG frame loop every 30s chunk Audio->>SQLite: transcription + speaker Audio->>SQLite: audio file end AI->>API: search query API->>SQLite: SQL lookup SQLite-->>API: results API-->>AI: JSON response ``` ## crates screenpipe is a Rust workspace with specialized crates: ```mermaid theme={null} graph TD APP[screenpipe-app-tauri
desktop app] SERVER[screenpipe-server
REST API · routes] DB[screenpipe-db
SQLite · types] VISION[screenpipe-vision
screen capture · OCR] AUDIO[screenpipe-audio
audio capture · STT] CORE[screenpipe-core
pipes · config] EVENTS[screenpipe-events
event system] A11Y[screenpipe-accessibility
UI events · macOS, Windows] AI[screenpipe-apple-intelligence
Foundation Models] INT[screenpipe-integrations
MCP · reminders] APP --> SERVER APP --> AI SERVER --> DB SERVER --> VISION SERVER --> AUDIO SERVER --> CORE SERVER --> EVENTS AUDIO --> DB VISION --> DB CORE --> DB A11Y --> DB INT --> SERVER ``` ## layers ### 1. event-driven capture screenpipe listens for meaningful OS events instead of polling at a fixed FPS. when an event fires, it captures a screenshot and walks the accessibility tree together — same timestamp, same frame. | trigger | description | | ------------------ | ----------------------------------------------------- | | **app switch** | user switched to a different application | | **window focus** | a new window gained focus | | **click / scroll** | user interacted with the UI | | **typing pause** | user stopped typing (debounced) | | **clipboard copy** | content copied to clipboard | | **idle fallback** | periodic capture every \~5s when nothing is happening | | what | how | crate | | ------------------- | -------------------------------------------------------------------------- | -------------------------- | | **screen** | event-triggered screenshot of the active monitor | `screenpipe-vision` | | **text extraction** | accessibility tree walk (structured text: buttons, labels, fields) | `screenpipe-accessibility` | | **OCR fallback** | when accessibility data is empty (remote desktops, games, some Linux apps) | `screenpipe-vision` | | **audio** | multiple input/output devices in configurable chunks (default 30s) | `screenpipe-audio` | ### 2. processing | engine | type | platform | when used | | ------------------ | --------------- | -------------------- | -------------------------------------- | | accessibility tree | text extraction | macOS, Windows | primary — used for every capture | | Apple Vision | OCR | macOS | fallback when accessibility is empty | | Windows native | OCR | Windows | fallback when accessibility is empty | | Tesseract | OCR | Linux | primary (accessibility support varies) | | Whisper | speech-to-text | local, all platforms | audio transcription | | Deepgram | speech-to-text | cloud API | optional cloud audio | additional processing: speaker identification, PII redaction, frame deduplication (skips identical frames). ### 3. storage all data stays local on your machine: * **SQLite** at `~/.screenpipe/db.sqlite` — metadata, accessibility text, OCR text, transcriptions, speakers, tags, UI elements * **media** at `~/.screenpipe/data/` — JPEG screenshots (event-driven frames), audio chunks ### 4. API REST API on `localhost:3030`: | endpoint | description | | ---------------------- | ------------------------------------------------------- | | `/search` | filtered content retrieval (OCR, audio, accessibility) | | `/search/keyword` | keyword search with text positions | | `/elements` | lightweight UI element search (accessibility tree data) | | `/frames/{id}` | access captured frames | | `/frames/{id}/context` | accessibility text + URLs + OCR fallback for a frame | | `/health` | system status and metrics | | `/raw_sql` | direct database queries | | `/ai/chat/completions` | Apple Intelligence (macOS 26+) | see [API reference](/cli-reference) for the full endpoint list. ### 5. pipes [pipes](/pipes) are AI agents (`.md` prompt files) that run on your screen data. they're executed by an AI agent that reads the prompt, queries the screenpipe API, and takes action. pipes live in `~/.screenpipe/pipes/{name}/` and run on cron-like schedules. ### 6. desktop app the desktop app is built with **Tauri** (Rust backend) + **Next.js** (React frontend): ```mermaid theme={null} graph LR subgraph tauri["Tauri shell"] RS[Rust backend
commands · permissions · tray] WV[WebView] end subgraph frontend["Next.js frontend"] PAGES[pages
chat · timeline · settings] STORE[Zustand stores] UI[shadcn/ui components] end subgraph backend["screenpipe-server"] API[REST API :3030] end RS --> WV WV --> PAGES PAGES --> STORE STORE --> UI PAGES --> API ``` ## database schema key tables: | table | stores | | ---------------------- | -------------------------------------------------------------------------------------------------- | | `frames` | captured screen frame metadata (includes `snapshot_path`, `accessibility_text`, `capture_trigger`) | | `ocr_text` | OCR fallback text extracted from frames | | `elements` | UI elements from accessibility tree (buttons, labels, text fields) with FTS5 search | | `audio_chunks` | audio recording metadata | | `audio_transcriptions` | text from audio | | `speakers` | identified speakers | | `ui_events` | keyboard, mouse, clipboard events | | `tags` | user-applied tags on content | inspect directly: ```bash theme={null} sqlite3 ~/.screenpipe/db.sqlite .schema ``` ## resource usage runs 24/7 on a MacBook Pro M3 (32 GB) or a \$400 Windows laptop: | metric | typical value | | ------- | -------------------------------------------------------------------------------- | | RAM | \~600 MB | | CPU | \~5-10% | | storage | \~5-10 GB/month (event-driven capture only stores frames when something changes) | ## source code | component | path | | -------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | API server | [screenpipe-server/src/](https://github.com/screenpipe/screenpipe/tree/main/crates/screenpipe-server/src) | | screen capture | [screenpipe-vision/src/core.rs](https://github.com/screenpipe/screenpipe/blob/main/crates/screenpipe-vision/src/core.rs) | | audio capture | [screenpipe-audio/src/](https://github.com/screenpipe/screenpipe/tree/main/crates/screenpipe-audio/src) | | database | [screenpipe-db/src/db.rs](https://github.com/screenpipe/screenpipe/blob/main/crates/screenpipe-db/src/db.rs) | | pipes | [screenpipe-core/src/pipes/](https://github.com/screenpipe/screenpipe/tree/main/crates/screenpipe-core/src/pipes) | | MCP server | [screenpipe-mcp/src/index.ts](https://github.com/screenpipe/screenpipe/blob/main/crates/screenpipe-integrations/screenpipe-mcp/src/index.ts) | | desktop app | [screenpipe-app-tauri/](https://github.com/screenpipe/screenpipe/tree/main/apps/screenpipe-app-tauri) | # ChatGPT — use your subscription with screenpipe Source: https://docs.screenpi.pe/chatgpt Connect your ChatGPT Plus or Pro subscription to power screenpipe's AI chat and pipes — no API key needed. ChatGPT if you have a ChatGPT Plus or Pro subscription, you can connect it directly to screenpipe. this lets you use OpenAI's latest models for chat, summaries, and pipes without managing API keys or paying extra. ## what you get * **AI chat** — ask questions about your screen history, meetings, and activity using GPT models * **pipes** — run scheduled or on-demand automations (summaries, time tracking, etc.) powered by your subscription * **no API key needed** — sign in with your OpenAI account, that's it ## how to connect 1. open screenpipe 2. click the **model selector** next to the chat input (shows "OAI", "Ollama", etc.) 3. select **ChatGPT** as your provider 4. click **sign in with ChatGPT** 5. a browser window opens — log in with your OpenAI account and approve access 6. done — you'll see a green checkmark and your available models ## choosing a model after signing in, you can pick from your available models in the dropdown. these depend on your subscription tier: | subscription | models available | | ---------------- | ----------------------------------------- | | **ChatGPT Plus** | GPT-4o, GPT-4o mini, and more | | **ChatGPT Pro** | GPT-4o, GPT-4o mini, o1, o1-pro, and more | the model list updates automatically based on what your subscription includes. ## using it for pipes pipes are automations that run on a schedule or on demand. when you connect your ChatGPT subscription, pipes use it automatically. examples: * **toggl sync** — auto-track time in Toggl based on your screen activity, every 30 minutes * **day recap** — one-click summary of what you accomplished today * **meeting summary** — summarize meeting transcripts with action items to set up a pipe: 1. go to **pipes** in the sidebar 2. pick a pipe or create your own 3. in the pipe settings, select your ChatGPT preset 4. enable the pipe ## how it's different from MCP screenpipe also supports [MCP integrations](/mcp-server) with ChatGPT Desktop, Claude, and other apps. that's a different thing: | | ChatGPT subscription | MCP integration | | ------------------ | -------------------------------------------------- | ------------------------------------------------------------------ | | **what it does** | powers screenpipe's built-in AI chat and pipes | lets external apps (ChatGPT Desktop, Claude) query screenpipe data | | **setup** | sign in with your OpenAI account inside screenpipe | add screenpipe as an MCP server in the external app | | **where you chat** | in screenpipe's chat window | in the external app (ChatGPT Desktop, Claude, etc.) | | **runs pipes** | yes | no | you can use both at the same time. ## signing out to disconnect your ChatGPT account: 1. open the model selector 2. click the **sign out** button next to your ChatGPT connection this removes the stored tokens from your machine. your data stays local. ## troubleshooting **sign-in window doesn't open?** * make sure screenpipe is running * try restarting the app and signing in again **"could not get token" error?** * your session may have expired — sign out and sign back in * check that your ChatGPT subscription is active at [openai.com](https://openai.com) **models not showing up?** * sign out and sign back in to refresh your token * some models are only available on higher subscription tiers **pipes failing with auth errors?** * tokens refresh automatically, but if it persists, sign out and back in still stuck? [ask in our discord](https://discord.gg/screenpipe). # Claude Code - AI coding with screen context Source: https://docs.screenpi.pe/claude-code Use screenpipe with Claude Code CLI to give Claude access to your screen history, meeting transcriptions, and app context while coding. [Claude Code](https://code.claude.com) is Anthropic's official CLI for agentic coding. with screenpipe integration, Claude can reference what you've been working on, recall information from your screen, and access meeting transcriptions. ## setup ```bash theme={null} claude mcp add screenpipe --transport stdio -- npx -y screenpipe-mcp ``` to make it available across all your projects: ```bash theme={null} claude mcp add screenpipe --transport stdio --scope user -- npx -y screenpipe-mcp ``` ## verify connection ```bash theme={null} # list MCP servers claude mcp list # or inside Claude Code, use /mcp ``` ## available tools once connected, Claude Code has access to these tools: | tool | description | | ---------------- | --------------------------------------------------------------------------- | | `search-content` | search screen OCR, audio transcriptions, input events with time/app filters | | `export-video` | create video exports from screen recordings | ## usage examples ask Claude Code to use screenpipe naturally: ``` > what was I looking at in my browser an hour ago? > find mentions of "kubernetes" from my screen today > show me audio transcriptions from my last meeting > what code was I reading in VS Code yesterday about async? > export a video of my screen from 2-3pm today > what did I type in Slack today? > show me my app usage stats for the past 2 hours > what did I copy to clipboard recently? > which apps did I switch between most today? ``` ## search parameters ### search-content (vision + audio + input) | parameter | description | | ---------------- | ---------------------------------------------------------------------- | | `q` | search query (optional - omit to get all content) | | `content_type` | `vision`, `audio`, `accessibility`, `input`, or `all` (default: `all`) | | `limit` | max results (default: 10) | | `offset` | pagination offset | | `start_time` | ISO 8601 UTC start time | | `end_time` | ISO 8601 UTC end time | | `app_name` | filter by app (e.g., "Chrome", "Slack") | | `window_name` | filter by window title | | `include_frames` | include base64 screenshots | | `speaker_ids` | comma-separated speaker IDs for audio filtering | | `speaker_name` | filter audio by speaker name | ## example workflows **recall context from earlier:** ``` > I was reading a blog post about rust async earlier today, > search screenpipe and summarize the key points ``` **reference meeting discussion:** ``` > search my audio transcriptions for what was discussed in standup > about the API refactor, then help me implement it ``` **debug with screen history:** ``` > I saw an error message flash on screen, search screenpipe > to find it and help me fix the issue ``` **find code examples:** ``` > search screenpipe for the python code I was looking at > in the browser yesterday about asyncio patterns ``` **track what you typed:** ``` > what did I write in Notion this morning? search for my > keyboard input using content_type=input ``` **recall clipboard history:** ``` > I copied something important earlier, search clipboard > events to find it ``` ## requirements * screenpipe running on localhost:3030 * Claude Code CLI installed * Node.js >= 18.0.0 ## troubleshooting **MCP not connecting?** * verify screenpipe is running: `curl http://localhost:3030/health` * check MCP status in Claude Code: `/mcp` * remove and re-add: `claude mcp remove screenpipe && claude mcp add screenpipe --transport stdio -- npx -y screenpipe-mcp` **queries returning empty?** * check screenpipe has data: `curl "http://localhost:3030/search?limit=1"` * ensure screen recording permissions are granted * verify the time range you're querying **permission errors?** * macos: check System Settings > Privacy & Security > Screen Recording * ensure screenpipe app is listed and enabled still stuck? [ask in our discord](https://discord.gg/screenpipe). # API reference Source: https://docs.screenpi.pe/cli-reference REST API reference for screenpipe: search, frames, audio, tags, and more. screenpipe serves a REST API on `localhost:3030`. use this to integrate with any tool or build custom automations. for the full interactive API reference with request/response schemas, see the [API reference tab](/api-reference). ## endpoints | method | endpoint | description | | ------ | ------------------- | ----------------------------- | | GET | `/search` | search screen & audio content | | GET | `/health` | server health check | | GET | `/audio/list` | list audio devices | | GET | `/vision/list` | list monitors | | GET | `/frames/{id}` | get frame data | | GET | `/frames/{id}/ocr` | get frame OCR text | | POST | `/tags/{type}/{id}` | add tags | | DELETE | `/tags/{type}/{id}` | remove tags | | POST | `/raw_sql` | execute raw SQL | | POST | `/add` | add content to database | | GET | `/search/keyword` | keyword search | | POST | `/audio/start` | start audio recording | | POST | `/audio/stop` | stop audio recording | ## search example ```bash theme={null} curl "http://localhost:3030/search?q=meeting&limit=10&content_type=vision" ``` ## search parameters | param | type | description | | -------------- | -------- | --------------------------------- | | `q` | string | search query | | `limit` | int | max results | | `offset` | int | pagination offset | | `content_type` | string | `vision`, `audio`, `input`, `all` | | `start_time` | ISO 8601 | filter start | | `end_time` | ISO 8601 | filter end | | `app_name` | string | filter by app | | `window_name` | string | filter by window title | | `browser_url` | string | filter by browser URL | | `min_length` | int | minimum text length | | `max_length` | int | maximum text length | need help? [join our discord](https://discord.gg/screenpipe). # cline - VS Code agent with screen memory Source: https://docs.screenpi.pe/cline Connect screenpipe to Cline to give this autonomous VS Code agent access to your screen history and context. [Cline](https://github.com/cline/cline) is an autonomous AI coding agent for VS Code with 30k+ GitHub stars. it supports MCP servers, so you can connect screenpipe to give Cline context about what you've been working on across all your apps. ## setup 1. open VS Code with Cline installed 2. open Cline settings (gear icon in Cline panel) 3. go to **MCP Servers** 4. add screenpipe: ```json theme={null} { "mcpServers": { "screenpipe": { "command": "npx", "args": ["-y", "screenpipe-mcp"] } } } ``` 5. click the refresh icon to reload MCP servers ## usage once configured, Cline can search your screen history while coding: ``` > I was reading documentation about async/await patterns earlier, > find it and help me apply those patterns here > what error messages have I seen in my terminal today? > find the API response format I was looking at in the browser ``` ## plan mode with context Cline's "Plan" mode works great with screenpipe: 1. switch to Plan mode 2. ask Cline to find relevant context from your screen history 3. let it create a plan based on what you've been working on 4. switch to Act mode to execute ``` > [Plan mode] I was researching authentication patterns earlier, > find what I was looking at and plan how to implement it here ``` ## available tools screenpipe provides: * **search-content** - search screen OCR, audio transcriptions, input events ## requirements * screenpipe running on localhost:3030 * VS Code with Cline extension * Node.js >= 18.0.0 need help? [join our discord](https://discord.gg/screenpipe). # cloud archive - free disk space automatically Source: https://docs.screenpi.pe/cloud-archive Encrypt and archive old screenpipe data to the cloud to free disk space while keeping your history accessible. cloud archive encrypts your old screenpipe data and uploads it to the cloud, then deletes the local copy to free disk space. it uses the same zero-knowledge encryption as [cloud sync](/cloud-sync) — we cannot read your data. ## how it works choose how many days of data to keep locally (7, 14, 30, 60, or 90 days). data older than this will be archived. your token is used to derive an encryption key locally using argon2id. data is encrypted with chacha20-poly1305 before upload — the same encryption used by cloud sync. screenpipe uploads data in small batches (up to 500 records at a time) every 5 minutes. this includes screen captures, ocr text, audio transcriptions, accessibility data, and ui events. after each chunk is confirmed uploaded, the corresponding local data and media files are deleted to free disk space. ## what gets archived frame metadata, app names, window titles, browser urls extracted text from screen recordings transcribed speech with speaker and device info ui text captured via accessibility apis keyboard and mouse activity metadata orphaned video/audio files are cleaned up after upload ## enabling cloud archive 1. open **settings → cloud archive** 2. select your retention period (how many days to keep locally) 3. toggle **enable cloud archive** you can also trigger an immediate archive run by clicking **archive now** in the status card. ## encryption details cloud archive reuses the same `SyncManager` and encryption pipeline as cloud sync: * **key derivation**: argon2id with a password derived from your auth token * **cipher**: chacha20-poly1305 (authenticated encryption) * **zero-knowledge**: the encryption key never leaves your device if you already have cloud sync enabled, archive uses the exact same encryption key to avoid any conflicts. review the rust implementation of cloud archive ## watermark-based tracking instead of marking individual records as uploaded, cloud archive uses a single **watermark timestamp**. all data before the watermark has been securely uploaded and can be safely deleted locally. this is simpler and more efficient than per-record tracking — one timestamp tells the system exactly where it left off, even if the app restarts. ## storage & limits your cloud storage usage is shown in the archive status card. storage limits depend on your screenpipe pro plan. ## important notes archived data is currently upload-only. download/restore is not yet available — local data older than your retention period will be permanently deleted from your device after upload. cloud archive requires a screenpipe pro subscription. the archive process runs automatically in the background every 5 minutes when enabled. questions? [join our discord](https://discord.gg/screenpipe). # cloud sync - zero-knowledge encryption Source: https://docs.screenpi.pe/cloud-sync Learn how screenpipe cloud sync keeps your data encrypted and private with zero-knowledge encryption. cloud sync lets you access your screenpipe data across all your devices with end-to-end encryption. we use zero-knowledge encryption, meaning we physically cannot read your data. ## zero-knowledge encryption your data is encrypted on your device before it ever leaves. we use your password to derive an encryption key locally - this key never leaves your device and we never see it. this means we physically cannot read your data, even if we wanted to. only you (and anyone you share your password with) can decrypt your data. ## how it works this password is used to derive your encryption key using argon2id (a secure key derivation function). before any data leaves your device, it's encrypted using chacha20-poly1305 (a fast, secure cipher). only encrypted blobs are sent to our servers. we store them but cannot read them. when you access data from another device, it's downloaded and decrypted on that device using your password. ## what we store * encrypted blobs (we cannot read these) * your account email * encrypted master key (encrypted with your password - we cannot decrypt it) * salt for key derivation (not secret, needed to derive the same key) * device metadata (device name, os, last sync time) ## what we cannot see all visual data is encrypted all audio data is encrypted all extracted text is encrypted never leaves your device derived locally, never transmitted we only see encrypted blobs ## verify it yourself screenpipe is open source. you can audit the encryption code yourself: review the rust implementation of our zero-knowledge encryption ## important notes if you forget your encryption password, your cloud data cannot be recovered. we recommend using a password manager. cloud sync requires a screenpipe pro subscription. sync happens automatically in the background and uses approximately 0.8 gb per monitor per day. questions? [join our discord](https://discord.gg/screenpipe). # continue - open-source copilot with screen context Source: https://docs.screenpi.pe/continue Connect screenpipe to Continue to give this open-source AI coding assistant access to your screen history. [Continue](https://github.com/continuedev/continue) is an open-source AI coding assistant with 20k+ GitHub stars. it works in VS Code and JetBrains IDEs, and supports MCP servers for external context. ## setup ### VS Code 1. install Continue from the VS Code marketplace 2. open Continue settings 3. edit `~/.continue/config.json`: ```json theme={null} { "mcpServers": [ { "name": "screenpipe", "command": "npx", "args": ["-y", "screenpipe-mcp"] } ] } ``` 4. restart VS Code ### JetBrains 1. install Continue from JetBrains marketplace 2. open Continue settings 3. add the same MCP configuration 4. restart the IDE ## usage once configured, Continue can access your screen history: ``` > @screenpipe what was I working on this morning? > find the error I saw in my terminal and help me fix it > what documentation was I reading about react hooks? ``` ## context providers Continue also supports context providers. you can add screenpipe as a custom context provider for more control: ```json theme={null} { "contextProviders": [ { "name": "screenpipe", "params": { "apiUrl": "http://localhost:3030" } } ] } ``` ## example workflows **code with context:** ``` > I was looking at how another project handles authentication, > find that code and help me implement something similar ``` **recall errors:** ``` > there was a typescript error earlier about types, > find it and explain how to fix it ``` **meeting follow-up:** ``` > what did we discuss in the standup about the API changes? > help me implement those changes ``` ## available tools via MCP, Continue gets access to: * **search-content** - search screen OCR, audio transcriptions, and input events * **export-video** - create video exports from screen recordings ## requirements * screenpipe running on localhost:3030 * Continue extension in VS Code or JetBrains * Node.js >= 18.0.0 need help? [join our discord](https://discord.gg/screenpipe). # contribute to screenpipe - open source screen recording Source: https://docs.screenpi.pe/contributing Contribute to screenpipe: report bugs, submit pull requests, join bounty programs, and help build the open source screen recording platform. for detailed contribution guidelines, build instructions, and development setup, please see our [contributing guide on github](https://github.com/screenpipe/screenpipe/blob/main/CONTRIBUTING.md). ### quick links * [report a bug](https://github.com/screenpipe/screenpipe/issues/new?labels=bug) * [request a feature](https://github.com/screenpipe/screenpipe/issues/new?labels=enhancement) * [join our discord](https://discord.gg/screenpipe) * [schedule a call](https://cal.com/team/screenpipe/chat) # FAQ Source: https://docs.screenpi.pe/faq Frequently asked questions about screenpipe. pipes are scheduled AI agents that run on your screen data. each pipe is a `.md` file with a prompt and a schedule. an AI agent reads the prompt, queries the screenpipe API, and takes action. examples: sync to Obsidian, track time in Toggl, send daily summaries. see the [pipes guide](/pipes) for details. * **minimum**: dual-core CPU, 2GB RAM, 20GB disk * **recommended**: quad-core, 4GB+ RAM, 50GB+ SSD * runs 24/7 on a MacBook Pro M3 or a \$400 Windows laptop * \~600 MB RAM, \~10% CPU, \~30 GB/month storage at 1 FPS all data is in `~/.screenpipe/`: * `db.sqlite` — metadata, OCR text, transcriptions * `data/` — MP4 screen recordings, audio chunks * `pipes/` — installed pipes to backup, copy `~/.screenpipe/` to another location. * check [github issues](https://github.com/screenpipe/screenpipe/issues) * join [discord](https://discord.gg/screenpipe) * follow [contribution guidelines](https://github.com/screenpipe/screenpipe/blob/main/CONTRIBUTING.md) **OCR**: `apple-native` (macOS), `windows-native` (Windows), `tesseract` (Linux), `unstructured` (cloud), `custom` **STT**: `whisper-large-v3-turbo-quantized` (default, local), `whisper-tiny`, `whisper-tiny-quantized`, `whisper-large`, `whisper-large-quantized`, `whisper-large-v3-turbo`, `deepgram` (cloud) yes. everything runs locally. no data leaves your machine unless you explicitly choose a cloud provider (deepgram, unstructured). you control what's captured with `--ignored-windows` and `--included-windows`. three ways: 1. **MCP server** — works with Claude Desktop, Cursor ([guide](/mcp-server)) 2. **pipes** — scheduled AI agents ([guide](/pipes)) 3. **REST API** — `curl http://localhost:3030/search?q=your+query` # screenpipe for developers Source: https://docs.screenpi.pe/for-developers Give your AI coding assistant memory of your screen. Use screenpipe with Claude Code, Cursor, Cline, Continue, and Gemini CLI. AI coding assistant with screen context. screenpipe gives developers a superpower: AI that knows what you've been working on. it captures your screen and audio 24/7 and makes it available to coding assistants via MCP and REST API. ## what developers use screenpipe for * **code search across time** — find that code snippet you saw in a PR review last week, even if you closed the tab * **meeting recall** — search what was said in standups, design reviews, or pair programming sessions * **context for AI coding** — give Cursor, Claude Code, or Cline memory of what's on your screen right now and what you worked on earlier * **automated workflows** — pipes that auto-track time in Toggl, sync daily activity to Obsidian, or generate standup reports ## integrations screenpipe works with any AI tool that supports MCP or HTTP APIs: | tool | integration | guide | | --------------- | ---------------------------------------------------------------------------------------------------------- | --------------------------- | | **Claude Code** | MCP server — Claude Code can search your screen history, find code you saw earlier, recall meeting context | [setup guide](/claude-code) | | **Cursor** | MCP server — add screenpipe as a context source for Cursor's AI | [setup guide](/mcp-server) | | **Cline** | MCP server — give Cline access to your full screen history | [setup guide](/cline) | | **Continue** | MCP server — add screen context to Continue's AI completions | [setup guide](/continue) | | **Gemini CLI** | MCP server — use screenpipe with Google's Gemini CLI | [setup guide](/gemini-cli) | | **OpenCode** | MCP server — search screen history from OpenCode | [setup guide](/opencode) | | **Ollama** | local AI — use any local model with screenpipe, 100% private | [setup guide](/ollama) | ## quick start 1. [download screenpipe](https://screenpi.pe/onboarding) 2. add the MCP server to your coding tool: ```json theme={null} { "mcpServers": { "screenpipe": { "command": "npx", "args": ["-y", "screenpipe-mcp"] } } } ``` 3. ask your AI assistant: "what was I working on in the last hour?" or "find the code snippet I saw in that PR review" ## example prompts once screenpipe is connected to your coding assistant: * "find the error message I saw in the terminal 30 minutes ago" * "what did we discuss in the standup about the auth refactor?" * "show me the API endpoint I was looking at in the browser" * "summarize what I worked on today for my standup" ## automate with pipes [pipes](/pipes) are scheduled AI agents that run on your screen data. developers use them for: * **time tracking** — auto-log coding time to Toggl based on active apps * **daily journals** — sync screen activity to Obsidian * **standup reports** — generate daily summaries of what you worked on ## get screenpipe screenpipe includes everything developers need — MCP server, pipes, cloud AI, and more. lifetime plans available starting at \$400. [download screenpipe →](https://screenpi.pe/onboarding) # gemini CLI - google's AI with screen context Source: https://docs.screenpi.pe/gemini-cli Use screenpipe with Gemini CLI to give Google's AI access to your screen history and audio transcriptions. [Gemini CLI](https://github.com/google/gemini-cli) is Google's terminal-based AI coding assistant. it supports MCP servers, so you can connect screenpipe to give Gemini context about what you've been working on. ## setup Gemini CLI uses MCP for external tools. add screenpipe to your config: ```bash theme={null} # add screenpipe MCP server gemini config mcp add screenpipe "npx -y screenpipe-mcp" ``` or edit your config file directly: ```json theme={null} { "mcpServers": { "screenpipe": { "command": "npx", "args": ["-y", "screenpipe-mcp"] } } } ``` restart Gemini CLI after configuration. ## usage once configured, Gemini can search your screen history: ```bash theme={null} gemini > what was I working on this morning? > find the error message I saw in my terminal earlier > summarize the documentation I was reading about kubernetes ``` ## available tools screenpipe provides these MCP tools to Gemini: * **search-content** - search screen OCR, audio transcriptions, and input events * **export-video** - create video exports from screen recordings ## example workflows **context-aware coding:** ``` > I was looking at a react component earlier that handled > form validation, find it and help me implement something similar ``` **debug from memory:** ``` > there was an error in my build output, find it and help me fix it ``` **recall documentation:** ``` > what did that API documentation say about rate limits? ``` ## requirements * screenpipe running on localhost:3030 * Gemini CLI installed * Node.js >= 18.0.0 need help? [join our discord](https://discord.gg/screenpipe). # install screenpipe Source: https://docs.screenpi.pe/getting-started Install screenpipe on macOS, Windows, or Linux. Record everything on screen and search screen history on Mac, Windows, and Linux. Start recording in minutes. ## desktop app download the [desktop app](https://screenpi.pe/onboarding) and follow the installation instructions. works on macOS, Windows, and Linux. the app manages recording, settings, and search — no terminal needed. ## CLI ```bash theme={null} npx screenpipe@latest record ``` ## verify it's running once screenpipe starts, it serves an API on `localhost:3030`: ```bash theme={null} # check health curl http://localhost:3030/health # search your screen history curl "http://localhost:3030/search" ``` ## connect to AI screenpipe works with any AI that supports MCP or HTTP APIs: | integration | how | | ------------------ | ---------------------------------------------------------- | | **claude desktop** | add screenpipe as MCP server ([guide](/mcp-server)) | | **cursor** | add screenpipe MCP to your project ([guide](/mcp-server)) | | **claude code** | use screenpipe MCP or curl the API ([guide](/claude-code)) | | **ollama** | configure in app settings, use any local model | ## what's next? * [search your screen history](/search-screen-history) — find anything you've seen * [set up pipes](/pipes) — scheduled AI agents on your data * [API reference](/cli-reference) — REST API endpoints and parameters * [join our discord](https://discord.gg/screenpipe) — get help from the community ## get the desktop app screenpipe includes everything you need out of the box — 24/7 screen recording, AI-powered search, pipes, MCP server, and more. lifetime plans available starting at \$400. [download screenpipe →](https://screenpi.pe/onboarding) # screenpipe — your screen's memory layer Source: https://docs.screenpi.pe/home screenpipe captures your screen 24/7, extracts text with OCR, and lets you search everything with AI. open source rewind alternative. local screen recorder with AI. local-first, privacy-focused. screenpipe records your screen continuously and makes everything searchable. it runs locally, stores data on your machine, and provides APIs for AI integration. ## what it does * **captures screen 24/7** — OCR extracts text from every frame * **local storage** — sqlite database + media files on your machine * **search API** — find anything you've seen via `localhost:3030/search` * **pipes** — scheduled AI agents that act on your screen data ([learn more](/pipes)) * **MCP server** — connect to Claude, Cursor, and other AI assistants ([learn more](/mcp-server)) ## quick start [download the desktop app](https://screenpi.pe/onboarding) — works on macOS, Windows, and Linux ## what's next? install screenpipe on macOS, Windows, or Linux find anything you've seen on your screen scheduled AI agents on your screen data connect screenpipe to AI assistants ## community need help? have ideas? [join our discord](https://discord.gg/screenpipe) — 10k+ members building with screenpipe. # MCP apps - create interactive UIs for screenpipe Source: https://docs.screenpi.pe/mcp-apps Build interactive UI components that render in Claude, ChatGPT, VS Code, and other AI assistants using the MCP Apps protocol. MCP Apps let you create interactive UIs that render directly inside AI chat interfaces. Instead of plain text responses, your tools can return dashboards, forms, visualizations, and more. ## how it works ``` ┌─────────────────────────────────────────────────────────────┐ │ 1. Tool declares a UI resource │ │ { _meta: { ui: { resourceUri: "ui://search" } } } │ │ │ │ 2. Host fetches the HTML from the MCP server │ │ GET ui://search → Returns HTML/JS bundle │ │ │ │ 3. Host renders in sandboxed iframe │ │