overview
screenpipe is a Rust application that continuously captures your screen and audio, processes them locally, and stores everything in a SQLite database. it serves a REST API onlocalhost:3030 for querying your data.
data flow
crates
screenpipe is a Rust workspace with specialized crates:layers
1. capture
| what | how | crate |
|---|---|---|
| screen | per-monitor frame capture at configurable FPS (default 1.0, macOS 0.5) | screenpipe-vision |
| audio | multiple input/output devices in configurable chunks (default 30s) | screenpipe-audio |
| UI events | keyboard, mouse, app switches, clipboard via accessibility APIs | screenpipe-accessibility |
2. processing
| engine | type | platform |
|---|---|---|
| Apple Vision | OCR | macOS |
| Windows native | OCR | Windows |
| Tesseract | OCR | Linux |
| Whisper | speech-to-text | local, all platforms |
| Deepgram | speech-to-text | cloud API |
3. storage
all data stays local on your machine:- SQLite at
~/.screenpipe/db.sqlite— metadata, OCR text, transcriptions, speakers, tags - media at
~/.screenpipe/data/— MP4 screen recordings, audio chunks
4. API
REST API onlocalhost:3030:
| endpoint | description |
|---|---|
/search | filtered content retrieval (OCR, audio, UI events) |
/search/keyword | keyword search with text positions |
/frames/{id} | access captured frames |
/health | system status and metrics |
/raw_sql | direct database queries |
/ai/chat/completions | Apple Intelligence (macOS 26+) |
5. pipes
pipes are AI agents (.md prompt files) that run on your screen data. they’re executed by an AI agent that reads the prompt, queries the screenpipe API, and takes action.
pipes live in ~/.screenpipe/pipes/{name}/ and run on cron-like schedules.
6. desktop app
the desktop app is built with Tauri (Rust backend) + Next.js (React frontend):database schema
key tables:| table | stores |
|---|---|
frames | captured screen frame metadata |
ocr_text | text extracted from frames |
video_chunks | video recording file metadata |
audio_chunks | audio recording metadata |
audio_transcriptions | text from audio |
speakers | identified speakers |
ui_events | keyboard, mouse, clipboard events |
tags | user-applied tags on content |
resource usage
runs 24/7 on a MacBook Pro M3 (32 GB) or a $400 Windows laptop:| metric | typical value |
|---|---|
| RAM | ~600 MB |
| CPU | ~10% |
| storage | ~30 GB/month at 1 FPS |
source code
| component | path |
|---|---|
| API server | screenpipe-server/src/ |
| screen capture | screenpipe-vision/src/core.rs |
| audio capture | screenpipe-audio/src/ |
| database | screenpipe-db/src/db.rs |
| pipes | screenpipe-core/src/pipes/ |
| MCP server | screenpipe-mcp/src/index.ts |
| desktop app | screenpipe-app-tauri/ |