plugins (pipes)
an essential part of screenpipe is that it is extensible, providing powerful extensibility through custom workflows that interact with captured screen and audio data. whether you need to tag activities, generate summaries, or send data to third-party services, plugins allow you to tailor screenpipe to your specific needs.
plugins in screenpipe are categorized into 2 types:
- pipes: these are plugins with native integration with screenpipe. they can run within the cli/lib/app without the need for manual execution. they run in a sandboxed environment, and can be written in Typescript or Javascript.
- standalone scripts: these are independent scripts that need to be run separately and may require additional setup.
1. pipes
overview of pipes
pipes in screenpipe are plugins designed to process data captured from your screen and audio feeds. they automate workflows such as tagging, summarizing, and integrating data with external platforms like notion, salesforce, or slack.
pipe workflow examples:
- pipe example: screenpipe | ai tag | notion update
- pipe example: screenpipe | ai tag | slack report
- pipe example: screenpipe | log daily activities
key features of pipes:
- custom workflows: automate repetitive tasks such as tagging screen activity or summarizing audio transcriptions.
- real-time processing: pipes can process captured data in real-time.
- integration capabilities: extend the functionality by sending processed data to third-party apps.
example use cases:
- ai activity tagging: automatically tag screen or audio activities based on ai models.
- meeting summaries: generate summaries of meeting discussions using combined screen and audio data.
- crm integration: automatically log interactions or captured data into a crm like salesforce.
available pipes
screenpipe provides a variety of pipes and standalone scripts that allow users to extend the platform's functionality with minimal effort. below is a table summarizing the available examples:
example | description | link |
---|---|---|
pipe phi3.5 engineering team logs | generate engineering team work logs | link (opens in a new tab) |
pipe post questions on reddit | post questions on reddit based on screen activity | link (opens in a new tab) |
pipe email daily log | send daily activity logs via email | link (opens in a new tab) |
pipe llama32 comment linear while you work | comment on linear tasks while you work using llama32 | link (opens in a new tab) |
pipe meeting summary by email | send meeting summaries via email | link (opens in a new tab) |
quick tour - developing pipes in screenpipe
screenpipe's plugin system, referred to as pipes, allows you to create customized pipes that automate workflows. these pipes can process, annotate, analyze, summarize, or integrate your screen and audio data with other tools.
users can begin by cloning an existing pipe, such as pipe-phi3.5-engineering-team-logs (opens in a new tab). you can use any javascript/typescript to build your use case, from simple to highly advanced. the primary requirement is that the pipe folder must contain a file named pipe.js
or pipe.ts
.
git clone https://github.com/mediar-ai/screenpipe.git
cp -r screenpipe/examples/typescript/pipe-phi3.5-engineering-team-logs screenpipe/examples/typescript/my-pipe
we use Bun to run the pipe within the Rust codebase.
check official bun docs (opens in a new tab) for more details on installing and running Bun.
note: you can also use nextjs to build your pipe, we will render the page in the pipe store. check pipe-simple-nextjs (opens in a new tab) for an example.
configuring pipes
we use pipe.json
to let others pass configuration options to your pipe through the UI or CLI.
we parse your pipe.json
file to render in the UI the input fields for your pipe so you need to follow the format.
here's a snippet of a pipe.json
for the pipe-post-questions-on-reddit:
{
"fields": [
{
"default": 60,
"description": "Interval in seconds to read your screen data and extract structured logs (will be used to summarize and send an email). Increase this value if using audio.",
"name": "interval",
"type": "number"
},
{
"default": "daily",
"description": "Frequency of summary emails: 'daily' for once a day at emailTime, or 'hourly:X' for every X hours (e.g., 'hourly:4' for every 4 hours)",
"name": "summaryFrequency",
"type": "string",
"value": "hourly:1"
},
]
}
running pipes
you can run your pipe manually with bun, for example:
# these are mandatory env variables
export SCREENPIPE_DIR="$HOME/.screenpipe"
export PIPE_ID="pipe-phi3.5-engineering-team-logs"
export PIPE_FILE="pipe.ts"
export PIPE_DIR="$SCREENPIPE_DIR/pipes/pipe-phi3.5-engineering-team-logs"
bun run examples/typescript/pipe-phi3.5-engineering-team-logs/pipe.ts
make sure to configure your pipe.json first
once the pipe is ready, follow these steps to make it work with screenpipe
:
-
download the pipe (from local or github url):
screenpipe pipe download path/to/pipe-name
-
enable the pipe (enabling will activate the pipe, similar to installing it in screenpipe):
screenpipe pipe enable pipe-name
-
run the pipe (restart screenpipe):
screenpipe
you can also use the UI to download and enable your pipe (add the path or URL, click the button, enable, config, restart screenpipe).
screenpipe-js SDK
cool stuff:
pipe.inbox.send
to send AI messages to the user and ask for confirmation before any dangerous actions.pipe.sendDesktopNotification
to send a desktop notification to the user.pipe.queryScreenpipe
to query the screenpipe data.
etc.
checkout the special api for interacting with screenpipe. (opens in a new tab)
closing thoughts
think of pipes like a local Zapier which costs 10x less, 10x less friction because you don't need to authenticate etc. and with the full context of your screen and audio data.
please send a PR with your pipe to the repo, we will review and merge it!
we also do bounties for new pipes, or if you talk about screenpipe on social media, we give $100+!
we're working on integration with Stripe, to allow you to monetize your pipes, feel free to reach out for early access.
outro
- we use bun to run the pipe within the Rust codebase.
- you can use
pipe.json
to let others pass configuration options to your pipe through the UI or CLI. - pipes have access to a special API that allows you to interact with your computer or special features like sending notifications, sending emails, etc.
- check this file (opens in a new tab) for more details on impl.
- in general we recommend using local LLMs like ollama as the data can be large and sensitive and JSON outputs.
- we recommend using vercel ai sdk (opens in a new tab) for faster dev.
known limitations
- you can only use JS/TS to build your pipes.
- you usually need to run the pipe in an infinite loop for example if it's something that process your data continuously, of course if this is for example an agent that summarize your day one shot it does not have to be infinite loop.