Vellum Ambient is a voice assistant that floats over any Mac app, looks at the screen you choose to share and draws on it to show you where to click. Vellum switched it on for every macOS user of its personal AI assistant on 24 September 2026. “Ambient is live today for all Vellum users on macOS,” the company posted on X at 17:00 UTC. TestingCatalog covered the release 40 minutes later under the name Ambient Assistants.
The pitch is that you should not have to describe your screen to an assistant that could simply look at it. “Your AI assistant had a blind spot. We fixed that,” Vellum wrote. Instead of screenshots, pasted links and long prompts, Vellum Ambient takes its context from whatever is open. You double-press the Fn key to call it up and talk to it. It then answers, points at the right control, or goes off and does the work in the background. Holding Fn turns Vellum Ambient into a dictation tool that types into any text field.
This article explains how Vellum Ambient works, using Vellum’s product page, its documentation and the open-source code behind it. It covers the Fn key controls, the skill that draws arrows and rings, privacy, pricing, and how the feature compares with agents that take over your mouse. One of those is Google’s Gemini Task mode, which reached trusted testers the same day.
Table of contents
- What Vellum Ambient Does: The Four Launch Features
- How the Fn Key Controls Vellum Ambient
- Pointing, Not Clicking: How Vellum Ambient Draws on Your Screen
- From Roadmap Idea to Launch: The Vellum Ambient Timeline
- Who Vellum Ambient Is For: Small and One-Person Businesses
- Vellum Ambient Privacy: What It Sees and Where It Goes
- Vellum Ambient Pricing and System Requirements
- How Vellum Ambient Compares With Other Screen-Aware Assistants
- What to Check Before You Rely on Vellum Ambient
- Vellum Ambient FAQ
- References and Further Reading
What Vellum Ambient Does: The Four Launch Features
Vellum’s product page describes Ambient as “an ultra-fast voice agent that floats on your desktop, sees what you see, and gets work done in the background.” It organises the launch around four numbered features. Each is a different way of using the same floating assistant, and each comes with its own example.
On-screen guidance
The first feature is guidance. “Ask how to do something, and your assistant points out where to click,” the page says. Its examples are blurring a face in CapCut and matching a payment to an invoice in QuickBooks. Vellum Ambient does not click for you here. It marks the control on your screen and talks you through it, so you do the work yourself and learn where things are.
Show what you mean
The second feature runs the other way. You talk and annotate your own screen to show Vellum Ambient what you want. Vellum’s example is circling an advert you like and saying “This Ad creative works best, generate 20 and push to Meta.” The circle replaces a long description of which creative you mean. That is the kind of context that usually takes a screenshot and a paragraph of explanation.
Delegate while you keep working
The third feature is delegation. “Double-press Fn to delegate while you keep working,” the page says. Your assistant “can research competitors or book a flight, checking in when it’s done or needs approval.” The demo shows a small notice reading “Need approval to book the flight” with an Approve button. The work runs in the background while you stay in the app you were using.
Speak to write
The fourth feature is dictation. “Hold Fn to write or edit with your voice, right where you’re typing,” Vellum says. You can dictate a customer reply, or select some text and ask Vellum Ambient to “tighten this up a bit.” In the product demo, the assistant answers “Just did!” once the edit lands.
| Feature | How you start it | Vellum’s example | Who does the work |
|---|---|---|---|
| On-screen guidance | Double-press Fn and ask | Blur a face in CapCut; match a payment in QuickBooks | You, following the marks |
| Show what you mean | Talk while annotating your screen | Circle an advert, ask for 20 more pushed to Meta | The assistant, from your marks |
| Delegate tasks | Double-press Fn | Research competitors; book a flight | The assistant, in the background |
| Speak to write | Hold Fn | Dictate a reply; rephrase selected text | The assistant, typing at your cursor |
How the Fn Key Controls Vellum Ambient
Vellum Ambient hangs off one key. Vellum’s launch post lists two gestures: “Double-tap Fn to summon it” and “Hold Fn to turn speech into text on any page.” On a MacBook, Fn sits in the bottom-left corner of the keyboard, so it is always within reach of your left hand.
Double-press to summon
A double press opens the floating assistant, which TestingCatalog describes as “a floating pill on top of any Mac app.” From there you can ask a question, share a window or hand off a task. Because the pill floats above the app you are using, you do not switch windows to reach Vellum Ambient, and it can see what you were looking at when you called it.
Hold to dictate
Holding Fn puts Vellum Ambient into dictation mode. You speak, and the words appear in whatever text field has the cursor, “for replies in any app,” as TestingCatalog put it. Standalone dictation apps have already taught many Mac users to hold Fn and talk. Vellum’s code names one of them directly: Wispr Flow is listed as a known “Fn claimant”. With both running, one hold of Fn “dictates twice”, so Vellum watches for Wispr Flow and can ask it to quit.
Three modes behind one key
The open-source code shows the hold gesture does more than type. Vellum’s dictation handler sorts each request into one of three modes. If text is selected, it is a command, and your words are read as an instruction to change that text. If the transcript starts with an action verb, it is an action. Otherwise it is plain dictation, cleaned up for typing. A language model makes the call, with a fallback list of 13 action verbs such as “email”, “slack”, “schedule” and “remind”.
| What you do | Mode | What happens |
|---|---|---|
| Hold Fn with the cursor in a text field | Dictation | Speech is cleaned up and typed at the cursor |
| Hold Fn with text selected | Command | Speech is read as an instruction to rewrite the selection |
| Hold Fn and start with a verb such as “email” | Action | The request is passed on as a task |
| Double-press Fn | Assistant | The floating pill opens for questions, sharing and delegation |
Pointing, Not Clicking: How Vellum Ambient Draws on Your Screen
The most unusual part of Vellum Ambient is that it can point. Most screen-aware agents are built to act: they move the pointer, click buttons and type into fields. Vellum’s annotation feature is built to show. It draws an arrow or a ring on the screen you are sharing, adds a short caption and leaves the clicking to you.
The screen-annotation skill
The drawing is done by a bundled skill called Screen Annotation, which sits in Vellum’s public GitHub repository. Its instructions are blunt about the purpose. “This is the opposite errand from computer use,” they say. “Nothing here clicks, types or drives anything.” A mark “is drawn clear of what it indicates and never takes the mouse, so what you point at stays visible and clickable the whole time.” The skill has just two tools, one to point and one to clear the marks.
Arrows by name, rings by coordinates
There are two ways to point. The preferred one is by name. Vellum Ambient asks the app’s accessibility layer for a control called, say, “color balance” and draws an arrow at it. This is the same information screen readers use, so the arrow lands where the control really is. The fallback is a ring, measured from the latest picture of the shared screen as fractions of its width and height. A change merged on 23 September lets the assistant draw a ring when a name lookup fails.
One step at a time
The skill reads like a teaching guide. “A mark is where to look next. A screen with four marks on it is not four times as helpful; it is a diagram,” it says. The tool accepts up to four marks, but the instructions push for one. Captions are capped at 80 characters and should be instructions such as “Click Share”, with everything else said out loud. For a task with several steps, Vellum Ambient should name the route first, point at step one and then wait.
How it knows you clicked
Vellum Ambient does not move on after a set time. When you click the control an arrow is pointing at, a message tells the assistant which control you clicked, and that mark comes down on its own. For a ring, or a click somewhere else, the assistant relies on a fresh picture of your screen and on what you say. If you switch the share to a different window or screen, marks measured against the old one are refused rather than drawn in the wrong place.
| Mark | How the target is found | When it is used | Limit |
|---|---|---|---|
| Arrow | The control’s accessibility name, such as “Send” | Pointing at a named button or field | Name up to 120 characters |
| Ring | Bounds measured from a fresh screen image | Regions, a requested circle, or a failed name lookup | Needs an up-to-date view |
| Caption | Written by the assistant | Beside every mark | 80 characters |
| Clear | Automatic, or on request | When a step is done or the share moves | Up to four marks at once |
From Roadmap Idea to Launch: The Vellum Ambient Timeline
Vellum Ambient did not arrive from nowhere. The idea has been in Vellum’s plans, and in its code, for most of the year. The public repository and the company’s own roadmap let you trace it almost week by week.
Listed as an idea in April
Vellum’s public roadmap, last updated on 22 April 2026, put four relevant items under “Exploring”, its label for ideas with no committed date. “Ambient context” promised that your assistant “can see what’s on your screen when you ask it to — so ‘this’ and ‘that’ actually mean something.” “Use Vellum in any app” and “Dynamic visual presence” described a floating panel inside any app. “Voice activation” described a wake word. Real-time voice sat one step closer, “On the horizon”. Five months later, most of that list shipped as Vellum Ambient.
| Roadmap item (22 April) | April status | What shipped on 24 September |
|---|---|---|
| Ambient context | Exploring | Screen sharing with live, on-screen guidance |
| Use Vellum in any app | Exploring | A floating pill over any Mac app |
| Dynamic visual presence | Exploring | The pill, plus arrows and rings on screen |
| Real-time voice | On the horizon | Voice conversation, built on July’s voice mode |
| Voice activation (wake word) | Exploring | Not shipped; a double-press of Fn instead |
Built in the open
The word “ambient” goes back further still. On 9 February 2026, two days after the repository was created, a change titled “Add ambient screen-watching agent” landed in Vellum’s original Swift app for the Mac. That app was retired in June, when Vellum replaced it with a new macOS client. In today’s code, Vellum Ambient is called the “companion”. Its floating surface first appeared on 6 August, and the pointing skill followed on 7 September.
The repository is MIT-licensed and had 1,313 stars and 196 forks on 25 September. Release 0.12.4 was built on 23 September, the day before the launch post. Anyone can read exactly how Vellum Ambient decides what to draw, which is unusual for a consumer assistant.
The voice work underneath
Vellum Ambient leans on voice mode, which Vellum rolled out in late July. TestingCatalog reported that it transcribes speech with Deepgram and speaks back through ElevenLabs. In a 24 July blog post, Vellum’s Alex Nork explained why the company kept a three-stage pipeline rather than a single speech-to-speech model: “You can’t put an agent in the middle of it, which means no tools, no memory, no skills, no approvals.” A fast “front” model answers simple turns and passes hard ones to a stronger model, covering the gap with a short spoken phrase.
Who Vellum Ambient Is For: Small and One-Person Businesses
Vellum aims Ambient squarely at people who run everything themselves. TestingCatalog’s thread said Vellum supports Ambient Assistants “for small and one-person businesses, where a single person manages advertisements, invoices, customer support, and the website.” That framing shapes every example the company chose.
Confusing dashboards, step by step
The main use is getting through software you do not open every day. TestingCatalog described “an ad dashboard with a confusing design”, with Vellum Ambient guiding users “step by step through the settings in QuickBooks or Shopify.” For a sole trader who visits the ad manager once a month, being shown the right toggle is often more useful than having an agent change it out of sight. It also means they know where the setting is next time.
Teaching a routine once
The launch thread also says users “can show a routine on screen and talk it through, so the assistant can run it later after its plan is approved.” The code has a matching “Teach” button, which opens a picker for what the session may read: a whole display, one window or a single Chrome tab. When you pick a window, a frame follows it around the screen. This is close to workflow automation by demonstration, with a human sign-off before anything runs.
Delegation with a check-in
Delegation suits the jobs a small business owner keeps putting off. Vellum’s examples are researching competitors and booking a flight. Vellum Ambient runs these in the background and comes back “when it’s done or needs approval”. Anything involving money or outgoing messages should hit that approval step. Vellum’s own documentation says that when its assistant drafts a message, it types it and “stops before pressing Enter”, sending only when asked.
Vellum Ambient Privacy: What It Sees and Where It Goes
An assistant that can see your screen raises obvious questions. Vellum’s answers come in three parts: what Vellum Ambient can capture, where that data is sent, and what needs your approval.
It sees what you share, when you share it
Vellum Ambient needs macOS’s Screen Recording permission to see anything, and Vellum asks for it the first time the assistant needs your screen. TestingCatalog reported that “no recordings are made in the background.” That matches Vellum’s older Screen Watch skill, which its documentation describes as “not always-on background monitoring — it’s explicitly triggered and time-bounded.” The sharing picker lets you narrow the view to one window or one tab rather than the whole display.
Screen images go to a cloud model
What you share is not processed only on your Mac. Vellum’s computer use documentation is plain about it: “Screenshots are sent to the AI model.” By default that model is Anthropic’s Claude, which Vellum says runs under “a zero-retention data policy”. Users who bring their own API key can switch to OpenAI, Google, OpenRouter or Fireworks, or point the assistant at a local Ollama model. If you show Vellum Ambient a customer record or a payroll screen, assume the image leaves the machine.
Permissions are checked by software, not the model
Vellum’s permission system sorts every tool into low, medium or high risk. At the default setting, low-risk actions run on their own and everything else asks first. Vellum says each check is “enforced by traditional software, not judged by the AI”, and that there is “no way to prompt-inject past a permission boundary.” The pointing tools are rated low risk because they cannot change anything. Computer use actions are rated medium or high risk and prompt for approval one at a time. Passwords and API keys sit in a separate service the model never sees.
| macOS permission | What it unlocks | When Vellum asks |
|---|---|---|
| Screen Recording | Seeing your screen | The first time the assistant needs to see it |
| Microphone | Voice input (hold Fn to talk) | The first time you use voice |
| Speech Recognition | Turning voice into text | The first time you use voice |
| Accessibility | Clicking, typing and finding controls by name | The first time it needs to use an app |
| Notifications | Status updates and reminders | Optional, on the first notification |
Sensible habits for a small firm
The practical steps are simple. Share one window, not the whole desktop. Close anything confidential before you double-press Fn. Keep the risk setting at Default or stricter. Then treat Vellum Ambient as you would any cloud service that handles client data under your cybersecurity policy, with a named owner and a short written rule on what staff may share.
Vellum Ambient Pricing and System Requirements
Vellum has not announced a separate price for Ambient. The launch post says it is live “for all Vellum users on macOS”, and on the published plans that includes the free tier. What you pay depends on your plan and on how much model time the assistant uses.
The plans
Vellum’s Base plan is free. It includes a small cloud machine with 1 vCPU, 2 GiB of memory and 6 GiB of storage, and you pay for usage with prepaid credits. Three Pro packages bundle a bigger machine, more storage and a monthly usage allowance: Mighty at $30 a month, Super at $100 and Ultra at $200. A Custom plan starts at $50. Vellum says it passes model costs through “at cost”, with no margin on tokens.
Credits and spending limits
One Vellum credit equals one US dollar, bought in $10 steps up to $100 at a time. Credits pay for model inference, web search, image generation and some paid third-party APIs, and they expire 12 months after purchase. A voice assistant that reads screenshots will use more inference than a text chat. So set the daily credit limit and the low-balance alert in the billing settings before you give Vellum Ambient to staff.
| Plan | Price a month | Machine | Storage | Notes |
|---|---|---|---|---|
| Base | Free | Small (1 vCPU, 2 GiB) | 6 GiB | Pay-as-you-go credits |
| Mighty | $30 | Small (1 vCPU, 2 GiB) | 10 GiB | Monthly usage; no priority support |
| Super | $100 | Medium (2.5 vCPU, 5 GiB) | 30 GiB | Subdomain, static IP, priority support |
| Ultra | $200 | Large (4 vCPU, 8 GiB) | 60 GiB | Largest preset package |
| Custom | From $50 | Medium or larger | From 10 GiB | Choose each component |
What your Mac needs
The Mac app needs macOS 15 Sequoia or later, on Apple Silicon or Intel, with about 500 MB of free space. Vellum also has a Windows app, but its documentation calls the current download a “dev build” that connects to a development environment, and the launch mentions macOS only. There is no Linux desktop app. The web, iPhone and Android apps reach the same cloud assistant, but the screen features of Vellum Ambient need the Mac.
How Vellum Ambient Compares With Other Screen-Aware Assistants
Screen awareness is becoming a standard feature of desktop assistants. The differences lie in what the assistant does with what it sees. The main choice behind Vellum Ambient is to point by default and act only when asked.
Guidance versus computer use
Most rivals start from computer use, where the agent drives the mouse and keyboard itself. Google’s Gemini Task mode, now with trusted testers, is built that way. Long, fully automatic runs are still hard for these agents. In our piece on Rabbit’s new desktop agent, Snorkel AI’s OSWorld 2.0 analysis found leading models completing only a minority of long, real-world workflows outright. Guidance sidesteps that risk. If the arrow is wrong, you simply do not click.
One assistant, several ways to see
Vellum Ambient also sits beside Vellum’s own older tools. The Computer Use skill reads the accessibility tree, takes a screenshot after each action and stops a session at 50 steps. Screen Watch reads your screen with text recognition every 5 to 30 seconds, for up to 15 minutes. Ambient adds the live, spoken layer on top. Like Anthropic’s Cowork browser for Claude Desktop, it gives an existing assistant a new surface rather than launching a separate product.
| Tool | How it sees | What it does | Who acts |
|---|---|---|---|
| Vellum Ambient, pointing | A shared display, window or Chrome tab | Draws arrows and rings with captions | You |
| Vellum Ambient, delegation | The screen plus your spoken request | Runs a task in the background, asks for approval | The assistant |
| Vellum Computer Use skill | Accessibility tree plus a screenshot after each action | Clicks, types and scrolls, up to 50 steps | The assistant, with approvals |
| Vellum Screen Watch skill | Text recognition every 5 to 30 seconds | Comments on your work for up to 15 minutes | You |
| Gemini Task mode (testing) | The desktop | Operates apps from an allowed list | The assistant |
Why the pointer matters for trust
Pointing also changes who is accountable. When an agent clicks, a mistake happens before you see it. When Vellum Ambient points, you see the mistake before it happens. For tasks with real consequences, such as a refund in QuickBooks or a budget change in an ad account, that gap is the difference between an error you catch and one you find next week.
What to Check Before You Rely on Vellum Ambient
Vellum Ambient is one day old, and a few cautions apply before a business builds habits around it.
Most evidence so far comes from Vellum
At the time of writing, the public material is Vellum’s own launch post, product page and demo video, plus TestingCatalog’s thread. We found no independent hands-on review. TestingCatalog’s earlier Vellum articles, on voice mode in July and the mobile apps in August, are labelled “Sponsored” on its site. The launch thread carries no such label, but it reads like a product summary rather than a test.
Pointing depends on how apps are built
Arrows rely on accessibility names, and not every app exposes good ones. Vellum’s own skill warns that some apps use internal names such as “ToolbarExportControl” instead of the visible label. Web apps, custom-drawn interfaces and canvas-heavy tools like video editors may have few named controls at all. There, Vellum Ambient falls back to rings measured from a screenshot, which are less exact.
Mac first, cloud always
The feature is Mac-only at launch, and it needs a network connection because the thinking happens in a cloud model. Teams on Windows or Linux will have to wait, and firms with strict data rules should read the privacy notice before anyone shares a screen. If you are weighing Vellum Ambient against other assistants for your staff, an AI strategy review that sets rules on screen sharing is a sensible first step.
Vellum Ambient FAQ
Is Vellum Ambient free?
Vellum says Ambient is “live today for all Vellum users on macOS”, and its Base plan is free. The assistant still spends credits on model use, so heavy use costs money on any plan. Pro packages run from $30 to $200 a month.
Does Vellum Ambient record my screen all the time?
No, according to TestingCatalog, which reported that “no recordings are made in the background.” The assistant sees what you share, when you share it, and macOS asks for Screen Recording permission first. What you share is sent to a cloud AI model for processing.
Can Vellum Ambient click buttons for me?
It can, through delegation and Vellum’s computer use tools, with an approval prompt for each action at the default settings. Its pointing mode deliberately does not click. It draws an arrow or a ring and waits for you to act.
How do I open Vellum Ambient?
Double-press the Fn key to open the floating assistant. Hold Fn to dictate into any text field, or to rewrite selected text with your voice.
Does Vellum Ambient work on Windows?
Not yet. The launch covers macOS 15 Sequoia and later. Vellum’s Windows app is currently a development build, and there is no Linux desktop app.
References and Further Reading
Introducing Ambient: your on-screen voice collaborator (Vellum)
Meet Ambient, launch post (Vellum on X)
Vellum has launched Ambient Assistants (TestingCatalog on X)
Screen Annotation skill (vellum-assistant on GitHub)
vellum-assistant open-source repository (GitHub)
The Permissions Model (Vellum Docs)
Privacy and Data (Vellum Docs)
Computer Use skill (Vellum Docs)
Installation and system requirements (Vellum Docs)
How We Gave Your Assistant a Voice (Vellum)
Vellum adds Voice Mode to personal AI assistant (TestingCatalog)
More AI coverage: explore Progressive Robot's AI Models, Tools & Releases hub — hands-on reviews, setup guides and benchmarks in one place.
