Umbra 0.8.0 by Black Rainbow Labs. Free and open source.

The open source coding agentfor any model, graded on 61 tasks

Name: Umbra
Author: Black Rainbow Labs

Each change to Umbra’s prompt and loop is run against 61 graded coding tasks, and a change that doesn’t help is taken out. Umbra 0.8.0 passed 91.3% of 183 runs. It reads the instruction files, skills and MCP servers you already have, and works with Claude, GPT or a model on your own machine.

curl -fsSL https://raw.githubusercontent.com/Black-Rainbow-Labs/umbra/main/install.sh | shpowershell -ExecutionPolicy ByPass -c "irm https://raw.githubusercontent.com/Black-Rainbow-Labs/umbra/main/install.ps1 | iex"

See the evals

Umbra on Claude Haiku 5.5 in a terminal: it ran the tests, changed percent // 100 to percent / 100 in cart.py, ran the suite again with all 4 tests passing, and replied with a table of order totals before and after the fix.

Umbra 0.8.0, 61 graded tasks in 9 categories, 3 runs each, 14 November 2026. Method and every result

61tasks
in 9 categories, each with a hidden check
91.3%
of 183 runs passed
95.8%
of prompt tokens served from the cache
382ms
to the first word of a reply
3.9MB
the Windows download, one program, nothing else to install

Evals

Each change to the loopis measured first

A coding agent is a prompt, a loop and some defaults, and a small change to any of them can make it worse without anyone noticing. We keep 61 graded coding tasks and run Umbra against them for each of those changes. A change stays only when the numbers improve.

What’s in it.

61 tasks in 9 categories: bug fixes, new files, precise edits, finding code, following instructions, large output, sessions, tasks that fill the context window, and a hard set. Each task has a hidden check.

What’s measured.

About 30 things for each run: whether it passed, time, tokens, cache use, requests, and what happens when a session is resumed or compacted.

What happened to each idea.

The ledger in the repository lists every experiment, its result, and whether it was kept or taken out. Saving long command output to a file raised the prompt tokens for each passed task by 32%, so it was taken out. Read the ledger

Hard set20

Context fills9

Sessions7

Bug fixes6

Precise edits5

Finding code4

New files4

Large output3

Instructions3

Each square is one run of Umbra 0.8.0 in run final-8df86c3. Tap, hover or focus a square for the task.

91.3%of runs passed
69.4 sfor each task
95.8%of prompt tokens from the cache

Results forthis release

Umbra 0.8.0, 61 graded tasks in 9 categories, 3 runs each, 14 November 2026.
Runs passed	91.3%
Mean score of the hidden checks	98.7%
Time for each task	69.4 s
Time to the first reply text	382 ms
Prompt tokens served from the cache	95.8%
New prompt tokens for each passed task	14,694
Long session runs passed	92.6%
Start time when resuming a long session	36 ms

Runs passed by category
Category	Tasks	Passed
Hard set	20	78.3%
Context fills	9	92.6%
Sessions	7	100%
Bug fixes	6	100%
Precise edits	5	93.3%
Finding code	4	100%
New files	4	100%
Large output	3	100%
Instructions	3	100%

The 0.5.2 and 0.6.2 baselines ran 33 tasks once, so they aren’t shown next to this result.

Tested on local models,on purpose

We run the evals on a 27B model on one PC. A smaller model makes more mistakes, so a weak prompt or a wasteful loop shows up in the pass rate straight away, and we can run every task three times at no cost.

We expect what the harness gains there to carry forward to frontier models, because the gains come from sending less, caching more and handling long sessions better. We haven’t measured that yet.

The model: ninfer-qwen3.8-27b-coletti-v9, served on one RTX 5090 with 32 GB of memory.

See every task and result

One reply,seven layers

This is one real frame of Umbra 0.8.0. Scroll to turn it around and take it apart, then each part is explained below.

Your setup
Commands
Tables
Math and diagrams
Code
Status

The fullscreen view

Tables, math and diagrams,in your terminal

A model’s reply is Markdown, and most terminals show it as raw text. Umbra draws the tables, colors the code, lays out the math and turns a Mermaid block into a diagram. Commands the agent ran fold into one line you can open.

Math.
Inline and block LaTeX, drawn with Unicode: fractions, roots, sums and matrices.
Diagrams.
Mermaid flowcharts, sequence diagrams and pie charts, drawn with box characters.
Code.
Syntax colors for about 25 languages, and colored diffs.
Command groups.
Several commands fold into one row with the time and the result. Open it to read the output.

$Umbra’s fullscreen view of one reply on Claude Haiku 5.5: a table of the three discount tiers with borders, a formula with a stacked fraction, a Mermaid flowchart of the checkout drawn with box characters, and a Python function with syntax colors.$

Long sessions

It keeps workingafter the context fills up

A long task fills the context window, and an agent that handles that badly forgets what it was doing. Umbra replaces old command output with a short note near 80% of the window, and summarizes if that isn’t enough. On the eval tasks built to fill the window, 92.6% of runs passed.

You can see it coming.
The context bar in the status bar fills from green to yellow toward the compaction mark.
The cache survives.
74.2% of prompt tokens come from the cache right after a compaction.
Do it by hand.
/compact with a focus summarizes the session before the next step.

Umbra on Claude Haiku 5.5 with a 9,000 token context window, adding type hints file by file: the context bar fills to the compaction mark, the compaction line appears in the transcript, the bar drops, and the task finishes.

Your setup

Your instructions and skills,on the first run

Umbra reads the instruction files you already wrote: CLAUDE.md, AGENTS.md, GEMINI.md, Cursor rules and the rest. It lists your skills, offers your slash commands and starts the MCP servers in your config files. You don’t copy or convert anything.

See what it found.
umbra instructions, umbra skills, umbra commands and umbra mcp --connect list everything before you run a task.
Skills load when they match.
Only each skill’s name, description and path go into the prompt. The model reads the SKILL.md when a task needs it.
Hundreds of MCP tools.
With 229 tools set up, a “Hello” takes 85% fewer input tokens, because tool definitions load when the model asks for them.

The Umbra header with its skills and MCP rows and their counts, and the skills picker listing the skills it found in three tools’ folders.

Control

See every command.Stop it and say why

An agent that runs commands on your machine should show them first. Umbra waits at each command until you press Enter or type why it shouldn’t run. Ctrl+C stops a command or a model call, and a line you type while it works reaches the model before its next step.

Three modes.
Confirm asks before each command, yolo runs them, and in human mode you type the commands yourself.
Refuse with a reason.
Type anything at the prompt and the command doesn’t run. The model is told what you typed and tries again.
Run it in a container.
--docker <image> runs every command in a new container, the place for yolo mode on code you don’t trust.

Umbra in confirm mode: a command refused with the reason keep the build folder, clean only the cache; Ctrl+C stopping orders.py; and a line typed while it works turning the answer into a table of totals by customer, 4578.32 in all.

Any model

Claude, GPT or the modelon your own machine

One flag changes the model, and the model picker changes it during a session. A local model on llama.cpp or Ollama needs no key, and the status bar shows its cost as free. A model with no tool calling works with --text.

What a local model needs.

Our eval runs ninfer-qwen3.8-27b-coletti-v9, a 27B model, on one RTX 5090 with 32 GB of memory. Umbra talks to it like any other server with an OpenAI API.

Where a hosted model does better.

The largest hosted models are stronger than a 27B model on hard tasks. Umbra runs those too, with one flag.

Provider	Example
`anthropic`The Anthropic API, with an API key or your Claude login	`anthropic/claude-haiku-5-5`
`local`llama.cpp’s llama-server on localhost:8080, no key	`local/qwen`
`ollama`Ollama on localhost:11434, no key	`ollama/llama3`
`openai`OpenAI, with OPENAI_API_KEY	`openai/gpt-5`
`openrouter`OpenRouter, with OPENROUTER_API_KEY	`openrouter/anthropic/claude-sonnet-5-5`

Any other OpenAI compatible server, such as vLLM, is one entry in the config.

A reply on Claude Haiku 5.5, the model picker open with Claude Haiku 5.5 and Claude Sonnet 5.5, and the next answer on Sonnet with claude-sonnet-5-5 and a cost of $0.03 in the status bar.

Sessions

Pick upwhere you left off

Umbra saves the session after every step, so a run that was interrupted or crashed can be continued. Resuming a long session takes 36 ms. You can fork a session, or branch it from an earlier message.

List them.
umbra sessions lists this folder’s sessions, newest first, and umbra resume opens a picker.
Branch from any message.
--resume <id> --at <n> --fork continues a copy that keeps only the first n messages.
Shared with your editor.
A session started in Zed or Unluminous can be continued in the terminal, and the other way round.

A session stopped with Ctrl+C, then umbra resume opening the picker on Orders total and 15% discount test, and the resumed session answering a new message.

Details

The detailsyou notice in use

Status bar

The numbers that matter, under the input line

The status bar shows how full the context window is and where compaction starts, the agent’s time, the cost so far and the model. With a Claude login it also shows your weekly usage.

Umbra’s status bar under the input line: the context bar at 3%, 6.1k of 200k tokens, the agent time, the cost and the model name claude-haiku-5-5.

MCP panel

Every MCP server, one key away

/mcp opens a panel of the MCP servers Umbra found, with their tools. Turn a server off or on for this session or for every session. Tool definitions load when the model asks, so a large setup costs little.

Umbra’s MCP panel listing chordical, termoil and unluminous with 122 tools in all, with chordical and termoil turned off and chordical turned on again.

Updates

A new release, in the same session

Umbra checks for a new release once a day. In the fullscreen view it offers to install it, checks the download against the release’s SHA256SUMS, and restarts into the same session.

An older Umbra offering the 0.8.0 release with Install and restart, Remind me tomorrow and Skip this version, then the same session after it restarts on 0.8.0.

In your editor

Run it in your editor’sagent panel

umbra acp speaks the Agent Client Protocol, so Zed, the JetBrains IDEs and Unluminous can run Umbra in their agent panel. The editor shows each command as a tool call and asks you before it runs.

Unluminous with cart.py open and Umbra on Claude Haiku 5.5 in its agent panel: the task, the reply saying percent // 100 was floor division and all 4 tests now pass, and the changed line in the editor with its change bar.

Zedsettings.json

"agent_servers": {
  "Umbra": { "type": "custom", "command": "umbra", "args": ["acp"] }
}

JetBrainsacp.json

{
  "agent_servers": {
    "Umbra": { "command": "umbra", "args": ["acp"] }
  }
}

Unluminousai-endpoints.conf

provider.N.name = umbra
provider.N.wire = acp
provider.N.command = umbra
provider.N.args = acp

How it works

One loop, one tool,one list of messages

Umbra’s loop comes from mini-swe-agent: the model runs bash, sees what happened, and decides the next step. Umbra adds your setup, confirm mode, the fullscreen view, sessions and compaction.

You give it a task.
Umbra builds a system prompt from your instruction files and the list of your skills, then adds your task.
It asks the model.
The reply asks for bash commands as tool calls, or with --text, as fenced blocks for a model without tool calling.
It runs each command in your shell.
The shell stays open between commands, so cd and export carry over. A command is stopped after 120 seconds unless the model asks for longer.
It shows the model what happened.
The output and exit code join the one list of messages. A reply with no command ends the turn, and your next message continues it.
Long runs are compacted.
Near 80% of the context window, old command output is replaced by a short note. If that isn’t enough, the model summarizes the middle of the session.

 ┌────────────┐     ┌────────────┐
 │ your task  ├────►│ model      ├────► reply
 └────────────┘     └──┬──────▲──┘
                       │      │
                 ┌─────▼──┐ ┌─┴──────┐
                 │ bash   ├►│ output │
                 └────────┘ └────┬───┘
                                 │
                             ┌───▼─────┐
                             │ compact │
                             └─────────┘

What it leaves out

What Umbraleaves out

Umbra has one tool, bash, and adds MCP tools beside it. These aren’t in this version. An MCP server can supply web search or a browser.

~~subagents~~
~~hooks~~
~~plan mode~~
~~todo tool~~
~~web search~~
~~edit tools~~
~~image paste~~
~~git checkpoints~~
~~MCP OAuth~~

If you need one of these built in today, Umbra isn’t the right agent yet.

Questions

Questionspeople ask first

What hardware does a local model need?

Our eval runs ninfer-qwen3.8-27b-coletti-v9, a 27B model, on one RTX 5090 with 32 GB of memory. We haven’t measured smaller cards. Umbra works with any llama.cpp, Ollama or OpenAI compatible server, so you choose the model that fits.

Does Umbra send my code anywhere?

Your prompts and the files the model reads go to the model provider you choose, and stay on your machine with a local model. Umbra itself makes two requests: a daily release check to GitHub, off with UMBRA_UPDATE_CHECK=0, and with a Claude login, a weekly usage check to Anthropic.

Does it work on Windows?

Yes, on x86_64 and arm64. Umbra runs commands with Git Bash, so install Git for Windows first.

Can I use my Claude subscription?

Yes. Umbra uses your Claude login when you’re signed in, or an ANTHROPIC_API_KEY. With the login, the status bar shows your weekly usage.

Will the eval results hold on Claude or GPT?

We expect them to, because the gains come from sending less, caching more and handling long sessions better, and none of that depends on the model. We haven’t measured it on a hosted model yet.

Do I have to convert my instruction files?

No. Umbra reads CLAUDE.md, AGENTS.md, GEMINI.md, Cursor and Windsurf rules, Copilot instructions and the rest where they already are, and lists them with umbra instructions.

Install

Install Umbra 0.8.0

curl -fsSL https://raw.githubusercontent.com/Black-Rainbow-Labs/umbra/main/install.sh | shpowershell -ExecutionPolicy ByPass -c "irm https://raw.githubusercontent.com/Black-Rainbow-Labs/umbra/main/install.ps1 | iex"

Or have your agent install it

Paste these instructions into any coding agent that can run a command. It installs Umbra for your system, checks it, and saves the model.

Install Umbra, the coding agent from https://github.com/Black-Rainbow-Labs/umbra.
On macOS or Linux run: curl -fsSL https://raw.githubusercontent.com/Black-Rainbow-Labs/umbra/main/install.sh | sh
On Windows run in PowerShell: irm https://raw.githubusercontent.com/Black-Rainbow-Labs/umbra/main/install.ps1 | iex
On Windows, check that Git for Windows is installed, because Umbra runs commands with Git Bash.
Then run `umbra --version` to check it is on the PATH, and run
`umbra setup --model anthropic/claude-haiku-5-5 --no-test` (or another provider/model I name) to save the model.
To add it to an editor that speaks ACP, the command is `umbra` with the argument `acp`.

Homebrew or Cargo

brew install black-rainbow-labs/umbra/umbra
cargo install --git https://github.com/Black-Rainbow-Labs/umbra --locked

Then choose a model and give it a task

umbra setup
umbra -m anthropic/claude-haiku-5-5 "create sum.py that prints the sum of 1 to 100, run it, and tell me the result"

The install scripts download the build for your machine from the latest GitHub release, check it against the release’s SHA256SUMS, and put it in your home folder, so neither needs administrator rights.

Downloads

Umbra is free and open source. A star on GitHub helps other people find it.

Star on GitHub

The open source coding agentfor any model, graded on 61 tasks

Each change to the loopis measured first

What’s in it.

What’s measured.

What happened to each idea.

Results forthis release

Tested on local models,on purpose

One reply,seven layers

Tables, math and diagrams,in your terminal

Math.

Diagrams.

Code.

Command groups.

It keeps workingafter the context fills up

You can see it coming.

The cache survives.

Do it by hand.

Your instructions and skills,on the first run

See what it found.

Skills load when they match.

Hundreds of MCP tools.

See every command.Stop it and say why

Three modes.

Refuse with a reason.

Run it in a container.

Claude, GPT or the modelon your own machine

What a local model needs.

Where a hosted model does better.

Pick upwhere you left off

List them.

Branch from any message.

Shared with your editor.