https://github.com/user-attachments/assets/5d7663b2-fe50-4a05-8f86-5bbd8b08f9b3

Why OpenPhalanx?

• ✅ Private: your repo, tests and git stay on your laptop. Prompts go only to your own server, over TLS pinned to its certificate, and the server keeps no code.
• ✅ Models that fit: the app shows which models fit your GPU, counting the KV cache and runtime memory, not just the download size.
• ✅ Whole-repo agent: the agent reads your files on demand and runs your own tests and commands (after asking). Add tools from any MCP server (GitHub, a database, a browser…) as connectors; you approve each call.
• ✅ Documents and pictures: the agent reads and writes PDF, Word, OpenDocument, RTF, EPUB and Excel files, and sees pictures (PNG, JPEG, GIF, WebP) when the model can. It also draws pictures from SVG. Videos and presentations aren’t supported.
• ✅ Managed context: when a conversation grows, older content is trimmed or summarized automatically.
• ✅ Web search: the server searches the web through its own private SearXNG when a request needs current information.
• ✅ Clusters: servers on one network can form a cluster and split a model too big for one GPU across them.

How fast is it?

Measured on our own hardware with OpenPhalanx 0.7.4: an RTX 3090 (24 GB) desktop and an RTX 4090 Laptop GPU (16 GB), alone and as a cluster. Full method and tables: BENCHMARK.md.

Speed, one user, nothing cached. “Writes” is how fast the answer streams; the other columns are the wait before it starts, by prompt length.

Qwen3.6-27B fits neither card alone; split across both machines over gigabit Ethernet it runs with a 64k context (a 60k prompt: 56 s to start, then 29 tok/s). gpt-oss-20b, Devstral-24B and the 27B models don’t fit the 16 GB laptop by the app’s estimate.

A real agent session. Ten turns of questions and edits on this repository, Qwen3.6-27B split:

Each turn resends the conversation so far; the server keeps what it has already read (SGLang’s RadixAttention), so only the new part is processed.

Shared by a team. Several people at once, 2k-token prompts:

Does the agent finish the job? Nine edit tasks on this repository, each checked by compiling or running the tests (three name no file): 7 of 9 with Qwen3.6-27B split, 14–41 s each; 9 of 9 with Qwen3.8-27B split in an earlier run.

Quick Start

Every GPU machine needs Linux, a working nvidia-smi (24 GB or above VRAM recommended, 8GB absolute minimum), Docker (your user in the docker group) and the NVIDIA Container Toolkit. The installers check these first.

A. One GPU machine with a desktop (the app)

curl -fsSL https://github.com/chophilip21/openPhalanx/releases/latest/download/install-server.sh | sh

1. Open Openphalanx. On Models, pick a model marked Fits and click Download.
2. On Server, press the power button. The first start downloads the server image (16 GB, once); loading a model takes 3–4 minutes.
3. When the button is green, a pairing code appears. Open port 9090/tcp to your laptops and go to Laptop.

B. One GPU machine without a desktop (oppxs)

curl -fsSL https://github.com/chophilip21/openPhalanx/releases/latest/download/install-server.sh | sh -s -- --headless
systemctl --user enable --now oppxs    # the server process; keeps running
oppxs models                           # what fits this GPU
oppxs download <model> && oppxs use <model>
oppxs start                            # serves the model; `oppxs stop` to stop
oppxs pair                             # prints the pairing code for a laptop

C. Several GPU machines as one (a model too big for one GPU)

One machine is the host: you start the model there and laptops pair with it. It must run the app (A). Every other machine is a member: install the app, or oppxs with the first command of B. All on the same version and the same wired network.

Between the machines, allow 9092/tcp, 9093/udp and 9100–9116/tcp, and keep them on a network you trust. oppxs status on a member shows its host and anything waiting for approval; oppxs leave takes it out again. A machine that doesn’t show up usually has a second server process holding its ports (the app and oppxs can’t both run): oppxs free-ports, or Free the ports in the app, stops the leftover.

Laptop (Linux or macOS)

curl -fsSL https://github.com/chophilip21/openPhalanx/releases/latest/download/install.sh | sh
oppx pair 192.168.1.77 ABCD-EFGH  # the host's address and pairing code (on the server itself: oppx pair self)
oppx status                       # certificate, device and model all ✓

oppx pair prints the server’s certificate fingerprint; check it matches the one in the app (or oppxs pair). Then, in any git repo:

oppx                              # start a conversation
oppx "fix the failing test"       # start with a request
oppx -c                           # continue the last conversation (-r to pick an older one)
oppx -p "explain src/main.rs"     # answer once and exit
oppx --update                     # update oppx and its engine
oppx mcp add <name> -- <command>  # add an MCP server's tools (/mcp in the chat)
oppx browser on | off             # headless browser for web UIs: start with every chat, or ask once (default)

Edits are never committed: review them with git diff and commit yourself. oppx unpair removes a laptop (or revoke it on the app’s Devices page).

Keys in the chat

Shift+Tab plan mode (no edits) · Esc interrupt · Ctrl-C twice exits · @file add a file · !cmd run a command · # note save to project memory · /help all commands

Project notes

oppx reads OPENPHALANX.md from the root of your repository at the start of every chat and follows it, like CLAUDE.md in Claude Code. Put there what it should always know: how to build and test, the layout, your conventions. OPPX.md and AGENTS.md are read too if they exist.

/init writes the file for you from the repository · /memory opens it in your editor · # note adds a line to it

Connect tools with MCP

Connectors are MCP servers whose tools the agent can call, such as Linear, GitHub or a database. They run on your laptop with your own credentials; the server never sees them, and you approve each call (y, N, or a for always this session).

oppx mcp add <name> -- <command> [args]                  # a local server (stdio)
oppx mcp add <name> --url <https://…> -H 'Name: value'   # a remote server (streamable HTTP)
oppx mcp list                                            # what's added; /mcp in the chat
oppx mcp remove <name>

Example, Linear, with a personal API key (Settings → Account → Security & access):

export LINEAR_API_KEY=lin_api_...
oppx mcp add linear --url https://mcp.linear.app/mcp -H 'Authorization: Bearer ${LINEAR_API_KEY}'

Keep the single quotes: the key is read from your environment at start and never written to the config. For a browser login instead (needs Node): oppx mcp add linear -- npx -y mcp-remote https://mcp.linear.app/mcp. Then ask in the chat, e.g. “list my open Linear issues”.

Every connector’s tool list is sent with each request, so add only what you use; oppx mcp add prints the cost and warns when the server’s model has a small context window. --project saves a connector in the repo’s .mcp.json (it asks before running one from a cloned repo).

From source

Needs Rust, Node.js 20+ and the WebKitGTK packages listed in CLAUDE.md. Testing and the rules for changes are in CONTRIBUTING.md.

git clone -b dev https://github.com/chophilip21/openPhalanx.git && cd openPhalanx
(cd app && npm install && npx tauri dev)    # server app
cargo install --path client/oppx && cargo install --path crates/openphalanx-server   # oppx (client), oppxs (server command)

Contributing

Contributions are welcome. The pull request workflow, PR title prefixes and review rules are in CONTRIBUTING.md.

By contributing, you agree that your contributions will be licensed under the project’s GNU General Public License v3.0 (GPLv3).

Learn more

• CLAUDE.md (architecture, security model, clusters, connectors, troubleshooting) · Releases (downloads and changelog)

Client CLI reference

Generated from oppx 0.8.1 (f3d1893).

Use an OpenPhalanx GPU server from this machine.

Run `oppx` in a git repo to start the OpenPhalanx chat (keys and commands follow Claude Code), or pair first with `oppx pair <server> <code>`.

Examples:
  oppx
  oppx "fix the failing test"
  oppx -p "explain src/main.rs"
  oppx -c   (continue the last conversation)
  oppx -r   (pick a past conversation)
  oppx --update

Usage: oppx [OPTIONS] [PROMPT]
       oppx <COMMAND>

Commands:
  pair     Pair with a server using the code shown in the OpenPhalanx app, or with the server on this machine: `oppx pair self` (no code needed)
  status   Check the connection, the device token and the model
  unpair   Forget a server and revoke this device's token on it
  aider    Run Aider with its own terminal interface (plain `oppx` uses OpenPhalanx's)
  proxy    Run only the local proxy, for other OpenAI-compatible tools
  search   Search the web through the server's private SearXNG
  mcp      Connectors: MCP servers whose tools the model can call in the chat.
  browser  The chat's headless web browser (Playwright), run on this machine.
  servers  List paired servers (tokens are never shown)
  use      Set the default server
  help     Print this message or the help of the given subcommand(s)

Arguments:
  [PROMPT]
          Start the chat with this request

Options:
      --config <CONFIG>
          Config file (defaults to the user config dir, e.g. ~/.config/oppx/config.json)
          
          [env: OPPX_CONFIG=]

      --no-web
          With no command: start without automatic web search

      --classic
          With no command: use Aider's own terminal interface instead of OpenPhalanx's

      --engine <ENGINE>
          How the chat works: "agent" (the model reads files on demand with tools) or "aider" (Aider sends whole files and a repo map every time)
          
          [env: OPPX_ENGINE=]
          [default: agent]
          [possible values: agent, aider]

  -p, --print
          Print mode: answer PROMPT once and exit (for scripts and quick questions)

  -c, --continue
          Continue the most recent conversation in this repo

  -r, --resume [<ID>]
          Resume a past conversation in this repo (pick from a list, or give its id)

      --update
          Update oppx and its coding engine to the latest version

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version

oppx pair

Pair with a server using the code shown in the OpenPhalanx app, or with the server on this machine: `oppx pair self` (no code needed)

Usage: oppx pair [OPTIONS] <SERVER> [CODE]

Arguments:
  <SERVER>  Server address, e.g. 192.168.1.77 or https://gpu.lan:9090; or `self`
  [CODE]    Pairing code from the app's Server page, e.g. K7QF-M2XD (not for `self`)

Options:
      --config <CONFIG>            Config file (defaults to the user config dir, e.g. ~/.config/oppx/config.json) [env: OPPX_CONFIG=]
      --fingerprint <FINGERPRINT>  Expected certificate fingerprint (as shown in the app); skips the prompt
  -y, --yes                        Accept the server's certificate without comparing fingerprints
      --device-name <NAME>         Name this device shows under on the app's Client page
      --as <NAME>                  Local name for the server (defaults to its address)
      --force                      Replace an existing pairing with the same local name
  -h, --help                       Print help

oppx status

Check the connection, the device token and the model

Usage: oppx status [OPTIONS] [NAME]

Arguments:
  [NAME]  Server name (defaults to the default server)

Options:
      --config <CONFIG>  Config file (defaults to the user config dir, e.g. ~/.config/oppx/config.json) [env: OPPX_CONFIG=]
  -h, --help             Print help

oppx unpair

Forget a server and revoke this device's token on it

Usage: oppx unpair [OPTIONS] [NAME]

Arguments:
  [NAME]  Server name (defaults to the default server)

Options:
      --config <CONFIG>  Config file (defaults to the user config dir, e.g. ~/.config/oppx/config.json) [env: OPPX_CONFIG=]
      --local-only       Only forget it locally; don't contact the server
  -h, --help             Print help

oppx aider

Run Aider with its own terminal interface (plain `oppx` uses OpenPhalanx's).

Everything after `--` (or any unrecognized argument) goes to Aider, e.g. `oppx aider -- --message "add tests" src/lib.rs`.

Usage: oppx aider [OPTIONS] [AIDER_ARGS]...

Arguments:
  [AIDER_ARGS]...
          Arguments passed through to Aider

Options:
      --config <CONFIG>
          Config file (defaults to the user config dir, e.g. ~/.config/oppx/config.json)
          
          [env: OPPX_CONFIG=]

      --server <SERVER>
          Server name (defaults to the default server)

      --no-web
          Don't let the server search the web automatically

  -h, --help
          Print help (see a summary with '-h')

oppx proxy

Run only the local proxy, for other OpenAI-compatible tools

Usage: oppx proxy [OPTIONS] [NAME]

Arguments:
  [NAME]  Server name (defaults to the default server)

Options:
      --config <CONFIG>  Config file (defaults to the user config dir, e.g. ~/.config/oppx/config.json) [env: OPPX_CONFIG=]
      --port <PORT>      Local port (default: a free one) [default: 0]
      --no-web           Don't let the server search the web automatically
  -h, --help             Print help

oppx search

Search the web through the server's private SearXNG.

Inside Aider: `/run oppx search "your query"` adds the results to the chat.

Usage: oppx search [OPTIONS] <QUERY>...

Arguments:
  <QUERY>...
          What to search for

Options:
      --config <CONFIG>
          Config file (defaults to the user config dir, e.g. ~/.config/oppx/config.json)
          
          [env: OPPX_CONFIG=]

  -n, --max <MAX>
          Number of results
          
          [default: 6]

      --server <SERVER>
          Server name (defaults to the default server)

  -h, --help
          Print help (see a summary with '-h')

oppx mcp

Connectors: MCP servers whose tools the model can call in the chat.

A connector is a program on this machine or a URL that offers tools
(GitHub, a database, a browser…). The chat lists them for the model,
and asks you before each call. In the chat, `/mcp` does the same.

Examples:
  oppx mcp add github -e GITHUB_TOKEN=… -- npx -y @modelcontextprotocol/server-github
  oppx mcp add docs --url https://mcp.example.com/mcp -H "Authorization: Bearer …"
  oppx mcp list
  oppx mcp remove github

Each connector's tool list is sent with every request, so on a model
with a small context window add only the ones you need.

Usage: oppx mcp [OPTIONS] [COMMAND]

Commands:
  add     Add a connector: a command to run (after `--`), or a URL.
  remove  Remove a connector (yours, or this repository's with --project)
  list    Show the connectors you added and this repository's (the default)
  help    Print this message or the help of the given subcommand(s)

Options:
      --config <CONFIG>
          Config file (defaults to the user config dir, e.g. ~/.config/oppx/config.json)
          
          [env: OPPX_CONFIG=]

  -h, --help
          Print help (see a summary with '-h')

oppx browser

The chat's headless web browser (Playwright), run on this machine.

The model uses it to open pages (your local dev server included),
click, type, and read the page and its console errors. By default the
chat asks once a session before starting it; `on` starts it with every
session without asking. Needs Node.js 18 or newer.

  oppx browser            show the setting and what is installed
  oppx browser on | off   start it with every session, or ask first (the default)
  oppx browser install    install it now (otherwise on first use)

Usage: oppx browser [OPTIONS] [COMMAND]

Commands:
  on       Start the browser with every chat session, without asking
  off      Ask once a session before starting it (the default)
  install  Install or update it now
  status   Show the setting and what is installed (the default)
  help     Print this message or the help of the given subcommand(s)

Options:
      --config <CONFIG>
          Config file (defaults to the user config dir, e.g. ~/.config/oppx/config.json)
          
          [env: OPPX_CONFIG=]

  -h, --help
          Print help (see a summary with '-h')

oppx servers

List paired servers (tokens are never shown)

Usage: oppx servers [OPTIONS]

Options:
      --config <CONFIG>  Config file (defaults to the user config dir, e.g. ~/.config/oppx/config.json) [env: OPPX_CONFIG=]
  -h, --help             Print help

oppx use

Set the default server

Usage: oppx use [OPTIONS] <NAME>

Arguments:
  <NAME>  Server name, as listed by `oppx servers`

Options:
      --config <CONFIG>  Config file (defaults to the user config dir, e.g. ~/.config/oppx/config.json) [env: OPPX_CONFIG=]
  -h, --help             Print help

Gateway API reference

Generated from docker/server/gateway.py.

Public API (port 9090, TLS)

Clients authenticate with Authorization: Bearer <device token>. oppx adds it; agents talk to oppx’s loopback proxy.

Admin API (127.0.0.1:9091)

Used by the server app; needs the per-launch x-admin-token.

CLAUDE.md

Working notes for developing this repo. This file holds the conventions, the architecture in brief, the commands and the layout; each topic’s detail (how it works, what was measured and verified) is in its own file under .claude/skills/, listed in Topics below. End-user docs are in README.md; the release history is in CHANGELOG.md.

Conventions

• Docs:
  • README.md is for end users only: what the project is, plus the minimum commands to run the server and a client. Keep it at 200 lines at most.
  • Everything operational or developer-facing goes in the matching topic file under .claude/skills/<topic>/SKILL.md, including what was measured and verified. This file stays an index: add here only what applies to every change. A new topic gets its own folder, a name and a description saying when to read it, and a row in Topics.
• Agents never commit directly: any agent launcher or config (Aider, oppx aider, etc.) must use --no-auto-commits --no-dirty-commits. The user reviews with git diff and commits themselves.
• Commits: never commit or push unless the user explicitly asks for that commit. Finish with an uncommitted diff and a summary.
• Model-agnostic: everything that makes the experience seamless must work with any model SGLang serves. Use constrained decoding (regex or JSON schema) rather than per-model tool-call or reasoning parsers, put unavoidable per-model differences in catalog data, and check accuracy with probe sets against the loaded model rather than tuning prompts to one model.
• Branching and releases: work happens on dev. A PR from dev into main is a release: its title prefix sets the version bump, in any letter case ([bug]/[fix] patch, [feature] minor, [major] major; [docs], [ci], [chore], [test], [refactor] don’t release). See “Releasing” below.
• Toolchain on the server node: cargo is in ~/.cargo/bin (not on the non-interactive PATH), so use export PATH=$HOME/.cargo/bin:$PATH. uv/uvx are in ~/.local/bin.
• Versioning: the backend image tag equals the workspace version in Cargo.toml. The release workflow bumps Cargo.toml, Cargo.lock, app/package.json, app/package-lock.json and app/src-tauri/tauri.conf.json together (scripts/bump_version.py); don’t bump by hand.
• Regular expressions: every pattern in the repository must pass scripts/check_regexes.py (CI runs it). It finds the patterns by itself (Python’s syntax tree, the TypeScript parser), compiles each, and proves with recheck that none backtracks exponentially or polynomially. Python’s re and JavaScript backtrack, so a harmless-looking \s+$ searched over a long run of spaces is quadratic. When it flags one, pin where a match may start ((?<![ \t])[ \t]+\n), keep a repeat from crossing its own delimiter ([^{}]*, _([^_]+?)_), or use string methods. A pattern only known at run time gets a # regex: runtime (why it is safe) comment; a pattern the model writes runs through grep -E, never Python’s re.
• Verify before claiming: run cargo test -p openphalanx-core, cargo clippy --workspace --all-targets, (cd app && npm run check), and for backend changes the lifecycle example (about 4 min, needs the GPU).

Architecture

flowchart TB
    subgraph ClientNode ["Client Node (Work Laptop)"]
        direction TB
        CLI["oppx CLI<br/>(Rust: pairing, pinned-TLS proxy)"]
        Agent["Coding agent<br/>(Aider, runs locally)"]
        FS[("Local Workspace<br/>git, tests, toolchain")]

        CLI -->|"Launches, injects endpoint"| Agent
        Agent <-->|"Reads repo, edits files, runs tests"| FS
        Agent <-->|"OpenAI-compatible HTTP<br/>(loopback)"| CLI
    end

    subgraph ServerNode ["Server Node (Primary Rig)"]
        direction TB
        GUI["Openphalanx GUI<br/>(Tauri / Rust Orchestrator)"]
        Gateway["Agent Gateway<br/>(TLS, device tokens)"]
        Master["SGLang Server<br/>(Inference Master)"]

        GUI -.->|"Orchestrates, pairs devices"| Gateway
        GUI -.->|"Orchestrates"| Master
        Gateway <-->|"Local HTTP API"| Master
    end

    CLI <==>|"HTTPS + device token<br/>(prompts only; no code stored)"| Gateway

    classDef clientStyle fill:#0f172a,stroke:#38bdf8,stroke-width:2px,color:#fff;
    classDef serverStyle fill:#1e1b4b,stroke:#818cf8,stroke-width:2px,color:#fff;
    classDef modelStyle fill:#422006,stroke:#fb923c,stroke-width:2px,color:#fff;

    class CLI,Agent,FS clientStyle;
    class GUI,Gateway serverStyle;
    class Master modelStyle;

Openphalanx splits the work along a single line: code and execution stay on the client, GPUs stay on the server.

• Client: the coding agent (Aider) runs on your laptop next to your repo, so it sees the whole codebase and runs your own tests, linters and git. The oppx CLI pairs the laptop with a server, then launches the agent pointed at a local proxy. The proxy pins the server’s TLS certificate and adds the device token, so the agent never handles either.
• Server: stores no code and executes nothing on a client’s behalf. It serves SGLang behind an authenticated gateway, and the Tauri GUI manages the container, VRAM safety, models and paired devices. SGLang’s RadixAttention caches repeated prompt prefixes, so multi-turn agent sessions skip most of the prompt processing.

Status

Requirements

Server node

• Linux x86-64 (the GUI is Linux-only for now).
• NVIDIA GPU with a recent driver (nvidia-smi must work). A 24 GB card runs the default Qwen2.5-Coder-14B-AWQ with a 32k context.
• Docker Engine, with your user in the docker group.
• NVIDIA Container Toolkit, registered with Docker (sudo nvidia-ctk runtime configure --runtime=docker).
• Disk: about 55 GB for the backend image (16 GB download), plus the model weights (5–80 GB depending on the model).

Building from source additionally needs Rust (stable, via rustup), Node.js 20+ with npm, and the WebKitGTK development packages:

sudo apt install libwebkit2gtk-4.1-dev libgtk-3-dev libsoup-3.0-dev \
  libayatana-appindicator3-dev librsvg2-dev libxdo-dev build-essential

Topics

Read the topic’s file before working on that part; add what you measure or verify to it. In Claude Code these are project skills, so each loads by its name when a task matches its description.

Development

cargo test -p openphalanx-core                                  # unit tests (no GPU or Docker needed)
cargo test -p oppx                                          # client unit tests
cargo run -p oppx -- pair <server> <code>                   # pair (also: status, unpair, servers, use; config via OPPX_CONFIG or --config)
# note: `cargo test`/`clippy` don't rebuild target/debug/oppx; run `cargo build -p oppx` before testing the binary
cargo clippy --workspace --all-targets
(cd app && npm run check)                                       # Svelte/TypeScript type check

cargo run -p openphalanx-core --example preflight               # pre-flight report for this machine
cargo run -p openphalanx-core --example catalog -- 32768       # catalog VRAM estimates for this GPU
cargo run -p openphalanx-core --example lifecycle               # full start → pair → token check → revoke run
cargo run -p openphalanx-core --example download -- <repo> <dir>  # verified, resumable HF download

docker build -f docker/Dockerfile.server -t openphalanx-backend:dev docker/   # the backend image by hand (the app builds its own; see below)
scripts/bench_session.py --oppx target/debug/oppx               # prefix-cache benchmark (in a scratch copy of a repo)
scripts/bench_speed.py --label 3090 --models <id,id>            # speed per model for BENCHMARK.md (app closed); --summary <results.jsonl…> for the tables
scripts/gen_docs.py --oppx target/debug/oppx --out target/docs  # docs sources; then: mdbook build target/docs
OPENAI_API_BASE=… OPENAI_API_KEY=… scripts/probe_routing.py     # via `oppx proxy --no-web`; run after any model change
OPENAI_API_BASE=… OPENAI_API_KEY=… scripts/probe_kv.py          # long-context recall + coding tasks run against tests (e.g. KV dtype checks)
scripts/eval_app.py --oppx target/debug/oppx [--task build,improve] [--runs 5]   # build and improve a small web app, scored per requirement in a browser; --agent claude for the baseline
python3 scripts/test_searchguard.py                             # the gateway's search guard (cache, pacing, pause when the engines refuse)
npm ci --prefix scripts/regexcheck && python3 scripts/check_regexes.py   # every regex compiled and checked for catastrophic backtracking (--list shows them)
python3 scripts/test_inputs.py                                  # the gateway's check of what the loaded model takes (pictures, audio, video)
scripts/eval_edits.py --oppx target/debug/oppx --engine agent   # 9 edit tasks in fresh clones (3 name no file), checked by compiling/testing; compare engines
~/.local/share/oppx/engine/tools/aider-chat/bin/python scripts/eval_nav.py --repo <scratch clone>     # 10 questions that name no file (via `oppx proxy`): answered, steps, stuck
~/.local/share/oppx/engine/tools/aider-chat/bin/python scripts/test_agent.py                          # agent loop unit tests, scripted model, no server
python3 scripts/test_permissions.py                             # the agent's command allowlist (read-only commands, saved rules, what is never remembered)
python3 scripts/test_reqtrace.py                                # the gateway's request trace (prefix chains, tree, labels, content switch)
~/.local/share/oppx/engine/tools/aider-chat/bin/python scripts/bench_agent.py --repo <scratch clone>   # agent turns headless, steps and tokens
~/.local/share/oppx/engine/tools/aider-chat/bin/python scripts/test_connectors.py   # connectors: MCP client + the agent's tool calls (no server needed)

The backend image tag follows the version in the root Cargo.toml, so bump both together when changing docker/. The base SGLang image is pinned by digest in docker/Dockerfile.server.

Troubleshooting

Repository layout

.claude/skills/          the developer notes by topic (one SKILL.md each; index in this file under Topics)
app/                     Tauri 2 + Svelte 5 server GUI (Linux); src-tauri/icons/ from icon.svg (assets/openphalanx-shield-green.svg,
                         on a square canvas centred on the shield, not the arrow): npx tauri icon src-tauri/icons/icon.svg -p 32,64,128,256,512, then rename
                         256 to 128x128@2x.png and 512 to icon.png. On Wayland the dock icon never comes from the binary
                         (GTK3 sends only the app id, `openphalanx`): it's the icon of the installed .desktop entry, so
                         `tauri dev` shows the installed package's icon
  src/                   frontend: pages, components, typed command bindings
  src-tauri/             Rust shell: Tauri commands, 2 s status monitor, log streaming
crates/openphalanx-core/ Docker, GPU, VRAM, model catalog, downloads, pre-flight, cluster, split plan, app updates (no Tauri, unit-tested)
crates/openphalanx-server/ oppxs: the server without the desktop app (daemon, control API and its commands)
packaging/oppxs.service  systemd user unit for oppxs (in the .deb; install-server.sh --headless writes its own)
crates/pinned-tls/       TLS pinned to a certificate fingerprint (shared by oppx and the cluster)
  catalog.json           curated models pinned to Hugging Face commits (built with scripts/catalog_entry.py); optional per-model
                         edit_format and reasoning_parser (passed as EDIT_FORMAT, --reasoning-parser)
docker/                  backend image: SGLang + gateway under supervisord (no agent code)
  server/gateway.py      TLS gateway: pairing, device tokens, admin API
  server/reqtrace.py     the request trace behind the app's Trace page (in memory, no dependencies)
  server/inputs.py       what the loaded model takes (from SGLang at load), and the refusal for anything else
  server/searchguard.py  cache, pacing and pause for web search, so the public engines don't block the server
client/oppx/             client CLI: config.rs (paired servers, 0600 file), tls.rs (fingerprint pinning),
                         api.rs (gateway calls), engine.rs (private Aider), mcp.rs (connector files), update.rs, main.rs (clap commands)
  frontend/              chat frontend: oppx_chat/ package + run.py (embedded in the binary)
                         (oppx_chat/documents.py reads PDF, Office, EPUB and pictures; writers.py makes them)
.github/workflows/       ci.yml (every PR/push), release.yml (merged PR into main -> release)
scripts/test_connectors.py connectors (MCP): the client against a stdio server, and the agent calling a tool (scripted model)
scripts/eval_app.py      the CHO-8 app task end to end, scored per requirement (eval_app_check.mjs: the browser checks)
scripts/probe_browser.py does the loaded model ask for the browser when a message needs a running page, and only then
scripts/probe_routing.py score the search router and ask/edit check against the loaded model
scripts/check_regexes.py every regex in the repository, compiled and proven free of catastrophic backtracking (regexcheck/: recheck, pinned)
scripts/catalog_entry.py catalog entries from Hugging Face (pinned commit, weight size, attention shape); --check re-verifies
scripts/bench_session.py 10-turn prefix-cache benchmark through a real oppx session
scripts/bench_speed.py   speed per model through oppxs (one user per prompt length, several users), for BENCHMARK.md
benchmarks/results/      the raw results behind BENCHMARK.md (one JSON line per measurement, per machine and date)
scripts/bump_version.py  version bump from a PR title, plus the CHANGELOG.md entry
scripts/gen_docs.py      docs site sources (README, CLI and API reference, CLAUDE.md and its topic files, changelog)
scripts/install*.sh      client and server installers (attached to each release)

Design decision: agent on the client

We compared three placements of the agent:

• Agent on the server, sent files per task: this was our first prototype (a /v1/run endpoint wrapping Aider in the image, since removed).
• Everything on the server, used over SSH.
• Agent on the client, server as inference only.

We chose the last. It keeps the codebase as the single source of truth on the client. It needs no shell access or workspaces on the server, which fits the pairing and device-token security model. It gives the agent full repo context and a local edit → test → fix loop. And it leaves VRAM pooling across machines as a server-side concern that clients never see.

The network cost of running the agent on the client is negligible. Measured on the RTX 3090 node with Qwen2.5-Coder-14B-AWQ, for an Aider-sized request of 7.2k prompt tokens and a 300-token answer:

Speed is decided by the GPU and, above all, by prefix-cache hits, which behave the same wherever the agent runs.

Building an app end to end: the CHO-8 audit and scripts/eval_app.py

Part of the developer notes; the index is in CLAUDE.md.

• Building an app end to end (CHO-8: search the web, open a Linear issue, scaffold and write a Vite + React app, start it, check it in the browser, close the issue; split Qwen3.6-27B at 64k, a real chat driven through tmux). What it broke on, and the fixes:

  • run has no terminal and leaves nothing behind (Workspace.run): stdin is closed, so a command that asks a question ends instead of waiting on the user’s terminal unseen (npm create vite sat there for the whole 120 s); it runs in its own process group, which the timeout or an interrupt kills (only the shell was killed, and npm create stayed alive); a timeout returns the output until then, and says to pass answers as flags or use start.

  • start (Workspace.start): a command that keeps running. Its output goes to a temporary log; the call returns once it has printed something and gone quiet for 2 s (15 s at most), or with the exit code if it ended. Called again with the same command it returns only the new output. Everything started is stopped when the chat exits (stop_jobs, at exit; not on SIGHUP or SIGKILL). It is asked like run, and the question still reads “Run this command?”.

  • Lookups may repeat after the files changed: an applied edit, a run or a start clears the repeat guard. Before, a read after a half-applied edit came back as “You already did exactly this at step 1” three times and the turn ended stuck, the model convinced the reads were cached.

  • create replaces a whole file that was read (REWRITE_PROMPT; the old text goes to the diff and /undo). As SEARCH/REPLACE, replacing Vite’s starter App.jsx meant copying all 122 lines first; the model wrote one sentence and no block.

  • A cut-off action is asked for again (_action): the grammar allows any whitespace between fields, and one action filled its 400 tokens with newlines after "missing" (11 s) and was then taken as answer. Action calls stop at a blank line, and one without a tool is retried once at temperature 0.7.

  • An empty plan is asked for again (_think), first character constrained, as for edit text: “(no plan written)” showed whenever the plan prompt was answered with the action.

  • Connector calls that only read are named as such in the reply’s facts: asked to mark the issue Done, the model called linear.get_issue and reported “Status changed from Backlog to Done” (it was still Backlog). Asked “Did you actually…?” on the new build, it checked and made the change.

  • The browser line says it is Playwright: told to “run Playwright”, the model went to npm install -D @playwright/test instead of the browser action.

  • OPPX_DUMP_REQUESTS now works for the agent engine too (each call with its reply and usage; agent._dump).

  • Result: all five steps done over four sessions with these hints from the driver: scaffold without prompts, the right Algolia host, “the server is already running” (it wanted npm run dev -- --host 0.0.0.0 through run), and the question about the issue. The app builds and loads 30 stories; typing LLM and submitting filters them with no console errors. Left as the model wrote it: it searches on submit, not as you type; it shows Algolia’s relevance order, not the front page; it invented an Algolia application id and key before being corrected; lucide-react is installed and unused.

  • “Improve the app” (five changes, then lint, build, dev server and a browser check), same model. The first two tries ended with nothing changed and “Let me start by making the changes.” followed by a raw {"enough": false, …} line in the reply. The model wrote "enough": true meaning “enough to start”, and the only tool that goes with it is answer. Fixes: the system prompt says that for a change request enough stays false until the edits are made; a change request that chooses answer a second time with nothing changed gets one step with only edit, create, run and start on offer (with lookups and plan on offer too, it planned twice more and answered again); an action tacked onto the end of a reply is neither shown nor kept (_ACTION_TAIL). Third try, no hints: all in one turn (56 model calls), lint and build clean, typing “Rust” filtered and highlighted with no console errors. It reported “front page by default” as done though the request to Algolia is unchanged: a change the harness can’t check is still only as true as the model.

  • A resumed conversation is shown (app.show_history, after the welcome box on -c/-r): the earlier messages and replies (the latest 30, 40 lines each at most; slash commands and our appended notes left out), then a line saying the steps in between aren’t kept. Before, a resumed chat looked like a new one. Seen in a terminal on the CHO-8 session (10 messages).

  • As a repeatable eval (scripts/eval_app.py, CHO-12): both prompts in a fresh repository, in print mode, with nobody helping, and scored by the script instead of the reply: it installs and builds the app, serves the build, opens it in headless Chrome (eval_app_check.mjs: the requests made, the story links, the console), types the keyword, and reads the issue tracker’s state. What a user would have said “always” to is allowed beforehand (npm, npx, mkdir, cp, mv, the browser, the tracker); Linear is a small stdio connector the script writes (same tool names, simple schemas), so runs leave nothing in a real workspace; a run gets “Continue…” up to twice while a check fails. The improve task starts from the app as the first run left it (scripts/fixtures/hn-dashboard, git-ignored: dummy apps are never committed, so a fresh clone has only the build task). --agent claude runs the same prompts through Claude Code with the same tracker and browser server.

  • First numbers (split Qwen3.6-27B, 64k; five runs each; Claude Code one run each):

    	oppx, before the fixes below	Claude Code
    build: every check passed	4 of 5 runs (3 in the first message), 2.0–3.9 min	1 of 1, 11.8 min, $1.23
    improve: every check passed	1 of 5 runs, 3.1–11.5 min, 44–140 model calls	1 of 1, 10.9 min, $0.90

    The build failure: the page shows nothing until a keyword is typed. The improve failures: “front page by default” reported as done with the request to Algolia unchanged, in 4 of 5 runs. In the run looked at closely a search result two steps earlier read “All stories that are on the front/home page right now: …/search?tags=front_page”; the model then wrote tags=story and called that the front-page endpoint. Claude Code’s apps used tags=front_page both times.

  • Fixes from those runs:

    • A review before the reply (REVIEW_PROMPT, CHO-14): on a request with several parts (numbered or bulleted items, or 400 characters; several_parts), once the model wants to answer, it writes one line per thing asked for with its evidence: an added line of the turn’s diff (Agent._diff, from the texts kept before each file’s first change, 9,000 characters at most), or a command or browser step and what it returned; otherwise NOT DONE. A NOT DONE line is to be done before answering, and the reply may not call it done. One more model call (500 tokens at most) on such turns only.
    • A read after a refused edit is not a repeat: the refusal says “Read the exact lines again and retry”, and that read came back as “You already did exactly this”, three times, until the turn ended stuck.
    • 80 steps for a turn that is changing files (MAX_STEPS_CHANGING; 40 otherwise): 4 of 5 first messages ended at step 40 mid-work, before their own review.
    • A connector’s name as a shell command is refused with what it is (_not_a_connector): with the browser’s tools on offer the model chose run with playwright browser navigate http://localhost:5173/, was declined, and went to the web to find out how to use Playwright.
    • Web searches are bounded (see the web-search topic, Search guard): 41 searches in that turn, the last ones for “a”, “b”, “c”.

  • With the review step (before the other four): improve passed every check in 3 of 5 runs, twice. Where the review ran, it caught the front page each time (“NOT DONE. The initial fetchStories(‘’) still queries tags=story…”); the failed runs never reached it (stuck at the step limit, or in the search loop), or knew it was undone and couldn’t find the parameter with search down.

  • With web search off (--no-web, the eval’s default since the engines blocked the server): the build task scored 0 of 7 twice. The system prompt still listed web_search, the action list didn’t have it, and the model typed web_search --query '…' as a shell command nine times in a row; a declined command was never a repeat, because run is exempt from the repeat guard. Fixes: without web search the prompt doesn’t list the tool and says it is off; one of the agent’s own actions typed as a command is refused with what it is (_not_a_program, which also covers connector names); a command that was declined or refused is a repeat when asked for again in the same request (declined); and only a command that really ran clears the repeat guard (Agent._ran). One run after that: 6 of 7 (the app’s index.html names a /favicon.svg it didn’t keep).

  • What the review can’t do: with web search off, improve failed “front page” in 5 of 5 runs. The review ran every time, and in most runs it accepted “fetches with no query” as the front page: it checks a claim against the diff and against what tools returned, and nothing in view contradicted the belief. One run asked for the parameter to be confirmed instead of claiming it.

  • The eval stops by itself when two runs in a row fail the same checks (--no-stop runs them all): a batch that repeats one failure is GPU time for nothing. Run one run and read it before a batch.

  • A scorer mistake worth knowing: Chrome asks for /favicon.ico when a page names no icon, and the static server’s 404 is logged as a console error. Two “console errors” in the first batch were that; the scorer now puts an empty one in the folder it serves.

  • Still open: a resumed session restores only the messages, not what tools returned; a search step shows “Web results (untrusted reference text):” as its summary instead of a count; a start for a second copy of a server that already runs isn’t caught; none of this was rerun from scratch in one session on the final build, nor on Qwen3.8.

Benchmarks: bench_speed.py and the numbers in BENCHMARK.md

Part of the developer notes; the index is in CLAUDE.md. The reader-facing page is BENCHMARK.md; this file holds how the numbers are made and what was found on the way.

How it runs

• scripts/bench_speed.py --label <machine> --models <id,id,…>: per model, oppxs use, oppxs context (the longest that fits when the asked one doesn’t, unless --no-fit), oppxs start (timed: start to serving), then SGLang’s bench_serving inside the backend container: one user at 2k/8k/32k prompts (512 out, 5 requests after one warm-up, prefix cache flushed), then 1/2/4/8 users (2k in, 256 out). Results: one JSON line each in <out>/results.jsonl; --summary <files…> prints the BENCHMARK.md rows from any number of them (both machines together).
• Its own settings and server identity (XDG_CONFIG_HOME, XDG_DATA_HOME under <out>; models linked to the real models folder): the app’s settings are untouched, and a machine that is a cluster member doesn’t ask its host before each start (the laptop, a member of the 3090’s cluster, failed with “can’t reach the host … take-over” before this). Web search is turned off in those settings.
• Split runs (--attach [--name …]): oppxs can’t host a split model (the split plan lives in the app), so the app on the 3090 is the host and the laptop a member; press Start in the app, then the script measures what serves in the host’s container and leaves it running. VRAM used is the host’s only.
• The desktop app must be closed (except with --attach) (it and oppxs can’t run on one machine at once); the script refuses otherwise.
• Raw results are committed under benchmarks/results/<date>-<machine>.jsonl (checked for home paths, addresses and tokens first); BENCHMARK.md and the README’s “How fast is it?” are written from them. Working copies go to target/bench/ (ignored).
• Only the standard library: on another server, scp the file and run it there. The laptop runs it from ~/oppx-bench/.

Test rig

• 3090 (this machine): RTX 3090 24 GB, power limit 350 W (max 375), driver 580.173.02, kernel 7.1.5, oppxs 0.7.4, SGLang 0.5.21.
• 4090 Laptop (ssh 4090): RTX 4090 Laptop GPU 16 GB, power limit 150 W (max 175), on AC, driver 580.126.18, kernel 6.18.7, oppxs 0.7.4.

Pitfalls found

• Prompts as text are ~10% longer than asked. random-ids decodes random token ids to text and the server re-tokenizes it longer: a “32k” prompt was 33.5–36k tokens, over a 32k context, every request refused, and bench_serving still counted them completed with a first token at 0 ms (it printed 8,614 tok/s). Fixed with --tokenize-prompt (exact lengths), and a result whose median first token is 0 ms is now an error. The first run (run1) is void for this reason.
• pkill -f bench_speed.py from a shell whose own command line contains that text kills the shell too.

Findings from the runs (open)

• Long prompts are slow to process on the 3090: Qwen2.5-Coder-14B AWQ, 32k prompt: 21.6 s to the first token (~1,500 tokens/s), against ~2,000 tokens/s at 8k. SGLang chose chunked_prefill_size=2048 (in the backend log). A larger chunk may help long prompts; not tried. This is the case a coding agent hits on its first request.
• oppxs context accepts any value when the model fits at no context (set_context in crates/openphalanx-server/src/main.rs: fit is None, so nothing is refused, and it prints “Context window set”); start then refuses with the real reason. Seen with gpt-oss-20b on the laptop (needs at least 17.9 GiB, 15.6 free). It should say the model doesn’t fit.
• Devstral-Small-2-24B AWQ (cyankiwi/…-AWQ-4bit) runs out of memory while loading on the 3090, although the estimate says it fits at 21.5k (catalog weights 15.0 GB): torch.OutOfMemoryError in compressed_tensors_wNa16.create_weights (ministral3.py) with 22.1 GiB in use, before any KV cache. The weights take far more memory loaded than the catalog’s 15 GB; the catalog entry or the estimate for this build is wrong. Not yet looked into. On the laptop the estimate refuses it (needs at least 24.0 GiB at the minimum context), which suggests the estimate already knows part of it.
• Qwen2.5-Coder-32B AWQ doesn’t fit the 3090 at any context by the estimate (needs at least 30.4 GiB at the asked 32k, and more than the 21.8 GiB free even at 2k), though it is the model people commonly run on a 3090 (as a 4-bit GGUF). The estimate may be too cautious here; not tried with Force load.
• On 16 GB (laptop) gpt-oss-20b and Devstral-24B fit at no context by the estimate (at least 17.9 and 24.0 GiB). Not tried with Force load.
• Split Qwen3.6-27B AWQ serves about 4 requests at once (3090 host + 4090 Laptop member, 64k, pp 2): 4 users 65 tok/s total, 8 users also 65 tok/s with the first token at 19.4 s instead of 4.3 s, so the rest queue. Likely the state cache of this hybrid (Mamba-style) model: the app passes --max-mamba-cache-size 16 (with --chunked-prefill-size 2048). Not looked into. Single user: 34.6 tok/s at 2k, 28.6 at 60k; prompt processing ~1,100 tok/s (56 s at 60k).
• The split results file carried the host’s LAN address (extra_args: --dist-init-addr); it is masked to <ip> in benchmarks/results/. Check every new results file before committing.
• The agent sometimes ends a -p run right after announcing an edit (eval_edits.py on split Qwen3.6-27B, 7/9): task 4 (“…the edit tool failed to match the search text. Let me retry with the exact content:”) and task 7 (“I’ll change the pairing code TTL from 10 minutes (600 seconds) to 15 minutes (900 seconds) in docker/server/gateway.py:101.”) ended with nothing changed. The reply is a plan, not a result; the agent should go on (or the loop treat an announced action as unfinished). For oppx-agent.
• bench_session.py hung on the first edit turn: after an edit the agent asks to run a check (Allow?), and nothing answered. It now declines every such question (n) and counts them; with a hang it prints the last screen. Session on split Qwen3.6-27B: 10 turns, 92% of 1.45M prompt tokens from the cache, reply on screen 0.2–1.1 s after a follow-up.
• Pairing for these runs: oppx pair self with OPPX_CONFIG in a scratch file (the user’s own pairing untouched), oppx unpair afterwards (revokes the device).
• The laptop reaches the first token slightly sooner than the 3090 (7B at 8k: 1.75 s vs 1.98 s), the 3090 generates ~40% faster (7B: 148 vs 108 tok/s; 14B: 81 vs 57). Expected: generation is bound by memory bandwidth (936 vs ~576 GB/s), prompt processing by compute.

Cluster: discovery, roles, joining, split serving, model sync

Part of the developer notes; the index is in CLAUDE.md.

• Concurrency (2026-10-16): a member’s report (every 5 s) asks its four probes together (the backend’s and the worker’s new log lines, collect_inventory, collect_serving, check_worker), so it takes as long as the slowest. The log lines are still queued backend first. take_over and the host’s invitations register every invite, then send them all at once, so one unreachable peer no longer holds up the others for its timeout. Model sync uses the parallel download (4 files at once).

Cluster

Servers on one network form a cluster: one host and its members. With the split strategy they serve one model together (see Split serving below). The service is cluster::Cluster in core. It runs inside the app, or headless as oppxs run (crates/openphalanx-server); both use ~/.local/share/openphalanx/cluster/, so run one per machine.

• Discovery: every server broadcasts a beacon (id, name, role, certificate fingerprint) on UDP 9093 every 3 s. A server is listed under Servers on this network only when its beacon is under 10 s old and a TLS health check, pinned to the fingerprint it announced, succeeds. Stopped or unreachable servers drop off within about 10 s. Broadcast only crosses one LAN segment, not Tailscale.

  • Fast join: a server that starts (app opened, or oppxs run) sends a query beacon twice; the others answer by unicast at once and check it immediately, so it is listed in about 0.3 s (measured on the 4090) instead of waiting for the next 3 s round.

• This machine is always “This machine” in the app (cluster panel, Machines, charts, logs, VRAM pool, the split summary) and in oppxs status. Its name is only what the other servers show for it (oppxs run --name, or Rename in the panel, which now says “seen by others as …”). Two machines with the same hostname had ended up with the same name, and the host looked like the member.

• Other servers are shown by address, or by a nickname given here (Cluster::label: nickname, else the host part of its URL, else the name it gives itself). Nicknames are per machine, keyed by server id, saved in state.json (nicknames), so they survive restarts and updates; set_nickname with no name removes one. Every place a server is named gets the label from the cluster view (MemberView.name, CandidateView.name, the host in a member’s role, invitations, the Rust-built messages and the split summary), so charts, logs and the VRAM pool follow; announced keeps the server’s own name for a tooltip. In the app, Nickname / Edit beside a server in the cluster panel (NodeName.svelte: save, remove, cancel); headless, oppxs nickname <server> [name]. The same store names paired client devices on the Trace page (by device id; a device has no address there, so its default stays the name it paired with).

• One server process per machine (core::ports): the app and oppxs share ports 9092, 9093/udp and 9094. A second one used to fail once and stay without a cluster service (invisible to the others). Now the app keeps retrying (serve_cluster: the API once the port is free; discovery retries its own bind), and the leftover can be stopped on request: ports::holders reads ss -H -ltnup, ports::free sends SIGTERM (so it stops its model and says goodbye) and SIGKILL after 8 s, only to processes named oppxs or openphalanx; anything else on those ports is named and left alone. oppxs free-ports [--yes]; the app’s Free the ports… asks first. Listing was run against the live app; the stopping path has unit tests for the parsing only.

• Roles (state.json): standalone, host or member. A host with no members is standalone again.

  • Clients pair with the host only: members hide the pairing section, refuse new_pairing_code, and skip the start-up pairing code.

• Joining needs consent on both sides:

  1. Add to cluster: the host sends an invitation (a single-use secret bound to that server, valid for 10 minutes).
  2. Approve on the invited server: in its app, or with oppxs approve, after checking the host’s fingerprint. It then joins over a connection pinned to that certificate and gets a 256-bit member token; the host stores only its hash.
  3. Trusted hosts: a server remembers the hosts it approved, and accepts their invitations without asking next time.

• Make host (hand-over):

  • The current host asks a server (on the network, or one of its members) to take over. That server approves (“Become host”) and invites everyone in the request.
  • The old host and its members accept automatically: the old host asked for it, and members learn the new host’s fingerprint from the old host in their report replies.
  • Clients must pair with the new host.

• Strategy (cluster::Strategy, the host decides; members show the host’s choice from report replies):

  • Split model (default): one model across the servers. Bigger models, about single-GPU speed, needs every server up and a wired network.
  • Replicas: a full copy per server. More concurrent users and resilience, no bigger models.

  It’s colour-coded in the cluster panel (violet / blue) with an ⓘ tooltip; oppxs strategy split|replicas does the same.

  • Locked while serving: set_strategy refuses a change while this server is serving (“Stop the server before changing the strategy”), and the panel shows the other option faded with a lock.
  • Pooled VRAM (split): on a host with online members, the Models page measures fits against the sum of each server’s free VRAM (its GPU with the most free memory, as last reported), and every server’s runtime memory is added to the need (vram::split_across). A donut (VramPool.svelte) shows each server’s share and the selected model’s need. The pre-flight VRAM check follows the split plan (plan_split in the app).

• Split serving (cluster step 3, split.rs; pipeline parallel with SGLang --pp-size N --nnodes N):

  • When: the host, with the split strategy and online members, starts a model that doesn’t fit its own GPU. A model that fits runs on the host alone: faster, and it keeps serving if a member drops out.
  • Plan (split::plan): layers are shared out in proportion to each server’s free VRAM after its fixed costs (runtime memory on every server, the input embedding on the first, the output head on the last), at least one each. Layer count and embedding size come from the model’s config.json (text_config for multimodal models). Passed as SGLANG_PP_LAYER_PARTITION. SGLang takes the minimum KV capacity across ranks, so each server sizes its own --mem-fraction-static from its stage.
  • Rank 0 is the host’s normal backend on the host’s network (--network host; NCCL and torch connect back to the address each rank announces, which a bridge would hide). The gateway then binds the admin API to loopback itself (ADMIN_HOST=127.0.0.1), SGLang moves to 9096, and SearXNG is published on 127.0.0.1:9098 only.
  • Members run a worker: the host puts a WorkerOrder (run id, model, rank, partition, host:9100 rendezvous, stage) in each member’s report reply. The member starts openphalanx-worker (the same image with start-sglang.sh as entrypoint, no gateway, host network, weights read-only) with its own GPU and LAN interface (NCCL_SOCKET_IFNAME/GLOO_SOCKET_IFNAME), and reports starting/running/failed. An order without a worker, or leaving the cluster, stops it. Its log goes to the host’s Logs page prefixed [split worker]. The worker starts in its own task, so the member keeps reporting while it builds the image (the first time, a ~16 GB download): awaited inside the report loop, the host saw no report for 12 s and paused on every first start. A member whose service restarts adopts a running worker with the order’s run id (docker label) instead of recreating it.
  • Before starting the host checks every member: the same OpenPhalanx version as the host (cluster::split_version_problem; a 0.3.0 member used to fail later, inside its worker, with “the backend image … isn’t on this machine”), NVIDIA runtime present, and the model synced (“still downloading (45%)” otherwise). Members need the backend image too; the worker fails with a clear message if it’s missing.
  • While serving: a member dropping out pauses serving; a failed worker stops it with that member’s reason. Stop clears the orders, so workers stop too. The Server page’s model pill shows the split (“rig-3090: 40 layers (…) · laptop-4090: 24 layers (…)”). The orders are saved to cluster/workers.json, so reopening the app on a running split keeps it (it used to withdraw the orders, which stopped the member’s rank and crashed rank 0); a member not heard from since the service started counts from that start, not from when it joined. Orders left from a run that is gone are dropped. While serving, the Models page counts each member’s VRAM as it was before its worker started.
  • Runtime memory per stage: each server is charged runtime memory for the weights it holds (vram::runtime_overhead), not the whole model’s. Charging the whole model’s to each left Qwen3.6-27B with a 30.9k-token KV cache at a 32k context and 5–6 GB unused per GPU. /v1/info also reports min(context, SGLang's KV tokens) as context_length, so clients budget against what the server can actually hold.
  • Same recurrent-state pool on every rank: --max-mamba-cache-size 16 (docker::SPLIT_STATE_SLOTS; read only by models with linear-attention layers). SGLang otherwise sizes the pool from each GPU’s free memory, and rank 0 hands out slot numbers from its own: with 23 slots on the 3090 and 20 on the 4090, Qwen3.8-27B died mid-request with “index out of bounds” (IndexKernel.cu) on the 4090, usually on the first request of a new conversation, and took the backend with it (three times in one afternoon of agent runs). With 16 on both, 20 fresh conversations and the edit eval ran without a crash.
  • Same settings on every rank: --chunked-prefill-size 2048 (docker::SPLIT_PREFILL_CHUNK; SGLang otherwise picks 4096 on a 24 GB card and 2048 on 16 GB, and the smaller rank crashed reshaping a 4096-token chunk), and the model’s dtype override if the catalog has one.
  • Ports between the servers: 9100 and the next 16 on the host, plus NCCL’s ephemeral ports both ways; keep the servers on one LAN without a firewall between them.
  • Trusted LAN only: the rendezvous (9100) and NCCL/gloo traffic aren’t authenticated, and SGLang passes objects between ranks with pickle (broadcast_pyobj in sglang/srt/utils/common.py). Anyone who can reach those ports while a split model starts or runs can potentially execute code on the servers. That’s inherent to multi-node SGLang; the cluster panel’s Split tooltip says so.
  • Planning by hand: cargo run -p openphalanx-core --example split_plan -- rig-3090=22.1 laptop-4090=15.5 prints the plan for the selected model and context (the host first, free GiB per server).
  • Verified by hand (containers with the app’s arguments, 3090 + 4090 laptop on gigabit Ethernet): Qwen3-8B-AWQ 20/16 layers, about 100 tokens/s, a 6.1k-token prompt in 2.3 s; Qwen3.6-27B-AWQ 42/22 layers at 32k context (18.6 of 24 GB and 11.0 of 16 GB used), correct on a 16.8k-token prompt in 13.7 s. The full app flow (Start in the GUI on the host, worker orders, Stop, the app reopening mid-run, a member’s service killed and restarted) was run on v0.5.0: about 4 minutes to serving, including the member building its image.

• Model sync:

  • The cluster’s model follows the host’s selected model (set_desired_model on select, on every pre-flight check and on Start; repo plus pinned commit; local-folder models are skipped). The host sends it to members in every report reply.
  • A member that already has it (the app’s models folder or the HF cache) reports “ready”; otherwise “missing”. It downloads only after the user agrees on the host: the Server page asks “laptop-4090 doesn’t have … (20.4 GB). Download it there?”, and approve_download lets members fetch it (verified, resumable), reporting progress and errors (retried). Choosing another model resets the approval.
  • Start waits up to 8 s for members to report on a newly chosen model before checking the split, so a fresh selection doesn’t fail on a stale report.
  • The Machines table shows each member’s status; headless, use oppxs model <catalog-id>|none.
  • Verified: the 4090 downloaded Qwen2.5-Coder-7B-AWQ (5.2 GB, about 92 MB/s, verified, marker written), and reported “already on this machine” after a restart.

• Member logs: members queue log lines (their backend’s docker logs since the last report, plus cluster events such as joins and downloads) and send up to 400 per report. The host keeps 3,000 per member. The Logs page has a machine dropdown; headless, use oppxs logs <member>.

• One controller (start guard): whoever presses Start first becomes the controller; the other side can’t start too.

  • Cluster::claim_start runs before pre-flight. A standalone server or host just marks itself serving (refused while it is handing control to someone). A member first takes the host role over through POST /cluster/v1/take-over, which the host refuses with 409 while it is serving or already handing over. Both checks happen under one Control mutex on the host, so two simultaneous presses can’t both win.
  • The app’s monitor keeps set_serving in step with the backend (starting or running). The host sends host_serving in report replies; a member then has locked_by_host in its view, and its power button turns violet with a lock and “ is running the cluster”. All it can do is leave the cluster (or close the app).

• Split pause: with the split strategy, a host that is serving stops its backend when a member has said goodbye or hasn’t reported for 12 s (dropped_members), and the Server page shows “Paused: dropped out of the cluster…” (state paused, red). Start again once it’s back, or remove it. Members send POST /cluster/v1/bye when the app exits (RunEvent::Exit) or oppxs gets Ctrl-C, so a clean exit pauses at once.

• Members report to the host every 5 s (/cluster/v1/report): inventory (CPUs, RAM, GPUs with load and temperature, Docker/NVIDIA runtime) and, if their backend runs, what it serves (model, gateway and inference metrics). The host shows a member offline after 20 s without a report. A member that gets 401 (removed, or the cluster was dissolved) becomes standalone.

• App:

  • Cluster panel on the Server page: pending invitations and host requests (Approve/Decline, with the sender’s fingerprint), this server’s role with Rename/Leave/Dissolve, and the servers on the network (Add to cluster / Make host).
  • In this cluster cards: the host has a red Remove on each member, a member has Leave on its own card. A removed server shows up under “Not in this cluster” again, and Add to cluster brings it back without asking there (it trusts the host it approved before).
  • Machines table: every server, with Make host/Remove for members.
  • Charts show one machine at a time (a dropdown); the tiles sum the cluster (six of them, always split evenly: 6 in a row, else 3 + 3, else 2 + 2 + 2, by the dashboard’s own width through a container query; auto-fit left GPU memory alone on a second row at in-between widths). Eight panels (Dashboard.svelte, sampled every 2 s into metrics.svelte.ts, 10 minutes kept): generation throughput, requests in flight, time to first token, queue depth, client requests, prefix cache hit, KV cache in use, GPU load and temperature.
    • Time to first token (p50, p95, p99): measured by the gateway, from a request’s arrival to the first generated token it sends back (Metrics.note_ttft), so waiting for a slot and the search router are included, as the client experiences it. Exact percentiles (nearest rank) over the streamed requests of the last 5 minutes (TTFT_WINDOW_S, at most 2,000); non-streamed requests have no first token and aren’t counted. SGLang’s own time_to_first_token_seconds histogram wasn’t used: its buckets are coarse (0.1, 0.2, 0.4 s…) and it starts counting after our queue.
    • Queue depth: waiting is requests the gateway holds back for a slot (4 in flight per device, 8 server-wide; requests_waiting), queued is SGLang’s own queue (num_queue_reqs). Both are in GET /admin/status under gateway (requests_waiting, ttft_p50/p95/p99, ttft_samples); a gateway from before this reports none and the panels stay empty.
    • Verified on the 27B server with 6 streamed requests at once from one device: 4 in flight and 2 waiting, then p50 0.32 s with the two that waited at 3.8 s and 5.0 s (p95/p99); a non-streamed request added no sample. The panels themselves were type-checked, not looked at in the app.

• Headless control: oppxs status | servers | invite <name> | make-host <name> | approve [<name>] | decline <name> | remove <name> | leave | dissolve. These talk to the running server on 127.0.0.1:9094 with a per-start token in control.token (0600). Use run --name <name> to set the display name; it’s kept.

• Verified between the 3090 (app and headless) and the 4090 laptop (headless), on the wired LAN:

  • discovery in both directions; a non-server machine (the HP laptop) never listed;
  • invite and approve; membership surviving restarts of both;
  • reports with the 4090’s GPU;
  • hand-over (the 4090 became host, and the 3090 moved over by itself);
  • dissolve; a stopped server gone from the list within 12 s.

Connectors (MCP) and the headless browser (Playwright)

Part of the developer notes; the index is in CLAUDE.md.

• Connectors (MCP) (client/oppx/src/mcp.rs, oppx_chat/mcp.py): tools from MCP servers, for the agent engine. They live on the client, like everything that executes: a connector is a program on the laptop or a URL, used with the user’s own credentials. The server runs none of it and never sees the definitions.

  • Add, view, remove:

    • oppx mcp add <name> [-e KEY=VALUE] -- <command> [args], or oppx mcp add <name> --url <https://…> [-H 'Name: value']; --project saves it for the repository, --force replaces one.
    • oppx mcp list (or just oppx mcp) shows yours and the repository’s; oppx mcp remove <name> drops one.
    • In the chat: /mcp lists them with tool counts and what they cost per request, /mcp tools <name> shows a connector’s tools, /mcp add … and /mcp remove … run the same oppx mcp commands and reconnect, /mcp reconnect retries one that failed. The welcome box and /status name them, and /context has a Connectors line.

  • Example, Linear (also in README.md): remote server, streamable HTTP, so no OAuth in oppx; pass a personal API key as a header, or let mcp-remote do the OAuth login over stdio:

    export LINEAR_API_KEY=lin_api_...
    oppx mcp add linear --url https://mcp.linear.app/mcp -H 'Authorization: Bearer ${LINEAR_API_KEY}'
    # or, browser login (needs Node on the client):
    oppx mcp add linear -- npx -y mcp-remote https://mcp.linear.app/mcp
    

    The single quotes matter: ${LINEAR_API_KEY} is expanded by oppx at start (see below), so the key stays out of mcp.json. Not yet tried against Linear’s live server.

  • One common format: the mcpServers block every MCP client reads (command/args/env, or type: http with url/headers), so a connector’s install snippet works unchanged. Yours are in mcp.json next to config.json (0600, it may hold tokens; OPPX_MCP_CONFIG overrides), a repository’s in .mcp.json at its root. ${VAR} and ${VAR:-default} in any value are read from the environment at start, so .mcp.json can stay free of secrets. On a shared name yours wins. oppx writes these files; the frontend only reads them.

  • A repository’s connector needs consent: it is a command from a file in the repo, so cloning a repo must not be enough to run it. The chat asks once per repository and definition (the hash is kept in ~/.local/state/oppx/mcp-approved.json; a changed command asks again); until then it shows as “waiting”. Print mode can’t ask, so it stays waiting. oppx mcp add --project counts as consent.

  • Transports: stdio (one JSON-RPC message per line; started in its own session, so Ctrl-C in the chat doesn’t reach it; its stderr goes to ~/.local/state/oppx/mcp-<name>.log and the last line is shown when it fails) and streamable HTTP (answers as JSON or as an event stream, session id kept). Not supported: the older SSE transport, OAuth (pass a token with -H), and resources, prompts and sampling. The client is our own (standard library, plus the engine’s httpx for HTTP), not the mcp package, which the engine’s environment doesn’t have.

  • Arguments are the tool’s own fields: the arguments call sees the action that chose the tool as a sentence (“I’ll call linear.save_issue now.”), and the arguments used are kept with the result, not merged into the action. Merged, they taught the model to write the whole action as the arguments ({"enough": false, "tool": …, "arguments": {"id": "CHO-7", …}}): Linear’s save_issue schema isn’t one SGLang’s grammar takes, so the unconstrained retry copied that shape and Linear refused it eight times. Such output is unwrapped anyway (clean_arguments: the inner arguments object, and our action’s keys dropped unless the tool declares them), and a call that failed is refused when made again with the same arguments, with its error, instead of running again.

  • How the model calls one (model-agnostic, no tool-call parser): the tools are never sent as the OpenAI tools parameter. The system prompt lists each as one line (<name>.<tool>: the first sentence of its description, at most 140 characters), and the names join the tool enum of the action schema. After the model picks one, a second call writes the arguments, constrained by that tool’s own inputSchema (response_format json_schema, $schema dropped) and shown the full description. If the server’s grammar refuses the schema (a 400), it is asked again unconstrained and checked here (a JSON object with every required key). A tool without arguments skips the second call.

  • Asking, as for commands (Permissions.judge_tool/answered_tool): a tool its connector marks readOnlyHint runs without a question, like a command that only reads. Any other tool is asked once: y (this time), a (always, this tool), c (always, every tool of that connector: the rule linear.*), N, or text that declines and is passed on as what to do instead. Rules are saved per project with the command rules (/permissions), and a deny rule (linear.save_issue, or linear.*) wins over read-only. Print mode declines what it would ask. Plan mode and /ask offer only the read-only tools. With Linear, reading an issue and listing its statuses used to ask twice before anything happened; now only the write asks. Results are marked as untrusted reference text and cut at 12k characters. The same call with the same arguments twice in a turn is a repeat. A connector call counts as the work that was asked for: “open an issue” doesn’t get the “nothing was changed” nudge or opening.

  • Context cost, and the warning: the tool list is in every request (about 50 tokens a tool in the one-line form; the schema is only sent for the tool being called).

    • oppx mcp add starts the connector once (python -m oppx_chat.mcp probe, when the engine is installed) and prints its tools and tokens, with the share of the server model’s context (/v1/info, 4 s limit). One that doesn’t start is saved anyway, with the reason.
    • It warns when the context is 32k or less (SMALL_CONTEXT), or when the list takes 10% of it or more (HEAVY_SHARE): “the server’s model has a small context window (32k tokens). This connector’s tool list takes about 212 tokens of every request (0.6%), and each tool result uses more. Keep only the connectors you need…”.
    • The chat gives the same warning after /mcp add (with the total of all connectors), and at start only for a heavy list, so it doesn’t repeat every session.

  • Prefix cache: connectors start before the first request (in parallel, 30 s each at most), so their tools are in the system prompt from the start. A /mcp add, remove or reconnect that changes the list rebuilds it: one uncached request, and the chat says so.

  • When a connector stops working (mcp.diagnose, ConnectorDown; a Linear login that expired showed only as “linear.save_issue failed: …”, again and again):

    • Why, from its error: it needs a new login (HTTP 401/403, “unauthorized”, “token expired”; or an mcp-remote connector that stops answering, since it waits silently for the login in the browser), its program isn’t installed, its program stopped, it can’t be reached, it didn’t answer in time. Matching is narrow on purpose: a tool that refuses its arguments (“the login field is required”) is not a lost login, and one slow call doesn’t take a connector down.
    • The steps back, per kind and per way it was added: for mcp-remote, /mcp reconnect <name> (which waits 3 minutes for the browser login) and, if that fails, deleting ~/.mcp-auth and coming back with oppx -c; for a token in a header, which environment variable it is read from and the oppx mcp add … --force line for a new one; for a missing program, what to install; otherwise the reconnect and where its log is.
    • In a turn (Hub.call): a program that stopped is started again once and the call repeated, without anyone noticing. Anything else marks the connector failed and raises ConnectorDown: the user gets a notice with the numbered steps at once, the connector’s tools leave the action list, the model is told not to retry or work around it, and the reply’s facts tell it to open with what wasn’t done and to pass the steps on, ending with “once it is back, ask me to continue”.
    • /mcp shows each connector that isn’t running with the reason and the steps; so does the line at start. /mcp reconnect <name> (also /mcp auth <name>) starts that one again with LOGIN_TIMEOUT (180 s) instead of the 30 s every connector gets at start, and says to continue when it is back; plain /mcp reconnect restarts all of them as before.
    • Tested: unit tests for the diagnosis of each kind, a killed connector restarted for the call, a lost login taking the connector down and one reconnect bringing it back, and the agent’s side (the notice, no second call, the steps in the reply’s facts). In the real chat with two broken connectors (one that never answers, one whose command doesn’t exist): the start line and /mcp showed the reasons and steps. Not yet tried: a real expired Linear login through mcp-remote, and /mcp reconnect linear opening the browser.

  • Tested: scripts/test_connectors.py (16 tests, no server needed): the client against a stdio server it writes itself, and the agent loop with a scripted model. By hand against the running server (Qwen3.8-27B-INT4, 32k context), with a test stdio connector and a test HTTP one (event-stream answers, session id, bearer header, 401 without it):

    • the real chat in a pty: the welcome line, /mcp, /mcp tools, /mcp remove, then “Save a note titled “Standup” with body “9am daily”…“: it picked notes.add_note, wrote {"title", "body"} from the schema, asked, and the note was saved; /mcp add reconnected and gave the small-context warning;
    • five headless turns: a note with an optional enum argument, a tool without arguments, a declined call reported as not done, and plan mode offering only the read-only tools; 2–9 s for each of those turns.
    • Not yet tried: published connectors (GitHub, filesystem), schemas with anyOf or $ref against SGLang’s grammar, and small models (Qwen2.5-Coder-14B).
• Browser (Playwright) (client/oppx/src/browser.rs, oppx_chat/browser.py; CHO-6): a headless browser the agent can drive to open pages (a local dev server or any URL), click, type, and read the page, its console errors and network requests. Like everything that executes, it runs on the client, not the server: it has to reach the user’s localhost, and it costs no GPU (one headless Chrome, a few hundred MB of RAM while a session uses it).
  • What it is: Playwright’s MCP server (@playwright/mcp, pinned to browser::MCP_VERSION, 0.0.83), run as a connector oppx starts itself (Hub.start, kept across /mcp reconnect; a user connector named playwright is ignored). Flags: --headless --isolated (profile in memory), --image-responses omit, --codegen none, --console-level warning, --file-paths absolute, output under ~/.cache/oppx/browser/<session>. Uses the system’s Google Chrome when there is one (--browser chrome), else Playwright’s Chrome for Testing in <engine>/playwright/browsers.
  • Install: on first use (the first “yes” in a chat runs oppx browser install), or oppx browser install / on: npm install --prefix <engine>/playwright (about 20 MB; 0.4 s from npm’s cache here), plus about 170 MB of Chrome for Testing only without a system Chrome. Needs Node.js 18+ (node on PATH); without it the chat says so and the model carries on. oppx --update updates it only when it was installed before.
  • Asked once a session, off by default: the agent has a browser action (one line in the system prompt, only when the session can ask). Choosing it asks “Start a headless browser (Playwright) for this session?”: y, a (always: oppx browser on, saved as browser_auto in config.json), N, or text passed to the model. A no holds for the session: the action leaves the list and the model is told not to ask again. With oppx browser on it starts with the session, before the first request, without asking. Print mode can’t ask, so it only has the browser when it is on. /browser shows the state; /browser start | stop | on | off.
  • Once started: its tools join the connector list (21 of its 25, about 750 tokens a request; the system prompt is rebuilt once). They run without a question for the rest of the session (Permissions.allow_for_session, never saved), except browser_evaluate (JavaScript the model wrote), which is asked every time; deny rules still win. Never offered: browser_run_code_unsafe, browser_file_upload, browser_drop (they reach the user’s files or run arbitrary code) and browser_take_screenshot (images are left out of results; the snapshot is the page as text).
  • Results: the page snapshot (an accessibility tree with refs such as e3 to click) and new console entries, which the server writes to files, are put into the result (browser.inline_files), since the model can’t open files outside the repository.
  • Repeats: the browser is a stateful connector (Hub.stateful). The same click twice is two clicks (the connector repeat guard refused the second click in the first live run); a read-only call counts as a repeat only when nothing changed the page since; the same page-changing call more than 5 times in a row is refused as a loop.
  • Choosing it: the model decides; when the browser is offered, a message that names a URL (localhost:5173, http…) or reads like it needs a running page (dev server, console errors, “actually shows”, blank page…) gets a one-line note that grep and read can’t see a running page (browser.hint). Measured with scripts/probe_browser.py (a small React app, 10 messages that need the page and 10–14 that don’t, the first three actions of the real loop) on split Qwen3.8-27B: without the note 15/20, with it 18/20, and after widening it 48/48 over two runs with 24 cases (no false starts on “add a footer to the home page component” and the like).
  • Tested: scripts/test_connectors.py (Browser: offered, started, its tools offered and hidden ones not, a no is final, a reconnect keeps it, inlining, the hint, repeated clicks), scripts/test_permissions.py (session allow); the real server through Hub.start (0.17 s to connect, navigate 1.9 s with Chrome starting, click by ref, console and network requests); and a real chat in a pty on Qwen3.8-27B: asked, y, installed on first use into the real engine folder, opened a local page, clicked Add twice, answered “2” with the three console errors. Not yet tried: the Chrome for Testing download (this machine has Chrome), macOS, and a real dev server (Vite) instead of a static page.

Documents and pictures: reading and making them

Part of the developer notes; the index is in CLAUDE.md.

What the agent handles

• One layout for every multi-part document: each part starts with a --- page 3 of 20 --- line (sheet, chapter), so read line ranges work as in any file, outline lists the parts and headings, and grep searches the text (git grep skips binary files, so grep_documents adds up to 30 documents per search). What read shows for a workbook can be passed back to create unchanged: the --- sheet Name --- lines become sheets.
• Text is NFKC-normalized, so the “fi” ligature in PDFs becomes “fi” and grep finds “file”.
• Documents are read-only as text: read_text (what edits use) refuses them with “create it again in full”. Converted text is cached per (path, mtime, size).
• The model writes text, never a binary file. create on a writable extension adds a line to the prompt saying what to write (writers.hint), and writers.write builds the file. The result says what was made (“Word document, 11 blocks”, “PDF, 1 page”) and asks the model to read it back to check. /undo restores a replaced document from its bytes (Hooks.original); the diff shown is of the text.
• Formulas: a number stays a number, =… becomes a formula. Files we write have no cached results, so the reader shows the formula itself when there is no stored value.
• Fonts for PDFs: DejaVu, Liberation, macOS Arial or Windows Arial, whichever is found first (Unicode). Without any of them fpdf2’s built-in Helvetica is used and text outside Latin-1 becomes ?.

Pictures and vision

• Set when the model loads: the gateway’s watch_inputs task reads SGLang’s /model_info as soon as SGLang is ready (has_image_understanding, has_audio_understanding; inputs.accepted), logs [gateway] the model takes: image, text, and keeps it for the container’s life. It is one local request after loading, so startup is unchanged: 213 s for the 14B and 218 s for the 9B, against 210–220 s before. No catalog flag is needed: SGLang loads a model’s vision part by itself (log line “Multimodal data loading enabled”) and knows. 28 of the catalog’s 84 models have a vision encoder by their config (Qwen3.5+, Gemma 4, Mistral 3, Llama 4, Kimi K2.6+, …); the coder models are text-only.
• Shown: /v1/info has vision and inputs, and so does /admin/status. oppx prints “sees pictures” or “text only” on its connect line and passes the flag to the chat as OPPX_VISION.
• Guarded at three levels, each before any model call:
  • Gateway (docker/server/inputs.py, for every client): a request whose messages carry a part the model can’t take gets HTTP 400 unsupported_input with the reason (“The loaded model (…) can’t take pictures: it takes text only…”). Video parts are refused for every model, and so are unknown part types. Measured: 8 ms for the refusal on the 14B; the image passed on the 9B; video got 400 in 5 ms on both. Tests: scripts/test_inputs.py.
  • Handing a file to the chat (@file, /add; commands.cannot_take): a picture for a text-only model, a video or a presentation is refused with the reason and “Nothing was sent”. /add keeps the other files. Punctuation after a mention (“@shot.png?”) no longer hides it.
  • Naming a file in a sentence: a one-line notice before the turn, not a stop, since “rename logo.png” is fine. The agent’s own read then refuses with “the loaded model can’t see pictures. Tell the user”, so the model doesn’t guess.
• How a picture is sent: shrunk to at most 1280 px on the longest side, JPEG at quality 85 (transparency on white), attached to the tool-result message under _images. Message content stays a string everywhere. Only _clean() turns it into [text, image_url…] parts, and only for messages that weren’t cut to save space.
• Request size: the gateway accepts 4 MiB (MAX_REQUEST_BYTES) and every request resends the whole conversation. So each request carries the newest pictures up to 2.5 MB (IMAGE_BYTES). Older ones are left out, with a note to read them again.
• Token estimate: pixels / 784 + 16 per picture, counted on top of the text estimate. It shows on /context as “Pictures”.

Starting a turn

Found on the 3090 with Qwen3.5-9B: without these, a 9B model answered “I can’t find a travel policy PDF” without looking, said “Let me read the file” and stopped, and wrote a Python script to make a PDF or PNG.

• A named file is opened up front, pictures and videos too. That way the model sees the picture, or the refusal. A bare file name counts when only one file in the repository has it.
• Documents the request is about (_documents_for): files named like its words (“travel policy” finds docs/travel-policy.pdf), or every file of a kind it mentions (“the PDF”, “the screenshot”) when there are at most 10.
• Files to make (_documents_to_make): “summary.docx: use create on summary.docx itself; you write Markdown… No script is needed”.
• Router examples: the ask/edit prompt (routing.INTENT_PROMPT) now says that asking for a file to be made is “edit” whatever goes in it. Before that, “Create summary.docx: a one-page summary…” was routed to “ask” and the summary came back in the chat. probe_routing.py has 7 more intent cases (documents and pictures, worded differently from the prompt’s examples).

Engine packages

engine::EXTRAS installs pypdf 6.19.0, fpdf2 2.8.9 (which pulls in fontTools and defusedxml) and resvg-py 0.5.0 next to Aider (uv tool install --with …). A private engine without them counts as not current, so oppx --update or the next launch installs it again. Without the extras, PDFs are still read through pdftotext when it is installed, and making PDFs or pictures, or seeing a scan’s pages, says “the user can add it with oppx --update”. Checked: the exact install command in a scratch OPPX_ENGINE_DIR gives the four dist-info folders find() looks for.

Measured (RTX 3090, 2026-10-16)

• Vision through the gateway: Qwen3.5-9B-AWQ (Qwen3_5ForConditionalGeneration, the same class as Qwen3.6), backend image 0.7.4, 32k context, MEM_FRACTION_STATIC 0.85. Weights 8.5 GB. Through oppx proxy and the gateway:

  • an 800×500 invoice: 400 image tokens, correct, 0.97 s;
  • the same with a JSON schema (constrained decoding and an image together): correct fields, 1.04 s;
  • a bar chart: highest and lowest bar right, 0.31 s;
  • the invoice again: 384 tokens cached, so the prefix cache covers images.
  • A scanned manual page: 962 image tokens at 100 dpi (827×1170), 2,145 at 150 dpi. Both answered correctly, so 1280 px is enough.

• Qwen3.6-27B-AWQ doesn’t fit the 3090 alone with its vision part. Weights take 19.07 GB, which leaves 3 GB. At 0.85 the hybrid (mamba) state cache had no room. At 0.93 with 4 requests it still had none. At 0.95 with 1 request, bf16 state and 8k context, it ran out of memory in CUDA graph capture. It runs split across two machines.

• End to end with oppx -p (Qwen3.5-9B, scratch repo):

  • read a PDF, a Word file and a PNG receipt: all answered correctly;
  • refused a video, without guessing, after the refusal text said so;
  • made summary.docx, expenses.xlsx (formulas computed by LibreOffice), travel-rules.pdf and badge.png, all through create.

• The files open in real software:

  • LibreOffice converts the generated .docx, .odt and .rtf to PDF with headings, nested lists, numbered lists, tables, code, bold, italic, é and €;
  • it opens the .xlsx and computes its formulas;
  • the fpdf2 PDF renders the same content.

• Tests: scripts/test_agent.py classes Pdfs and Documents cover:

  • the round trip of every format through writers and documents;
  • workbooks with sheets, numbers and formulas;
  • SVG drawn to all four picture formats;
  • a GIF’s first frame;
  • pictures attached only for a model that sees, and dropped when their message is cut;
  • the 2.5 MB budget;
  • video and presentation refusals;
  • RTF unicode;
  • the turn-start helpers.

  The fpdf2, resvg and pypdf parts skip when the engine lacks them.

Open

• The vision flag and the gateway’s check need the new gateway.py and inputs.py in the image: the app’s image tag changes with them (they are in BUILD_FILES), so the app rebuilds it on the next start. Against an older server the client sees vision: false, and pictures are refused.
• The VRAM estimate (vram.rs) doesn’t count a vision encoder or image tokens. The 9B fit with room to spare; the 27B shows the encoder matters on a 24 GB card.
• Excel dates come back as Excel’s day numbers (number formats aren’t applied). Charts, images and comments inside Office files aren’t read.
• EPUB tables read back as one line per cell row with blank lines between rows.
• Only Qwen3.5-9B was tried end to end. Gemma 4’s vision path through SGLang is untested.

Backend without the GUI, and the headless server (oppxs)

Part of the developer notes; the index is in CLAUDE.md.

Run the backend without the GUI

Useful on a headless machine or while developing the backend. Build the image from the repo (the app does the same from the copy it carries), then run it; the container serves the model in MODEL_PATH (a Hugging Face repo id or a path inside the container):

docker build -f docker/Dockerfile.server -t openphalanx-backend:dev docker/
export ADMIN_TOKEN=$(openssl rand -hex 32)
mkdir -p ~/.local/share/openphalanx/backend-state

docker run -d --name openphalanx-backend --gpus all --ipc=host \
  -p 9090:9090 -p 127.0.0.1:9091:9091 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -v ~/.local/share/openphalanx/backend-state:/state \
  -e ADMIN_TOKEN \
  openphalanx-backend:dev

# When the admin API answers, issue a pairing code:
curl -s -X POST -H "x-admin-token: $ADMIN_TOKEN" http://127.0.0.1:9091/admin/pairing

# Readiness (sglang turns "ready" after ~4 min):
curl -s -H "x-admin-token: $ADMIN_TOKEN" http://127.0.0.1:9091/admin/status | jq .sglang

Optional environment variables: MODEL_PATH, CONTEXT_LENGTH, MEM_FRACTION_STATIC (default 0.85; the GUI computes it from free VRAM instead), SGLANG_EXTRA_ARGS. Without APP_WATCHDOG_S (which only the app sets) a backend started this way runs until you stop it. The GUI shows a container started this way as “running outside the app”, and can stop it.

Headless server (oppxs)

The server as a command, for machines without a desktop and for scripting. It comes inside the .deb (/usr/bin/oppxs, with a disabled systemd user unit) and as a static binary in each release; install-server.sh --headless installs only that, on any Linux, x86-64 or arm64.

oppxs run [--name NAME]                 # the server itself; a systemd user unit keeps it running
oppxs models [--all]                    # catalog: download size, VRAM need at the current context, fit, state
oppxs download <model>                  # verified, resumable; a catalog id or a unique part of one
oppxs use <model> | oppxs context <n>   # what `start` serves (the app's settings.json)
oppxs long-context on|off            # let models with a YaRN mode go past their native window (off by default)
oppxs port [admin|control] [<n>]        # show or move the client (9090), admin (9091) or control (9094) port
oppxs ports                             # every port the server uses and who holds it
oppxs web-search on|off                 # the private SearXNG for clients (on by default)
oppxs gpus [all | 0,2]                  # this machine's GPUs and which the server may use (all by default; several: experimental)
oppxs start | stop | status             # start follows progress until it serves or fails
oppxs pair | devices | revoke <device>  # pairing code with the fingerprint; paired clients
oppxs logs [-n LINES] [<member>]        # the backend's log, or a cluster member's
oppxs report [what happened]           # a problem report (machine, backend, kernel log's signs of trouble) and a link that opens a GitHub issue
oppxs servers | invite | make-host | approve | decline | remove | leave | dissolve | strategy | model

• How it’s built: core::service::Service is the orchestrator without a UI: start (cluster claim, pre-flight, which refuses a model with broken files, then server::start), stop, status (crash detection with diagnose_crash, adopting a backend that already runs, set_serving for the cluster), pairing and devices through the admin API. oppxs run runs it with the cluster service and a monitor that looks every 2 s, which also feeds the gateway’s watchdog. The app still has the same logic in its own command layer (app/src-tauri/src/lib.rs) and is to move onto Service.
• Control API (127.0.0.1:9094, bearer token from cluster/control.token, 0600): GET /control/server, POST /control/server/{start|stop|pair}, GET /control/devices, DELETE /control/devices/{id}, plus the cluster routes. models, download, use, context and logs without a member need no running server: they read and write the same settings and model folders as the app.
• Stopping it stops the model (Service::shutdown on Ctrl-C or SIGTERM), like closing the app; a killed oppxs leaves it to the watchdog. So systemctl --user restart oppxs reloads the model.
• Service: systemctl --user enable --now oppxs; loginctl enable-linger $USER starts it at boot without a login. The user needs Docker access (the docker group).
• Same rules as the app’s slider and checkboxes (in oppxs itself, main.rs): context and long-context are refused while a backend runs; context refuses more than the selected model’s maximum (pointing to long-context on when that would allow it) or more than fits the VRAM free now; long-context off brings a longer context back to the native window; start --force first sets the context to the longest that fits, or the 2k minimum; models marks damaged copies “broken: download it again”; status shows the context a start would serve, never above the model’s maximum. Tried against scratch settings (XDG_CONFIG_HOME), without starting a backend.
• Limits for now: it serves models that fit its own GPU and can be a cluster member; hosting a split model is started from the desktop app (plan_split lives there). The commands talk to oppxs run only, not to a running desktop app, and the two can’t run on one machine at the same time (ports 9092–9094, shared state).
• Tested on the 3090 (scratch settings, Qwen2.5-Coder-14B-AWQ at 16k): run, start (3 min 51 s to serving, image build included), status, pair and a real oppx pair and oppx status against it, devices, revoke (the client is then refused), logs, stop (2.8 s), a second start, and SIGTERM to the daemon (it exited in 3 s with both containers gone and the GPU’s memory free). The backend carried APP_WATCHDOG_S=60. Not yet run: the static musl build, the .deb with oppxs inside and install-server.sh --headless (they need a release), and oppxs as a cluster host.

Model catalog and VRAM estimate

Part of the developer notes; the index is in CLAUDE.md.

Model catalog and VRAM estimate

• Which build: for each size, the smallest build the pinned SGLang runs on Ampere and Ada: official 4-bit (AWQ, GPTQ, W4A16) first, then a community 4-bit build, then FP8, then the publisher’s 16-bit build. A community build must come from a known quantizer (TheBloke, casperhansen, cyankiwi, QuantTrio, Red Hat AI, TechxGenus, stelterlab, hugging-quants) or be widely downloaded; the smallest wins when several qualify. Code Llama comes from the ungated codellama org (the meta-llama copies are gated). Not used: NVFP4 (refused on the 3090: “Current platform does not support w4a4 nvfp4”), dense MXFP4 in compressed-tensors (“CompressedTensorsW4A16Sparse24 is not supported”), bitsandbytes, GGUF, and gated repos (the downloader has no Hugging Face token, and every official Llama is gated). Left out for now: Mistral Large 3 (NVFP4, or Mistral’s own format without config.json), DeepSeek V4.1 (DeepseekV41ForCausalLM isn’t in SGLang 0.5.21) and the 1.5B R1 distill (no usable 4-bit build).
• Adding models: write a spec (id, name, family, params, quant, and quantized_by for community builds), run scripts/catalog_entry.py specs.json, and paste the entries into catalog.json. scripts/catalog_entry.py --check verifies every entry’s weight size, attention shape and context against Hugging Face.
• Release date (released, shown as a year on the Models page; the full date orders “Newest first”): the repo’s creation date on Hugging Face; community quantizations take their base model’s date (base_model in the spec).
• dtype (optional, passed as SGLang --dtype to every rank): for checkpoints whose declared dtype SGLang can’t run. The 4-bit Qwen3.6 builds (cyankiwi 27B, QuantTrio 35B-A3B) declare float16, but SGLang keeps the Gated-DeltaNet state in bfloat16 and the first prefill failed with “Index put requires the source and destination dtypes match”; they use bfloat16 (verified on the 27B; the 35B-A3B by analogy).
• FP8 KV cache (--kv-cache-dtype), measured with scripts/probe_kv.py: not enabled for any model. On the 3090 (Ampere) only fp8_e5m2 runs (fp8_e4m3 fails to compile in Triton on sm_86). It gives gpt-oss-20b a 128k context on one card (199k KV tokens), but costs accuracy: recall at 24k tokens 1/3 against 3/3 with 16-bit KV, coding 4/5 against 5/5. Qwen2.5-Coder-14B gains nothing: its native window is 32k. Worth re-checking fp8_e4m3 on Ada (4090) and on Qwen3.6, as a per-model catalog field only if it passes.
• Check the architecture first: the pinned SGLang must have the model class (sglang/srt/models/ in the image). All current entries were checked against SGLang 0.5.21.
• Weights counted: only root-level .safetensors (subfolders like gpt-oss’s original/ and metal/ are skipped), and Mistral’s duplicate consolidated*.safetensors are dropped when HF shards exist. The downloader applies the same rule.
• KV cache (vram::ArchSpec), as SGLang allocates it:
  • full-attention layers × context;
  • sliding-window layers (Gemma 4, gpt-oss) × 0.8 of the context (--swa-full-tokens-ratio);
  • Gemma 4’s full layers use their own shape (global_head_dim, num_global_key_value_heads);
  • hybrid linear-attention models (Qwen3-Next, Qwen3.6) × 1.9, for the recurrent-state pool (--mamba-full-memory-ratio 0.9);
  • MLA (DeepSeek V2/V3, GLM-4.7-Flash, GLM-5, Mistral Small 4): one latent of kv_lora_rank + qk_rope_head_dim per token and layer, stored as one head of half that width (catalog_entry.py and model::arch_from_config do the same);
  • Kimi-Linear and K3 list their KV-cache layers in linear_attn_config.full_attn_layers (the rest are linear); GPTBigCode configs (Granite Code 20B/34B) use n_layer/n_head/n_embd, and multi_query means one KV head;
  • every layer_types attention kind other than sliding-window and linear counts as full (GLM-5’s deepseek_sparse_attention). DeepSeek V4’s compressed attention is counted as one 512-wide KV head on every layer, an overestimate.
• YaRN (yarn catalog field) is opt-in, off by default (settings.long_context). Qwen2.5-Coder 7B, 14B and 32B carry it at factor 2 (32k → 64k). The switch is a checkbox under the context slider (ContextSlider.svelte, shown only for a model that has the mode; set_long_context, locked like the slider, and turning it off brings the context back to the native window), or oppxs long-context on|off. With it off, server::resolve drops the field, so the model keeps its native maximum and nothing is passed to SGLang. With it on, it raises a model’s maximum to native × factor and passes --json-model-override-args (rope scaling plus max_position_embeddings, which SGLang 0.5.21 derives the limit from) only above the native window, to every split rank.

  • Why it failed: measured on Qwen2.5-Coder-7B-AWQ (3090, needle recall, start-sglang.sh as the app runs it): without YaRN 3/3 at 26k tokens; with YaRN 3/3 at 3k, 2/3 at 8k, 0/3 at 26k and 44k (Qwen’s own snippet alone the same). The cause was the override, not YaRN: with transformers 5 rope_scaling is an alias of rope_parameters, which also holds rope_theta, so replacing it dropped the model’s base and SGLang’s get_rope_config fell back to 10000 (Qwen2.5 uses 1,000,000). Checked on CPU with SGLang’s own get_config + get_rope_config in the backend image: theta 1e6 without an override, 10000 with ours or Qwen’s snippet, 1e6 with rope_theta inside the block.

  • Fix: catalog::Yarn has a required rope_theta, sent inside rope_scaling; scripts/catalog_entry.py fills it from the model’s config.json and --check verifies it.

  • Measured with the fix (3090, needle recall, 10 positions per size, real prompt tokens, start-sglang.sh as a worker runs it; the old “26k” sizes were nominal, about 0.68 of that in real tokens):

    Real tokens	7B no YaRN	7B old override ×4	7B fixed ×4	7B fixed ×2	14B no YaRN	14B fixed ×2
    4k	8/10	7/10	9/10	8/10	10/10	10/10
    8k	7/10	2/10	7/10	–	–	–
    16k	7/10	0/10	4/10	7/10	8/10	6/10
    25k	9/10	–	8/10	–	4/10	4/10
    31k	9/10	0/10	7/10	6/10	8/10	8/10
    41k			–	3/10		5/10
    47–50k			4/10	5/10		6/10
    61–67k			0/10	5/10
    101k / 127k			1/10, 2/10

    The fix restores recall inside the native window (7B: 35/50 against 40/50 without YaRN and 9/40 with the old override). Beyond it, factor 4 is unusable on the 7B above ~60k; factor 2 holds about half the needles up to 61k on the 7B and 5–6/10 at 41–47k on the 14B (whose in-window scores are 4–10/10 on this probe, so the probe is noisy). A 24 GB card holds about 48.6k KV tokens for the 14B, so YaRN would take it from 32k to ~48k, not 64k.

  • Why factor 2 and opt-in: Qwen documents factor 4, which the table rules out. The 32B and the 16-bit builds are untested, and the app’s own start with the box ticked hasn’t been run (the override it sends is the one measured here by hand). rope_theta in a spec is filled in by catalog_entry.py.

• FP8 on Ampere is counted at 16-bit size (fp8_upcast), but SGLang 0.5.21 switches FP8 to Marlin on SM 8.0–8.8 (can_auto_enable_marlin_fp8), which keeps 8-bit weights. Worth measuring on the 3090 (dense and MoE) and dropping the upcast if it holds.
• Validated on hardware: Qwen2.5-Coder-14B-AWQ and gpt-oss-20b (MXFP4) on the 3090, and Qwen3.6-27B-AWQ split across the 3090 and the 4090 laptop. Qwen3.8-27B (Red Hat AI’s W4A16 only, the smallest build) has the same config.json as Qwen3.6-27B apart from the transformers version, and it declares bfloat16, so it needs no dtype override; it hasn’t been started yet. The rest is not yet validated; a model matrix run (routing probes, scripts/eval_edits.py, tokens/s) is still to do. MXFP4 (gpt-oss) is counted at 4 bits, since SGLang’s Marlin and Triton kernels keep it packed on Ampere.

oppx agent engine: actions, guards against stuck turns, speed, truthful replies, context

Part of the developer notes; the index is in CLAUDE.md.

Documents and pictures (what read, outline, grep and create do with PDF, Office files, EPUB, pictures and videos) are in the documents topic.

• What runs side by side (2026-10-16):

  • The ask/edit check (routing.wants_edit) runs in a thread (app.ROUTER) while the turn opens named files, finds matching documents and ranks related definitions. Agent.turn takes it as a Future and waits only before its first model call.
  • The definitions index is warmed at session start (Workspace.warm; 0.76 s for this repository with a cold tag cache), behind a lock, so a first message during the warm-up waits for it instead of building it twice.
  • Connectors start in parallel threads under one shared deadline; joined one by one, several stuck ones used to add up their timeouts.
  • What stays sequential, on purpose: the steps of a turn (each depends on the model’s last action), and a request’s lookups (local and fast).

• agent (oppx_chat/agent.py, no UI code; app.make_agent/agent_turn connect it): the model reads the repository on demand, like Claude Code, instead of being sent whole files and a repo map with every request. Each step it picks one action as JSON constrained by a schema (model-agnostic, no tool-call parser; one schema per tool under anyOf, since with a single flat schema models put a path in query or end on a grep; a server that refuses anyOf gets the flat one): find (definitions and files by name in any spelling, from Aider’s tags plus a regex, built once per session), grep (git grep, grouped by file, code before documentation, at most 40 lines and 6 per file, the rest as counts), list, outline (Aider’s tree-sitter tags, regex fallback), read (line ranges; files up to 300 lines whole, ±6 lines around a range), plan (a short free-text plan), edit, create (a new file, or the whole of one that was read), run (asks y/N), start (a command that keeps running, such as a dev server), web_search (/v1/search), answer. A path that doesn’t exist comes back with the real ones it may have meant. Every action opens with "enough": true|false and, when false, "missing": the one thing still needed. Text payloads are then written free-form in a second call: edits as Aider SEARCH/REPLACE blocks applied with Aider’s fuzzy do_replace, the reply streamed. Calls carry oppx_utility, so the gateway’s search router stays out.

• The model decides what a message needs. The ask/edit check (routing.wants_edit) is a hint, not a gate: edit tools are always there unless the user switched them off (plan mode, /ask), so nothing has to be typed to get an answer or a change. On a message that reads as a question, the first edit or create is answered with one check (“change it only if they asked”); chosen again, it runs. A change request that names no file gets a forced plan before its first edit (every place that must change, in order).

• A follow-up is read with what came before it (found in a real session: asked to add a sort feature, the agent got stuck, then answered “Now I’ll implement the sort feature”, and “yes, go ahead.” got a second promise and no edit):

  • Cause: the ask/edit check saw the latest message alone, and on its own “yes, go ahead”, “go ahead”, “yes” and “continue” are questions (measured on Qwen3.6-27B: all four came back ask; “do it” came back edit). Every guard that pushes a turn to an edit hangs on that answer (the first move is never answer, the “you haven’t changed anything” nudge, the step with only the doing tools), so the turn ended on its first action, answer.
  • The check gets the conversation (routing.intent_input): the end of the assistant’s last reply (600 characters) and the change still owed, if any. The prompt says how to read a message in that light (a go-ahead after an offer or an announcement, “continue” or “where is the edit?” while a change is owed, an answer to a question the assistant needed: edit; a refusal, thanks, a question about something else: ask), with examples.
  • A change still owed (Agent.pending): a change request that ended with nothing changed and no connector call (stuck, interrupted, failed, or ended on a promise) is kept, and given to the check with the next messages. The turn that changes something settles it; three turns about something else, /clear, or plan mode forget it.
  • A reply that announces a change is held to it (Hooks.announces, routing.announces_change, one yes/no call after a reply in a turn that changed nothing): “Let me make the changes now” is followed by one step with only edit, create, run and start on offer, then the turn goes on as a change request. An offer (“Want me to?”) or an answer is left alone and waits for the user. Once per turn; not in plan mode; not after a stuck turn.
  • Measured on Qwen3.6-27B (scripts/probe_routing.py --reasoning, median 110 ms a call): first messages 16/16 as before; follow-ups read with the last reply 24/24 (23/24 at first: a bare “continue” with nothing before it came back edit, and the case’s expectation was changed to that, since it is what a resumed session means by it); the announcement check 16/16. Not yet run on Qwen2.5-Coder-14B or gpt-oss.
  • In a real chat (dev build, 0.7.1 server): “What would it take to add a sub function…? Don’t change anything yet” changed nothing; “yes, go ahead.” then planned, edited src/lib.rs and ran cargo check. The announcement path is unit-tested only.

• A command that keeps failing the same way (the same session: data[0] on a JSON object, sent five times, with the fix written out in “Taking stock” and “Rethink” in between):

  • The action after a refused repeat is sampled (RETRY_TEMPERATURE 0.7, Agent._sample): at temperature 0 the same conversation gives the same action again, whatever the model wrote in plain text before it. (That this was the cause there is an inference from the transcript, not something measured.)
  • A command that failed twice stays refused until a file is edited (failed_times). Another command that works still lets one that failed once run again (an install, then the build), but a print(keys) in between no longer lets a broken one-liner run again and again.
  • The refusal says what happened (“the command you just sent is, character for character, the one that failed: if you worked out a fix, it is not in it”), and so does the stuck reason (“no corrected command was ever sent, so none was rejected”): the reply had told the user the system rejected its corrected command.

• “Let me read the file first” as the text of an edit (seen in a real session: Error: no SEARCH/REPLACE block found in your reply, which began: 'Now I need to add the thumbnail… Let me read the current state of the file first.' {"tool": "read", …}): the model had chosen edit, and when asked for the edit’s text found it needed the file as its own earlier edit had left it. Nothing was written and the next step recovered, at the cost of one model call, an alarming line on screen and a failed edit on the count. Now a reply without a SEARCH block that carries a lookup as JSON (misplaced_lookup: read, grep, find, list, outline) is that lookup: it runs in the same step and the result opens “Not edited yet: you said you need to look at something first, so that was done” (LOOKED_FIRST); it counts neither as a change nor as a failed edit. Any other text without a block is still the error. For create, the same; and a reply that is a sentence plus any action is refused instead of being written into the file as its content (it would have been). Unit-tested; not yet seen in a real chat.

• A page nobody looked at (after a React change the agent ran the lint and said “Lint passes”; “did you check your code on playwright?” was the user’s next message, and they need not have known it could): when files that show in a page were changed (PAGE_FILES: .jsx, .tsx, .vue, .svelte, .html, .css …) and no browser tool ran in the turn, the note before the reply says a lint or a build doesn’t show whether it works in the page and to look (the browser action, which asks the user once, or the playwright.* tools when it runs), and the reply’s facts say to state that it wasn’t checked in a browser and that it can be. After a “no” to the browser the reply only says it wasn’t checked. Unit-tested; not yet seen in a real chat.

• Confirmed gaps closed (CHO-10), each checked against the code first:

  • An edit to a file that changed on disk is refused (Workspace.seen, mark, check_current): a digest of each file is kept from when the model last read or wrote it; edit, and create over an existing file, compare it first and answer “…has changed on disk since you read it (the user, a formatter or a command changed it). Read it again…”. The read that follows is allowed (a read after a refused edit is not a repeat) and the edit then goes in on top of the other change. The agent’s own edits update the digest, so two edits in a row need no read in between. A file /undo restored counts as changed.
  • The question before start reads “Start this command and leave it running? (it is stopped when the chat ends)” (Agent.asking, read by app.on_confirm). Seen in a real chat.
  • A web search step’s summary is “2 results: …”, not the heading of the result text; “(from your earlier search, not searched again)” when it was answered from the session.
  • Plans: one cut at its token limit ends on its last whole line or sentence (whole_sentences; a list number is not a sentence end), a request with several parts gets 600 tokens for it (PLAN_TOKENS_LONG), and plan chosen right after a plan is refused as a repeat without counting towards the two plans a turn may have.
  • Still open there: no action reads a web page; one started command can’t be stopped or restarted; a resumed session restores the messages, not what the tools returned.

• Never stuck in silence (found on Qwen3.8-27B, where a question ended with {"tool": "answer", …} as the reply):

  • Circles: an exact repeat, a re-read of lines already shown, a reworded search with the very same result, or one pattern grepped in a third file in a row counts as a repeat (not when the earlier result was cut from the window; then it may be fetched again). Two repeats force one Rethink in free text (what is known, what is missing, an action not tried yet, the top-level folders not looked in). Two more end the turn as stuck.
  • Long turns: every 10 steps without a change the model has to take stock in free text. Without it, a simple question ran to 36 steps of file-by-file greps.
  • Stuck is said out loud: the UI prints ⚠ Stuck: <reason> (Hooks.notice; repeats, or all 40 steps used), and the reply must open by saying so, then what was found, what is missing and what would unblock it. Agent.stuck holds the reason.
  • The reply is always readable: it is held back while it could still be a JSON action (the conversation is full of them) or empty; such a reply is asked for again with the first character constrained by regex, and if that fails too the harness writes its own (“I couldn’t produce an answer… What I did: …”). JSON the user asked for is shown.
  • Dot folders: Workspace.path stripped leading . and / characters, so .github/workflows/release.yml could not be opened and an edit task ended with “paste the file”.

• Speed (timed on split Qwen3.8-27B: about 70% of a turn is the model writing its steps at 36 tokens/s, 25% reading new tool output; start-up is 1.6 s for a trivial -p request):

  • A starting point: when the message names no file, definitions whose names contain its words (whole name parts, rarest first, tests left out; Workspace.related) are given before the first step. A “pairing codes” request had cost ten greps in the wrong folders.
  • Where else: a grep limited to a folder also lists the files outside it that match.
  • Several lookups in one step: a lookup may carry "then": up to two more lookups (find, grep, list, outline, read), run without another model call.
  • Fewer tokens a step: missing is at most 90 characters; a plan or stock-take at most 350 tokens. Stock-taking counts model calls, not lookups.
  • Edit text: an edit or new-file prompt answered with the JSON action again (four times in a row once) is asked for again inside the same step, first character constrained; the action reads as a sentence during that call.
  • A change request never opens with answer: with likely definitions in view the model declared it had enough and offered to make the change (“say the word”); the first move of such a turn has no answer.

• Measured on split Qwen3.8-27B (3090 + 4090, 64k context), scripts/eval_nav.py (10 questions about this repo that name no file, each reply checked for the fact and the file): 10/10 answered, 5.3 steps and 23 s on average, none stuck, no unreadable reply, nothing edited. Before the speed changes: 6.4 steps, 30 s. With a free why in place of enough/missing: 10.9 steps, 42 s, one stuck. scripts/eval_edits.py (9 tasks, three that name no file): 9/9, 17–41 s each except two at about 90 s, where the model also updated the docs and messages that state the old value. Unit tests with a scripted model: scripts/test_agent.py. Not yet run on a small model.

• Checked or said to be unchecked: before the reply to a turn that changed files, the model is told once to run the project’s quick check for what it touched (through run, so the allowlist and the y/N/a question apply), and the facts given with the reply say how that went: nothing run (“say they are not verified”), the last command after the latest edit failed (“say the check failed”), or it passed. A “Done.” for an edit nothing had been run against is what prompted it. On Qwen3.8-27B in -p mode (which can’t ask): it edited, read package.json, tried cd app && npm run check, was declined, and the reply ended “the change is unverified”. A plan or stock-take is shown without the JSON action a model tacks onto its end (strip_action).

• Guards for small models (each found in the evaluation): exact repeats and re-reads of lines already shown are refused, and after two only answer is allowed; reading a big file page by page gets a “grep instead” hint; answering from grep hits alone, or ending a change request with nothing changed, gets one nudge; before answering a change, the model is told which files it actually changed (it claimed edits it never made); a file must be read before it can be edited; an ambiguous SEARCH (matching several places) is refused with the line numbers; a failed SEARCH returns the closest real lines; an edit that changes the {}()[] balance gets a warning; an empty edit from a reasoning model is recovered from reasoning_content. Two failed web searches in a turn take the tool away (it rephrased a failing search 14 times). In plan mode or /ask the message says edits are off; without that, asked for a change, it answered that it had made it. Files named in the message are opened up front (small whole, big as an outline); /add and @file files are listed for the model. Edit replies without the filename line above a SEARCH block get it added (the agent knows the file; Qwen3.6 dropped it every time). Truthful replies: an edit counts as a change only when it applied (a refused one gets “Nothing was changed”; counting the attempt made the “files you changed” check confirm an edit that never happened). Every reply is written with the facts: files changed in this request, files changed earlier in the conversation (Agent.changed_session; /undo removes them), and, when a change was asked for and none was made, an instruction to open with that. Repeat guard (_held_back): a reply made only of sentences from the last 6 replies to other messages is held back while it streams, never shown, and written again once with the earlier copies hidden for that one request and temperature 0.7 (with them in view, Qwen2.5-Coder-14B wrote the same text again). A reply that opens with one new sentence and then repeats is still shown. Replayed on Qwen2.5-Coder-14B (the change request it gives up on, then “Did you make that change?” and three follow-ups): before, it described a change it never made and repeated that five times; now every reply says nothing was changed, and eval_edits.py stays 6/6. On reasoning models every call sends chat_template_kwargs reasoning_effort: low and enable_thinking: false (with only the first, Qwen3.6 put its whole action into reasoning_content and the turn ended with raw JSON as the answer); an empty content falls back to reasoning_content.

• Context: the conversation only grows by appending, so SGLang’s prefix cache serves most of each request; near the limit, old tool output is cut first (the latest 6 kept), then the oldest turns. The system prompt (rules, top-level folders, OPENPHALANX.md/OPPX.md/AGENTS.md) is built once per session. Sizes are estimated at 3 characters a token, corrected by the largest real/estimate ratio the server has shown this session (usage.prompt_tokens; Qwen3.6 ran ~5% over on code, and prompt + max_tokens over the window is a 400). A context refusal sets the ratio from its numbers, fits again and resends once. Once over budget it cuts down to 75% of it (FIT_LOW): cutting just enough changed an early message on every step near the limit, so each step recomputed the whole ~28k-token prompt (29–32% cache hits). /context and /cost show the agent’s own numbers.

oppx chat frontend: modules, Aider hooks, context manager, sessions, prefix cache

Part of the developer notes; the index is in CLAUDE.md.

• Chat frontend (client/oppx/frontend/: the oppx_chat package plus run.py, embedded with include_str! via agent::FRONTEND):

• Modules, lowest layer first; each imports only from the ones above it (the map is in oppx_chat/__init__.py): config (environment from oppx), term (output, spinner, Esc), render (streamed answers, diffs), context (ContextManager), mcp (connectors: the MCP client and its tools; no UI imports), agent (the agent engine; no UI imports), oppx_io (Aider’s IO, prompt, keys), routing (ask/edit check), cache (prefix-cache patches), commands (slash commands, memory), app (Aider patches, turn loop, main). A new module must also be listed in agent::FRONTEND; a unit test checks this.

• Engines (oppx --engine agent|aider, OPPX_ENGINE; default agent):

  • The agent engine is in the oppx-agent skill, its permission rules in oppx-permissions, and the app-building eval in agent-app-eval.
  • aider: Aider’s own loop, with everything below (context manager, cache patches, ask/edit routing). Slash commands, /undo, sessions, the status bar and Esc work in both.
  • Measured (RTX 3090, this repo at v0.4.1): a typical Aider request was ~25k tokens, ~60% whole files the model had mentioned and ~30% repo map, with 8–19 s to the first answer and a 42% cache hit rate. The agent’s questions use 2–14k-token requests with 76–93% served from cache, answering in 2–6 s. Edits (scripts/eval_edits.py, 6 tasks checked by compiling and testing, 3 runs): Qwen2.5-Coder-14B-AWQ agent 18/18, Aider 15/18 (Aider can’t do the multi-file rename); gpt-oss-20b agent 5/6 three times, Aider 4/6. Split Qwen3.6-27B-AWQ (3090 + 4090 laptop, v0.5.0): eval_edits.py agent 6/6 (24–52 s a task); a 10-turn bench_agent.py session that fills the window went from 3 failed turns and a 61% cache hit rate to none and 89% with the fixes above.

• How it runs: oppx writes the package to ~/.cache/oppx/frontend-<version>-<hash>/ and runs run.py with the Python of the Aider venv (the python next to the real launcher, or its shebang), so aider, prompt_toolkit and rich are importable.

• How it hooks into Aider: it sets aider.main.InputOutput to its own OppxIO, and base_coder.WaitingSpinner to its Thinking spinner. It calls aider.main.main(argv, return_coder=True), then loops over coder.run_one() itself, handling SwitchCoder the way Aider’s main loop does.

• What it changes on screen:

  • It hides Aider’s announcements, URLs and Tokens: lines.
  • Edit blocks are hidden while they stream, and each turn ends with a Claude-style diff (⏺ Update(file) plus colored lines).
  • Edit-retry noise collapses to one line.
  • Routine confirmations are auto-accepted. Commands the model proposes, and going over the context window, are asked.

• Aider version: it relies on Aider internals, so it’s tested with aider-chat 0.86.2 and warns on other versions; installs are pinned to that version. Crashes go to ~/.local/state/oppx/last-crash.txt with a hint to use --classic.

• Claude Code parity:

  • Keys: Ctrl-C clears the line, twice on an empty line exits; Ctrl-D exits; Esc (or Ctrl-C) interrupts the model; Shift+Tab toggles plan mode (/ask, no edits); \+Enter or Esc-Enter adds a new line; ? shows shortcuts.
  • Input prefixes: !cmd becomes /run, @file adds the file, # note appends to OPENPHALANX.md. Shell commands run with PAGER/GIT_PAGER=cat and without the Esc watcher, so !git log doesn’t open less and a command that prompts gets the keys (before, both hung the chat).
  • Commands: /clear /compact /cost /context /status /model /init /memory /review /export /doctor /vim /resume /undo /permissions /exit, (/undo restores the files the last editing turn changed, once; nothing is committed), plus Aider’s /add /drop /run /test /lint /web, our /search, and /mcp (connectors: the connectors-and-browser topic). Other Claude-only commands report “not available”.

• Esc interrupt: EscWatcher holds the terminal in cbreak mode during a turn and turns a lone Esc into SIGINT. It pauses whenever a question needs an answer. Keystrokes typed during a turn are kept: finished lines are queued as the next messages, and a partial line is pre-filled.

• Typing during a turn is shown (term.ahead, announce_queued): the terminal doesn’t echo while a turn runs (cbreak mode), so type-ahead worked and showed nothing, which read as “nothing can be typed”. The spinner’s line now ends with what is being typed and a cursor block (✻ Thinking… (2s · esc to interrupt) > and which one is pub▌), cut from the left to fit the width since the line is redrawn in place; with nothing typed it says “type to queue a message”, and with lines waiting “2 queued”. Enter prints “Queued for when this is done: …” into the transcript (at the spinner’s next tick, never into a streaming answer) and the line is sent as the next message when the turn ends. Backspace, Ctrl-U (the line) and Ctrl-W (the last word) work; a pasted character split across two reads is kept. Not shown while the final answer streams (the spinner is off then): what is typed there is still kept and comes back in the next prompt. Seen in a real chat (tmux): the typed text on the spinner line, the queued note, the queued message run next, and a half-typed line pre-filled at the prompt after it.

• Interrupts: Aider’s keyboard_interrupt (which exits on a double press) is replaced. A mid-stream interrupt is detected from the “I see that you interrupted…” note Aider records; check the coder returned after SwitchCoder too.

• Ask/edit routing: each normal message goes through a one-word ask/edit decision by the server’s model through the proxy (routing.ask_choice). In the agent engine the message is read with the end of the assistant’s last reply and any change still owed (routing.intent_input), and a second one-word check asks whether a reply announced a change (routing.announces_change); see the oppx-agent topic. It’s one regex-constrained token on regular models (about 40 ms, 16/16 on probes, including text-only requests such as essays). On reasoning models (/v1/info reasoning, passed as OPPX_REASONING), SGLang applies the regex only after the reasoning, so it’s a short free answer at low effort (chat_template_kwargs reasoning_effort: low, enable_thinking: false) instead, and the first allowed word counts. That scores 16/16 on gpt-oss-20b in about 260 ms. These calls carry oppx_utility: true, so the gateway never routes or searches them. Questions run as /ask, so they can’t edit; a file named in an answer is added to the chat but doesn’t trigger Aider’s “I added these files” follow-up request (it made the model answer twice). Without it, Qwen-14B in diff mode deleted mul when asked “what does mul return?”. The gateway skips the web-search router for requests that carry regex or response_format.

• Context manager (oppx_chat/context.py):

  • Hooks: it wraps Coder.format_messages, so every request Aider builds is fitted first.

  • Budget: (context − 4096 reserved for the answer) / factor. The factor covers the gap between Aider’s generic token count and the model’s real tokenizer: 1.10 until calibrated (26,065 tokens at 32k), then the measured ratio × 1.03. After each answer, ContextManager.calibrate counts the request it just sent with the server’s tokenizer (POST /v1/tokenize via the proxy, in a background thread) and keeps a running average. On Qwen2.5-Coder the ratio is about 1.01, giving 27,541 tokens.

  • Making room, cheapest loss first:

    1. summarize the conversation (summarizer.summarize_all);
    2. set aside files the model requested that are least recently used;
    3. halve the repo map down to 1,024 tokens;
    4. set aside older files you added or that were edited.

    Files used in the current turn are never set aside. Set-aside files stay visible as signatures in the repo map.

  • Repo map: capped at 8,192 tokens (map_mul_no_files=1). Aider alone lets it grow to about 28k when no files are in the chat, which caused a 46k-token request on this repo.

  • File requests: the model’s requests go through confirm_ask. A single file over 60% of the budget is refused with a reason.

  • Calibrating early: an uncalibrated request above 60% of the budget is measured with /v1/tokenize before it’s sent. The 1.10 guess was 20% short on Qwen3.6, so its first large request failed every time and nothing ever calibrated.

  • Server refusals: SGLang’s “Input length (N tokens) exceeds the maximum allowed length (M tokens)” (and “longer than the model’s context length”) is classified as a context overflow (_get_ex_info), not a connection error, which Aider retried for about a minute with nothing on screen. The numbers set the ratio and the server’s limit, and run_turn refits and resends once. --allow-auto-truncate is deliberately not used: SGLang would cut the prompt’s start (system prompt, repo map) without telling the agent.

  • Final guard: check_tokens is replaced, so a request is never sent over the limit, and Aider’s “proceed anyway / providers won’t charge” text is suppressed.

  • Visibility: /context shows the breakdown, and the status bar shows the percentage used.

• Progress display: one TurnSpinner runs from Enter to the end of the turn. It’s paused by any output (out() and the spinner share OUT_LOCK) and restarted on each wait; Aider’s WaitingSpinner is mapped onto it. Coder.show_pretty is forced to True. Aider disables streaming and its spinner whenever it picks a non-``` fence (any Markdown file with code blocks in the chat), which looked like a freeze. The edit-block hider understands every fence Aider may pick: ```, four backticks, and <source>, <code> and <pre> tags.

• Editing safeguards:

  • Aider’s --no-suggest-shell-commands is always set. Aider otherwise offers to run any bash block in a reply, even quoted file content.
  • A change request that yields no edit gets one model-agnostic retry in whole format (_whole_file_retry; small files only, output hidden, diff shown), then a “No changes were made” notice.
  • The content before an edit comes from OppxIO.write_text.

• Questions on strict models: questions reuse the edit prompts (for the cache), so cache.QUESTION_RULE adds a question exception to the system prompt and reminder. gpt-oss ranks system rules over the note in the message and otherwise answered questions with edit blocks. Aider’s few-shot examples go in the system prompt (examples_as_sys_msg); sent as turns, gpt-oss took them for the real conversation.

• Server errors (oppx_io.server_problem): litellm’s errors and Aider’s provider hints (“Check your API key”) become one plain message per turn: revoked (pair again), starting up or busy, unreachable, or certificate changed. A refused turn skips the whole-file retry, and Aider’s “Retrying in N s” lines are hidden.

• Model info: oppx reads GET /v1/info at start-up and passes OPPX_MODEL_ID (shown in the welcome box, status bar and /status) and a --model-settings-file with the server’s edit_format and use_repo_map: true (Aider otherwise assumes whole and no repo map for our model name). Reasoning output is hidden by hide_reasoning, and the spinner keeps going while the model reasons. That covers <think> spans, a lone </think>, Aider’s “► THINKING / ► ANSWER” markers, and Aider’s internal <thinking-content-…> tag: with SGLang’s separate reasoning_content, Aider 0.86 opens with that tag but closes with the configured one. OppxIO.ai_output strips reasoning from the session file too, so -c/-r don’t resend it.

• Memory: OPENPHALANX.md (and OPPX.md and AGENTS.md if present, matched in any letter case, so OPPX.MD works) in the repo root is loaded read-only every turn. /init asks the model to write it.

• Sessions: a message is written to the session file once: the prompt writes what is typed (as Aider does), and the agent turn writes only what didn’t come from the prompt (-p, lines typed during a turn; OppxIO.input_logged). Both used to write, so every typed message was in the file twice, and -c/-r restored it twice. One file per conversation in ~/.local/state/oppx/history/<repo>-<id>/<YYYYmmdd-HHMMSS>.md, with a shared input.history. The old single per-repo file is migrated as session 00000000-000000. -c and -r pass --restore-chat-history.

• Prefix-cache friendliness (oppx_chat/cache.py; measured with scripts/bench_session.py). SGLang only reuses an unchanged prompt prefix, and two things used to change near the start of every request, giving 3–7% cache hits over 10 turns:

  • Repo map: Aider re-ranked it around the chat files and the names each message mentions. It’s now ranked once for the whole repo and cached per (file list, map size). The context manager’s map size only shrinks within a session (reset by /clear).
  • System prompt: questions ran in Aider’s ask mode, which has its own prompts. AskCoder now gets the editing coder’s prompts, and the question carries ASK_NOTE; ask mode never applies edits. The map heading’s {other} word is fixed too.

  Result: 50% of prompt tokens served from cache over the same 10 turns, and follow-up questions start answering in 0.6 s instead of 4.8 s. The rest is inherent: new or edited files, files moving into the history after the first edit, and compaction. OPPX_DUMP_REQUESTS=<file> writes every request as JSON lines to find where consecutive requests diverge.

• Testing: drive it in a pty (Python pty.fork, 120×40 via TIOCSWINSZ). The driver must answer cursor-position requests (ESC[6n → ESC[30;1R), or prompt_toolkit never draws the status bar. Render the raw bytes with pyte to see the real screen.

oppx client CLI: pairing, config, terminal UI, engine, –update

Part of the developer notes; the index is in CLAUDE.md.

Client (oppx)

cargo install --path client/oppx              # or: cargo build -p oppx (binary in target/debug/oppx)
oppx pair <server> <code> [--fingerprint FP | --yes] [--as NAME] [--device-name NAME] [--force]
oppx pair self                                # the server on this machine: no code or fingerprint to copy
oppx status [NAME]                            # certificate pinned ✓, token valid ✓, model ready + context
oppx [PROMPT] [--no-web] [--classic]          # chat frontend (--classic: Aider's UI); unpaired: banner + how to pair
oppx -c | -r [ID] | -p PROMPT                 # continue latest session · resume (picker or id) · print mode (one answer)
oppx --update                                 # update oppx (release download, or source pull + rebuild) and its engine
oppx aider [--server NAME] [--no-web] [-- aider args]   # local Aider via pinned loopback proxy; never commits
oppx proxy [NAME] [--port N] [--no-web]       # proxy only; prints OPENAI_API_BASE and a per-run local key
oppx search "query" [-n N]                    # web search via the server's SearXNG (Markdown; use /run in Aider)
oppx mcp add NAME [-e KEY=VALUE] -- COMMAND [ARGS]    # add a connector (MCP server) that runs on this machine
oppx mcp add NAME --url URL [-H 'Name: value']        # ... or one reached over HTTP
oppx mcp [list] | oppx mcp remove NAME        # what's added (yours and this repo's) · drop one
oppx browser [on | off | install]             # the chat's headless browser: setting and install (status without an argument)
oppx unpair [NAME] [--local-only]             # self-revoke on the server, then forget locally
oppx servers | oppx use NAME

• oppx pair self (local_pairing): reads the backend container’s admin token with docker inspect, asks the loopback admin API for a pairing code and the certificate fingerprint, then runs the normal pairing against 127.0.0.1 with that fingerprint pinned; saved as local. It needs Docker access, which already lets a user read that token by hand, so it grants nothing new; without it the command says to pair with a code. The code it requests replaces the one the app is showing. Tested against the gateway alone.
• Config: ~/.config/oppx/config.json (0600), or OPPX_CONFIG / --config.
• Agent isolation: the agent sees only the per-run local key; the device token stays in oppx.
• Testing without a GUI: pairing codes can be issued through the admin API: curl -X POST -H "x-admin-token: $ADMIN_TOKEN" http://127.0.0.1:9091/admin/pairing. The token is in the container env: docker inspect openphalanx-backend --format '{{json .Config.Env}}'.
• Terminal UI (client/oppx/src/ui.rs, console + indicatif):
  • There’s an OPENPHALANX banner: large 5-row block letters at 95 columns or more, a compact 2-row version below that, with a green-to-blue gradient. It also has spinners on stderr, rounded panels, and ✓ / ! / ✗ rows.
  • Output is plain when stdout or stderr isn’t a terminal, or when NO_COLOR is set. This matters because /run oppx search output goes into Aider’s chat. Keep it that way.
  • To preview in a pty, use script -qec "stty rows 40 cols 120; oppx status" /dev/null. The rows matter: a pty with 0 rows reports no size, so the compact banner is used.
• Startup checks run together (preflight_inner): health, whoami and /v1/info go out at once, one round trip of waiting instead of three over a WAN. They are then checked in the old order, so the errors read the same. A certificate that doesn’t match the pin fails the TLS handshake before any request is written, so nothing, the token included, is sent to it either way. The connect line ends with “sees pictures” or “text only”.
• Coding engine (client/oppx/src/engine.rs): users never install Aider.
  • oppx keeps a private engine in ~/.local/share/oppx/engine/ (OPPX_ENGINE_DIR overrides): bin/uv, a uv-managed Python 3.12 (UV_PYTHON_PREFERENCE=only-managed, so a system 3.13 can’t break it), and aider-chat==agent::AIDER_VERSION installed with uv tool install into tools/, with engine::EXTRAS (pypdf, fpdf2, resvg-py; see the documents topic) next to it. A private copy missing an extra counts as not current and is installed again. About 740 MB.
  • Lookup order: the private engine at the pinned version, then an aider on PATH at exactly the pinned version (handy on dev machines), otherwise install the private one (engine::ensure). Versions are read from the venv’s aider_chat-X.Y.Z.dist-info folder, not by importing Aider (2 s).
  • uv is reused when present (the private copy, PATH, ~/.local/bin, ~/.cargo/bin); otherwise Astral’s installer runs with UV_UNMANAGED_INSTALL into engine/bin, with no shell-profile changes.
  • The frontend runs on the venv’s python next to the real launcher. uv writes a #!/bin/sh trampoline instead of a Python shebang when the path is long, so the shebang alone isn’t reliable.
  • To test a first install, run with HOME pointing to an empty folder and PATH=/usr/bin:/bin (verified: about 7 s on this network, uv download included).
• --update (client/oppx/src/update.rs) has two modes:
  • Release builds (OPPX_RELEASE_TARGET set at build time by the release workflow): read GitHub’s latest release, download oppx-<target>.tar.gz, check it against SHA256SUMS, unpack next to the current binary and rename over it. A 404 means no release yet.
  • Source builds: work on the checkout oppx was built from (CARGO_MANIFEST_DIR/../..), and refuse if it has local changes. Fetch and fast-forward, then rebuild with cargo install --locked --path client/oppx when anything was pulled, or when the commit embedded at build time (build.rs, shown in oppx --version) differs from HEAD. To test in isolation, use a scratch clone with a local bare remote and CARGO_INSTALL_ROOT set to a scratch folder.
  • The binary that runs is the one updated. cargo install writes to cargo’s bin folder; when the running oppx is a copy elsewhere (e.g. ~/.local/bin, first on PATH), a source update used to say “installed” while oppx --version stayed at the old version. It now copies the new build over the running file (replace_running: written beside it, renamed over it) or fails saying where the new one is, and the last line gives the version read from the installed binary. Run as in the recipe above with the copy in another folder: 2d07f09 → c1d49fc in both places. A release binary updating itself was run too (the real 0.5.1 release → 0.6.1).
  • Both modes then install or refresh the private engine.
• Stand-in agents: a fake aider script on PATH (printing its args and env) is a quick way to test oppx aider without the real agent.

oppx permissions: what runs without asking, saved and session rules, the question

Part of the developer notes; the index is in CLAUDE.md.

• Permissions (oppx_chat/permissions.py, no UI code, modelled on Claude Code; scripts/test_permissions.py, 39 tests):
  • The line is read the way sh reads it (tokenize, segments; CHO-25): quotes decide what is a pattern and what is shell syntax, so grep 'a|b$' f is one command with one pattern and ls x#; rm y is two commands (# starts a comment only at the start of a word). Loop and condition keywords are structure, not commands: for f in a; do rm $f; done is judged as rm $f. $HOME, $PWD and the like are replaced by their value; a ~ is expanded as sh would; any other variable (it may hold a secret, or anything), ${x:-y}, {a,b} and ~user make the word unknown, and a line with an unknown word is asked every time. A loop’s own variable is fine when the loop’s words stay in the project. cd is followed: later names are relative to where it went (a bare cd goes home). Checked against the real sh on 3,000 random quoted lines: the same words every time.
  • Never asked: a command whose every part (split on &&, ||, ;, |) only reads, inside the project:
    • ls, cat, rg, grep, head, wc, find (also with -exec of something that only reads), cd, test/[, ps, pgrep, lsof, md5sum, tac, nproc, nvidia-smi …; sed that only prints or substitutes to its output (sed -n 1,5p, sed 's/a/b/'), awk without system(), pipes or output files; < file from the project.
    • git: status/log/diff/show/blame/ls-files/grep, and the listing forms of branch (--contains, --merged), tag, remote, stash list|show, reflog, worktree list, submodule status, config (--get, --list, or one name); git -C <folder in the project>.
    • A version check of a known tool (node --version, cargo --version, python3 -V), command -v, cargo tree|metadata, npm ls, pip list, docker ps, go env.
    • curl that only fetches from this machine (curl -s localhost:5173, also with -o /dev/null -w …): the dev server just started.
    • Not when an argument points outside the project (below), nor with an argument that makes it write or run something (find -delete, sort -o also written -oFILE, tree -o, rg --pre, git --output, git grep -O<pager>, git -c, uniq in out, jq … env, tail -f, ps e, --files0-from, file -f), nor behind an environment assignment, an earlier export, or a wrapper.
  • One yes is enough for a session (Permissions.answered, answered_tool, session_yes): y allows what was asked about until the chat ends, and saves nothing. For a command that is the rule “always” would have saved (cargo build, the folder outside the project, the > rule); for a connector tool it is that tool, of whatever connector, also one added later. o is “only this once”. Commands that are asked every time (rm, sudo, git push, git checkout -- .) have no such yes. Why: in a real session the same playwright.browser_evaluate call was asked about three times running, and the rule had to hold for any tool, not for that one.
  • Asked once: y (for this session; a typed “ok”, “sure”, “yes please”, “go ahead” counts as yes), o (only this once), N, a (always), or any other text, which declines and is passed to the model as what to do instead. a saves an allow rule for the project in ~/.config/oppx/permissions.json (OPPX_PERMISSIONS). A rule names what is run, and is matched word by word after stripping assignments and wrappers (env -u X, timeout 60, nice):
    • cargo test, npm run, git commit, python -m pytest, python scripts/x.py, bash scripts/test.sh, ./scripts/x.sh with its path; never a bare interpreter or shell, and a tool’s name alone covers only the bare call (git * by hand for all of it).
    • A tool’s own options before the subcommand are left out (GLOBAL_FLAGS): make -j8 test is make test, npm --prefix app run x is npm run, cargo +nightly fmt is cargo fmt, kubectl -n dev get is kubectl get. gh and docker compose keep a third word (gh pr view).
    • A subcommand that runs another program names that program (RUNNERS): uv run pytest, npx vite, go run ./cmd/x; what it runs is judged like any command, so uv run rm … is asked every time. xargs cmd is xargs cmd: what it runs is known, the names it is handed are not, so it never counts as only reading.
    • That exact command only (Key.exact) when it uses a tool’s dangerous side or an option this doesn’t know: find . -name '*.pyc' -delete, sed -i …, awk with system(), curl -d/-o, git -c …/--git-dir, docker exec …, cargo --config x test. A rule find or sed saved by an older version doesn’t cover these either.
  • Outside the project (_where, grant_dir; the dirs list): a rule says what a command does, not where. An argument outside the project is asked about whatever the rules say, also for an allowed tool (cp x ~/.bashrc with cp allowed). a then allows that folder (“files in /srv/logs”, shown in the question and in /permissions; removed with its path), for every command. For a file directly in a folder every program shares (/tmp, /var/tmp, /dev/shm) it allows that one file only, and the folder itself (ls /tmp) is never offered: in the first live run “always” on cat /tmp/x.log had opened all of /tmp. Never offered for a folder that holds credentials or is too wide: /, /home, the home folder itself, /etc, /proc, ~/.ssh, ~/.aws, ~/.gnupg, ~/.config (the rules file is there), ~/.docker, ~/.kube … (BROAD_DIRS, SECRET_HOME). The pattern of a grep/rg/sed/awk/jq, the value of -e, --glob, -name, --format and what echo prints are text, not places; /api/v1 is a pattern because no /api exists.
  • Output redirected into a file of the project (cargo test > out.txt): asked, and a saves the rule >, after which redirects into project files aren’t asked about (the command itself still needs its rule). Outside the project, into .git, or after a cd out of the project: asked every time. 2>/dev/null, >/dev/null and 2>&1 are nothing.
  • Asked every time, never remembered:
    • sudo, su, rm, dd, mkfs*, shutdown, reboot, chmod, chown, truncate, shred, kill, pkill, crontab;
    • git that loses work: push, reset, clean, rebase, rm, checkout -- … / checkout <an existing file> / -f, restore (unless --staged), branch -D, tag -d, stash drop|clear, commit --amend (_destructive; git checkout main, git stash, git branch x stay rememberable);
    • publishing and deleting elsewhere: npm|cargo|uv|poetry publish, docker push/prune, kubectl delete, gh … merge|delete|close, gh release create;
    • whatever runs text as code or hides what runs: bash/sh -c, eval, ssh, python -c, node -e, env -S, and the wrappers setsid, pkexec, busybox, strace, flock, script, runuser, chroot, systemd-run (OPAQUE);
    • anything that can’t be taken apart safely: command substitution, a background job, a here-document, case, an unclosed quote, an unknown variable.
    • A rule for one of these written into the file by hand is ignored.
  • For this session (s; Permissions.offer_session, never written to disk): offered when a line can’t get a saved rule only because a part is an interpreter given its code inline (python3 -c, node -e). It allows that interpreter’s one-liners, and the line’s other parts, until the chat ends. On such a line a saves what a rule can name (curl) and allows the inline code for the session. Not offered for the rest of the never-remembered list (sudo, rm, git push, a shell running text, anything that can’t be taken apart); deny rules and plan mode still refuse. Why: asked to sort an API’s results, Qwen3.6 read the JSON with six curl … | python3 -c '…' variants and the user was asked about every one, with curl already allowed. The run line of the system prompt now also says which commands run without a question and to read JSON with jq or python3 -m json.tool.
  • The question (OppxIO.ask_options; commands, connector tools, the browser): the answers are listed one per line, always in the same order (yes, always, session, no, free text), each key in its own colour (green yes, teal for the lasting ones, red no), under a bold yellow heading. They used to sit in one grey parenthesis after “Allow?” and were easy to miss. Seen in a pty; not yet in a live chat.
  • A command that just failed isn’t run again unchanged (failed_runs): the same line after a non-zero exit, with no edit and no other command that worked in between, is a repeat. In that session the model asked for a failing one-liner twice more.
  • Deny rules win over everything, read-only commands included, and refuse without asking (/permissions deny <rule>); the model is told not to work around it. They are looked for in every command a line may run (commands_in): behind wrappers (setsid make, sudo make, xargs make), in loops and conditions, in the text given to bash -c, eval and env -S, in find -exec, behind runners (uv run make), and in a line that can’t be taken apart (echo `make x`, X=$(date) make x, make x &). deny linear refuses every tool of that connector, like linear.*.
  • Plan mode and /ask keep run, but only for commands that read; anything else is refused without a question (allowed or not).
  • Storage: each change is read-modify-write on the file under a lock (permissions.json.lock), and rules are re-read when the file changes, so two sessions on one project share rules and neither loses the other’s (four threads saving 15 rules each: all 60 kept). The file is written 0600. Lists per project: run, deny, tools, dirs. A "global" section written by hand applies to every project.
  • /permissions lists allow rules, allowed folders and deny rules; allow <rule>, deny <rule>, remove <rule>, clear. allow linear.save_issue (a connector that is connected, or name.*) is saved as a tool rule.
  • Connector tools: y (this tool, until the chat ends), a (this tool, saved), c (every tool of the connector, saved), o (only this once), N, or text. A tool marked asked-every-time (Permissions.every_time: playwright.browser_evaluate, JavaScript the model wrote) is never saved and no rule covers it, written by hand or for its whole connector; a yes covers it for the session like any other tool, so a page that tells the model to run something is then no longer stopped by a question in that session. That is the price of not asking again, taken on purpose. A connector that came with the repository (.mcp.json) isn’t taken at its word about readOnlyHint: its “read-only” tools are asked (judge_tool(…, trusted=False)), and can be remembered.
  • Type-ahead: terminal input still pending when a question appears is discarded (OppxIO._ask_line), so a line typed as the next message can’t answer it.
  • Found in the audit and fixed: a rule saved for ./scripts/x.sh never matched again (and would have matched any x.sh); git -C . push slipped past the never-remember list and was offered a bare git rule; cargo "test x" matched the rule cargo test; > 1 (a file named 1) was treated like >&1; cat ~/.ssh/id_rsa and cat /etc/passwd ran unasked; uniq a b, tree -o, rg --pre, git grep -O, git reflog expire, hostname x, date -s counted as read-only; printenv (it prints secrets) was on the read-only list; two sessions overwrote each other’s rules; typing a where “always” wasn’t on offer declined the command; plan mode couldn’t run even git log. Found by the live run: saying “always” to cargo --version saved the rule cargo, which covered every cargo command.
  • The CHO-25 audit (22 defects, each reproduced first, each now a test named test_dNN_…): the line was split with shlex, which drops everything after a # inside a word (ls src#x; touch P ran unasked, and walked past deny rules); do/then/!/{ were taken for the program, so for …; do rm -rf x; done lost “asked every time” and “always” saved the rules for, do, done; env -u DEBUG cargo test saved the rule DEBUG; setsid, pkexec, busybox … got a blanket rule; a flag was taken for the subcommand (npx -y, kubectl -n, make -j8); uv run, npm exec, docker exec were rules for anything; one harmless find/sed/awk/curl gave a rule for find -delete, sed -i, system(), curl -d @.env; “always” on cat /tmp/x.log saved cat, which then read ~/.ssh/id_rsa; git -c core.sshCommand=… fetch rode on the rule git fetch; git checkout -- ., git branch -D, git stash clear, npm publish, gh pr merge, kill could be remembered; sort -oFILE, git grep -Osh, jq -n env, date -f/etc/x, cat {/etc/passwd,x} counted as read-only; deny rules were walked around by a wrapper, a loop, bash -c, a redirect; quoting was thrown away, so grep '${name}', grep '|' f, rg '()', grep 'foo$', awk '{print $1}' were asked (some every time, some with a rule named after the file); node --version, python -u x.py, bash scripts/test.sh could never be remembered; a typed “ok” declined the command; playwright.browser_evaluate could be allowed forever; the file was 0664 and unlocked. Found on a second pass over the new code and fixed before it shipped: cd && cat .ssh/id_rsa (a bare cd goes home, and later names were taken as the project’s), cd /tmp && echo x > f, cat list.txt | xargs cat (a list the model wrote can name any file), echo $SECRET_TOKEN, ps e, curl file:///etc/passwd, git branch --sort=x newbranch, xxd in out.
  • Differences from Claude Code that remain: file edits are applied without asking (as in its accept-edits mode; /undo reverts the last turn’s), so an allowed test runner can run what the model just wrote; rules are per project path (a worktree or a moved repository asks again) plus a hand-written global section, with no shared repository file; curl to this machine is never asked, though a local service may act on a GET.
  • Tested: the rules and every answer to the question are unit-tested (39 tests). The CHO-25 rewrite in a real chat (tmux, the dev build against the 0.7.1 server, Qwen3.6-27B, a scratch repository and its own rules file): grep -rn 'fn main() {$' src, node --version and for f in src/*.rs; do wc -l $f; done ran unasked; cargo --version > out.txt 2>&1 was asked with “always: > (output into this project’s files)”, and after a git log --oneline > log.txt ran unasked; rm -f out.txt log.txt was asked with no “always” and a typed ok ran it; cat /tmp/x.log offered “always: files in /tmp”, after which head -1 /tmp/x.log ran unasked and ls ~/.ssh was asked with no “always” (declined with a note, which the model followed); after /permissions deny cargo build, for i in 1; do cargo build; done was refused without a question; /permissions listed the > rule, the folder and the deny rule; the rules file was 0600; linear.save_comment was asked with y/a/c/s/n (declined). Not seen live: a repository connector’s read-only tool, the browser’s asked-every-time tool, and type-ahead being discarded. The sessions below were on the old gate; “Do you understand the current changes?” was run earlier on Qwen3.8-27B in -p mode (its git commands ran unasked; a correct summary in 42 s). A real terminal session on Qwen3.8-27B (driven through a pty): a to cargo --version saved a rule and the next cargo --version ran unasked; rm -f … was asked, answered with “do not delete anything; run ls scripts instead”, and the model skipped the rm and ran ls scripts; /permissions listed the rule. A second session covered the rest: after /permissions deny make, make --version was refused without a question and the model said the deny rule forbade it; under /ask, git log -1 --oneline ran unasked and cargo --version was refused without a question; with a test MCP server, notes.add_note was asked once, a saved it, and the next call ran unasked (both notes were written). That session also showed deny make didn’t cover make --version (a tool’s name alone matched only the bare call, which is right for allow rules and wrong for deny rules); fixed, and re-run.

Releasing: CI, the release and publish workflows

Part of the developer notes; the index is in CLAUDE.md.

Releasing

• CI (.github/workflows/ci.yml, every PR and push to dev/main): app build and type check, cargo clippy -D warnings, cargo test for core and oppx, ruff (syntax and undefined names) for the gateway, frontend and scripts, the connector tests (scripts/test_connectors.py), the agent tests with the document packages, the gateway’s input check (scripts/test_inputs.py), the regex check (scripts/check_regexes.py: every pattern compiled and proven free of catastrophic backtracking, with its own tests in scripts/test_check_regexes.py), shell syntax, and the PR title check for PRs into main. The GPU lifecycle test stays manual.
• Release (.github/workflows/release.yml, a merged PR into main with a release prefix, or run by hand with a bump and a changelog line): scripts/bump_version.py bumps every version file and adds the CHANGELOG.md entry. The workflow commits “Release vX.Y.Z” to main, tags it, fast-forwards dev when possible, then starts Publish for the tag.
• Publish (.github/workflows/publish.yml, workflow_dispatch with a tag; redo a release with gh workflow run publish.yml --ref main -f tag=vX.Y.Z). It always runs on main (the github-pages environment only deploys from main) and checks out the tag for everything it builds:
  1. oppx for x86_64/aarch64-unknown-linux-musl (static) and aarch64/x86_64-apple-darwin, with OPPX_RELEASE_TARGET set; oppxs for the two Linux musl targets; the app as .deb (with oppxs and its unit inside: beforeBundleCommand builds it, bundle.linux.deb.files places it) and AppImage (no oppxs: an AppImage is one file).
  2. A GitHub release with the archives, SHA256SUMS, install.sh, install-server.sh, and notes from main’s CHANGELOG.md.
  3. The docs (scripts/gen_docs.py + mdBook) on GitHub Pages.
• No backend image is published. The app carries the build files (docker::BUILD_FILES: the Dockerfile, gateway.py, inputs.py, reqtrace.py, searchguard.py, start-sglang.sh, supervisord.conf, embedded with include_str!) and builds openphalanx-backend:<version>-<hash of those files> on the first start (docker::ensure_image): Docker pulls the official SGLang image, pinned by digest in the Dockerfile, and adds ~45 MB on top. A changed gateway or Dockerfile gets a new tag and is rebuilt, so neither users nor developers run a stale image; older tags of ours are removed after a build. Cluster members build the same way when a split worker starts. A custom image setting is pulled instead.
• Why two workflows: a tag pushed with GITHUB_TOKEN starts no workflow, but a dispatch does. A pull_request run can’t deploy Pages, because its ref is the PR’s merge ref, not main.
• One-time repository settings: Actions → workflow permissions “Read and write”; Pages → source “GitHub Actions”. main is protected by a ruleset (PR and status checks required), and GITHUB_TOKEN can’t be put on a ruleset’s bypass list, so the release commit is pushed with a write deploy key: its private half is the RELEASE_DEPLOY_KEY secret (ssh-key on the checkout in release.yml), and “Deploy keys” is a bypass actor on the ruleset. Without it the push is refused with GH013 (v0.6.0’s first run: the admin’s bypass covers the merge, not the workflow’s push). The commit and tag are pushed with --atomic, so a refusal leaves no tag.
• Dependency updates (Renovate, renovate.json): the Mend Renovate app opens update PRs into dev (never main, where each PR would need a release prefix and cut a release), titled [chore] …, once a release has been public for 7 days (minimumReleaseAge, with internalChecksFilter: strict so no PR appears before that). Minor and patch updates come grouped in one PR, majors one each, and the lock files are refreshed on Monday mornings; nothing auto-merges. The SGLang base image digest gets its own PR (labelled needs-gpu-test: it changes every backend tag, so run the lifecycle example before merging), as do the pinned recheck/typescript of the regex check. The CI runtimes are deliberately at the bottom of the supported range and Renovate leaves them alone: Python 3.12 (the private Aider engine), Node.js 20, and ubuntu-22.04 for the .deb/AppImage builds. The Dependency Dashboard issue lists what is pending or held back. Check a change to the file with npx --package renovate renovate-config-validator --strict renovate.json.
• Lint the workflows locally with docker run --rm -v "$PWD:/repo" -w /repo rhysd/actionlint:latest.

Ports, files and the security model

Part of the developer notes; the index is in CLAUDE.md.

Ports and files

Open 9090/tcp in your firewall for the clients’ network, and 9092/tcp plus 9093/udp between cluster servers. Never expose 9091 or 9094.

Security model

• TLS: the public API uses a self-signed certificate created on first start. Clients pin its SHA-256 fingerprint when they pair (shown in the GUI), so later connections can’t be intercepted.

• Pairing codes: 8 characters, about 39 bits. Each is single-use, valid for 10 minutes, and burned after 5 wrong attempts, with a 1 s delay after each miss.

• Device tokens: each client trades a pairing code for its own random 256-bit device token. The server stores only the SHA-256 hash, and you can revoke any device from the GUI.

• Pairing expiry: pairings last 1 day, 1 week (default), 1 month, 1 year or never (Client tab; settings.pairing_ttl_days). The app passes it as DEVICE_TTL_DAYS at start and changes it live with PUT /admin/pairing-policy; it applies to every device, counted from when it paired. Devices paired before expiry existed count from the upgrade (since in backend-state/pairing_policy.json). An expired token gets a 401 “pairing expired”; oppx status, the chat and oppx search say to pair again, and oppx status shows how long the pairing has left.

• Admin API: the admin API is reachable from the server machine only, and requires a random token generated at each launch.

• Connectors (MCP): run on the client with the user’s own credentials; the server never sees their definitions. A repository’s .mcp.json starts nothing until the user agreed to that exact definition, every tool call is confirmed (or allowed for the session), and what a connector returns goes to the model as untrusted reference text. Like any tool output, it becomes part of the prompts sent to the server.

• Web search: SearXNG is reachable only from the gateway (private network, no ports) and keeps no logs. Automatic search is on by default (the user’s choice; --no-web opts out). Router-written queries leave the server for public search engines, so they can carry fragments of a request. Results are injected as untrusted reference text. The gateway keeps the latest queries and their results in memory for 15 minutes (the search guard’s cache), never on disk, and paces and pauses searches so the engines don’t block the server.

• Safety bars (gateway, any client):

  • forces n=1 and drops best_of, and clamps max_tokens to half the context;
  • answers 413 for bodies over 8 characters per context token, before SGLang sees them;
  • allows at most 4 requests in flight per device and 8 server-wide, queueing for up to 120 s before a 503 with retry-after;
  • limits each device to 120 requests a minute (429).

  The settings live in gateway.py, and the MAX_ACTIVE_REQUESTS, MAX_DEVICE_REQUESTS, QUEUE_WAIT_S and RATE_PER_MINUTE variables override them. Slots are released through an idempotent cleanup that also runs as the response’s background task, so a client that disconnects before streaming starts can’t leak one. SGLang itself rejects over-length input with a clean 400 and queues overload: 12 concurrent 20k-token requests all completed.

• Split model: needs a trusted LAN; the traffic between ranks is unauthenticated and pickle-based (see the cluster topic, Split serving).

• No code stored or executed: the server keeps no code and runs no commands for clients. Request bodies (prompts) are never logged; only per-device request and token counts are kept, plus the in-memory request trace (timings, token counts, hashes). The trace holds prompt text only while the owner has switched Capture content on, never on disk (see the trace topic). Model weights are mounted read-only, and the backend never downloads weights on its own.

Server GUI: models, start, VRAM fit, notifications, updates

Part of the developer notes; the index is in CLAUDE.md.

Using the server GUI

1. Get the app

From source, in development mode with hot reload:

cd app
npm install
npx tauri dev # start whole gui server

# you can also start in headless way, especially for none-host nodes.
cargo run -p openphalanx-server -- run --name dev

Or build an installable package and install it:

cd app && npm install && npx tauri build --bundles deb    # or: --bundles appimage
sudo apt install ../target/release/bundle/deb/Openphalanx_0.3.0_amd64.deb

2. Choose a model

Open Models:

• Context window (slider at the top) is a server setting: the most tokens one request can use, passed to SGLang when the server starts. It doesn’t filter models; it changes each model’s KV-cache need, so the VRAM column and badges follow it. The panel shows how far the selected model can go on this GPU. The same slider (ContextSlider.svelte) sits in the Server page’s model card; both write settings.context_len. It saves when the thumb is let go (pointer, key or blur, not only change: WebKit doesn’t fire change after a drag whose value the code moved, so the slider showed 64k, nothing was saved and Start used 32k). It’s locked while a backend is starting, running or stopping (and set_context_len refuses then), since the running model keeps the context it started with. It also can’t go past the selected model’s maximum for the VRAM free now (get_context_limit: the same vram_budget as the Fits badges, this GPU or the split pool, re-read every 5 s): dragging stops at that exact value, the track beyond it is hatched red, and set_context_len refuses anything longer. A saved context above the limit (after switching models, or when another program took VRAM) shows a warning and Use .
• The table lists Qwen, Gemma 4, gpt-oss, Devstral and Mistral, Granite, DeepSeek, GLM, Llama, Phi, Kimi and MiniMax, each from its smallest to its largest size, plus code-specialized models (StarCoder2, Code Llama, Codestral, Seed-Coder, Yi-Coder, OpenCoder, Mellum, IQuest-Coder, KAT-Coder, OpenHands LM), filterable by family, by “only models that fit”, by “only downloaded models”, and by a search box (lib/search.ts: every word must match the name, family, repo, size, quantization, quantizer or license, ignoring case and punctuation; typos are forgiven only when nothing matches as typed), sorted by release date or download size (either direction; newest first by default), 15 per page. Official repos, plus widely used 4-bit Community quantizations (labelled with who made them) where the publisher ships none.
• The table shows each model’s VRAM need at the chosen context window, with a Fits / Tight / Won’t fit badge measured against your GPU’s free memory, and the longest context that would fit. The need is deliberately conservative and much larger than the download size. It covers weights as loaded, KV cache for the full context plus 25%, and runtime memory. Best fit for this GPU marks the largest model that fits with headroom.
• Every downloaded model has a delete button (except the one the server is running). A copy the app downloaded is removed from its models folder; one found in the Hugging Face cache is removed from that cache (its whole models--org--name folder, every revision), after a warning that other tools may share it. Anything outside those two places is refused.
• A downloaded model whose files are damaged shows Broken with the reason (model::broken): a file the download recorded is missing or has another size, config.json or the shard index doesn’t parse, a shard is missing, or a .safetensors file doesn’t end where its header says the last tensor does (cut short or overwritten). Only sizes and headers are read (5–16 ms for the 14B and 27B models here), cached until a file’s size or date changes. The pre-flight Model check then fails with “…files are broken (…). Delete it on the Models page and download it again”, which blocks Start. A cluster member counts a damaged copy as missing, and a split worker refuses to start with one. Content damage that keeps the size isn’t caught (that would need re-hashing ~10 GB on every check).
• Click Download on a model that fits. Progress shows at once (a progress bar; it pulses until the size is known), then each phase: listing files, checking files already on disk (a resumed .part file is re-hashed first, which takes a while for big shards), and the transfer with its speed (up to 4 files at once, download::PARALLEL_FILES; the line names every file in flight). Measured on the 3090 node, 2026-10-16: Qwen2.5-Coder-14B-AWQ (10 GB, 3 shards) in 111 s against 215 s one file at a time, every shard SHA-256-checked. A download cancelled at 3 GB resumed all three .part files and finished verified; the size is reported as soon as the file list is in, and until then the empty bar pulses (no moving fill, which read as progress resetting). A second click while it runs is harmless. Every message banner in the app has an (X) to close it (Notice.svelte). The default, Qwen2.5-Coder-14B-Instruct-AWQ, is already selected, and it is reused if it is in ~/.cache/huggingface.
• To use your own model, enter a local folder (with config.json and .safetensors) or a Hugging Face URL under Add your own model. Its VRAM need is checked before anything downloads.

3. Start

On Server, check that the pre-flight list is green, then press the power button:

• The first start builds the backend image: Docker downloads the official SGLang image (about 16 GB, once) and the app adds its gateway on top (about a minute). Loading the model then takes about 3–4 minutes.
• The running model (name, quantization, context) shows under the headline with a lock, and as Running on Models; it’s fixed until the server stops (the tooltip says so). The context comes from the container’s CONTEXT_LENGTH.
• The button turns green when SGLang is ready, and a pairing code appears. Use it to pair a client. Codes are single-use and expire after 10 minutes, but paired devices stay paired.
• Logs shows live SGLang output (before the first start it says so, instead of Docker’s “No such container”). Devices lists paired clients and lets you revoke them.

Start is disabled whenever free VRAM is below the least the selected model starts with. Close other GPU programs, or choose a smaller model or context length.

Fits, tight, won’t fit (vram::check): the comfortable need (total_bytes: weights, KV cache for the context plus 25%, runtime at 2 GiB + 15% of the weights) plus 1.5 GiB is Fits. Down to the least the model starts with (Requirement::min_bytes: weights, KV for exactly one window, runtime at 2 GiB + 8% of the weights) is Tight: it starts, with a warning. Below that is Won’t fit. The context slider’s maximum uses the same floor. In the tight range mem_fraction_static shrinks the runtime reserve first, down to the lean one.

• The fraction is taken of free memory, not of the card. SGLang keeps free at start × (1 − fraction) as its runtime reserve. We used to divide by the card’s total, so memory other programs held was subtracted twice: with 3 GiB taken by another process, a 2.8 GiB reserve became 6.2 GiB and the 14B model got a 14.9k-token KV cache for a 32k context, with 4 GB idle (the same effect left the split 27B run short). Fixed: 34.2k tokens in the same test.
• Verified at the bottom of the tight range (3090, a filler process holding 3 GiB, Qwen2.5-Coder-14B-AWQ at 32k: 18.7 GiB free against 20.7 comfortable and 18.5 least): it loaded, served a 31.4k-token prompt, a 700-token generation and two 23k-token prompts at once, with no out-of-memory error; about 0.5 GB stayed free.
• Notifications (lib/notify.svelte.ts, Notifications.svelte, mounted in App.svelte): what the Server page has to say sits in a stack at the top right of the window, over the page instead of inside it. Before, up to seven banners sat around the power button; each moved the page as it came and went, and a closed one came back when its text changed (free VRAM, a download’s percentage) or the page was opened again. A condition (a member lacks the model, with its Download button; a member is downloading; GPU memory almost full) has one id, updates in place, has no close button and leaves when the state does. A hint or an error (model selected, “follow it in Logs”, a failed action, the context note) can be closed. Leaving the page takes its notes along. Why a start is blocked stays where it was, under the headline. The context slider’s “would run out of VRAM, so the slider stops there” is plain text now: its close button did nothing while the slider sat at the limit. Rendered in a harness page in both themes (a condition, one with an action, an error, a hint, text changing every 300 ms without the note moving); not yet seen in the running app, and the other pages’ own error banners (Notice.svelte) are unchanged.
• Checking at launch (probe in nav.svelte.ts, Home.svelte): the first pre-flight comes before cluster members have reported (they report every 5 s and count as offline until the first one), so it saw this GPU alone and said “not enough VRAM” or offered Force load for a model the cluster could hold. Now the Server page first says what it is checking: the headline reads “Checking this machine”, a ring sweeps out from the power button, and the line under it and the pre-flight VRAM row (a spinner) say “Checking free GPU memory…”, “Looking for the cluster’s servers: 1 of 2 answered…” (as host), “Connecting to …” (as member) or “Working out whether fits…”. Start, Force load and the red VRAM verdict wait until the GPU and the checks are in, every known member is online with a report (or the host link is up), and a pre-flight asked for after that has come back; at most 20 s for the cluster (OFFLINE_AFTER_SECS), at least 1.5 s so it doesn’t flash. Once only per run of the app; skipped when a backend is already running. Type-checked, not yet seen in the running app. The Models page’s Fits badges don’t wait.
• Force load: when not enough VRAM is the only failing check, the Server page shows a checkbox (“Force load. I understand it may fail to load, or crash when it runs out of GPU memory…”); ticking it enables the power button (start_server with force, server::start_with; oppxs start --force). It starts with the lean reserve and gives SGLang the rest (vram::mem_fraction_forced); the gateway then reports whatever context the KV cache holds. Every other check still blocks. The tick lives in nav.svelte.ts (forceLoad), so it survives a visit to another page; it is dropped once the checks no longer ask for it. While ticked, the VRAM messages (the line under the headline, the pre-flight row, the bar and the slider’s “doesn’t fit” note) are amber instead of red. Ticking it also sets the context window to the longest the model fits right now (get_context_limit), or the 2k minimum when none does, with a notice; if the model then fits, the checkbox goes away and Start is simply enabled. Unticking puts the earlier window back.

Closing the app stops the server (shutdown in the app, on RunEvent::Exit): the backend and SearXNG it manages are stopped and the split orders withdrawn, so nothing keeps the GPU’s memory without a window to stop it from. Ctrl-C, kill and logging out (SIGINT, SIGTERM, SIGHUP) are turned into a normal exit first. A member’s app also stops its split worker (Cluster::goodbye, which oppxs runs on Ctrl-C and SIGTERM too). Two exceptions: a backend started by hand (“running outside the app”) is left alone, and an update’s relaunch keeps the backend for the new copy to adopt. An app that crashes or gets kill -9 can’t run any of this, so the gateway has a watchdog: the app sets APP_WATCHDOG_S=60 on every backend it starts, the gateway notes each authenticated admin call (the app polls every 2 s), and after 60 s without one it stops the container (“No OpenPhalanx app has checked in for 61 s…” in the log). Verified with the gateway alone: stopped 16 s after start with nobody polling at a 15 s limit; stayed up through 40 s of polling, then stopped 17 s after the last call.

4. Updates

At launch the app asks GitHub for the latest release (app_update.rs in core; quiet when offline). If it’s newer, a dialog offers to download it; Check for updates under the version in the sidebar asks again.

• How it installs depends on how the app was installed (Install::detect):
  • AppImage ($APPIMAGE): the new AppImage is downloaded next to it and renamed over it.
  • .deb (the binary belongs to a package, dpkg-query -S): pkexec apt-get install on the downloaded file, so the system asks for the password. The file is downloaded as /tmp/openphalanx-update/<asset name> (download_path): apt accepts a file on its command line only when the name ends in .deb, and the first version passed .Openphalanx_0.6.1_amd64.deb.download, so every package update failed with “E: Unsupported file … given on commandline”. Reproduced and checked with apt-get -s install on the real 0.6.1 package (old name refused; new name “Inst openphalanx [0.6.0] (0.6.1 local-deb)”); the password prompt and relaunch after it still haven’t been run.
  • Built from source or tauri dev: the dialog says to git pull and rebuild, with no install button.
• Links (GitHub in the sidebar, What’s new, View on Hugging Face) go through openLink (lib/api.ts). The opener permission needs a URL scope: capabilities/default.json allows https://github.com/* and https://huggingface.co/*. With opener:allow-open-url alone every link was refused without a message, so none of them ever worked; a refusal now shows as an error. Not yet clicked in a running app.
• Checked before anything changes: the download must match its line in the release’s SHA256SUMS (the same trust model as oppx --update: GitHub over TLS). Tauri’s updater plugin would add signatures, but needs a signing key in CI and a latest.json per release.
• Relaunch: a detached sh -c 'sleep 2; exec <app>' starts the new copy after this one exits (Tauri’s restart() starts it first, and both would bind the cluster port). The backend container keeps running and the new app adopts it, a split run included (workers.json).
• To test: build an AppImage from a copy with the version lowered (Cargo.toml, tauri.conf.json, package.json), run it, and update to the latest release.

5. Problem reports, and what happened before a crash

Found the hard way: a split start ended with the host frozen and force-rebooted, and afterwards nothing said what the app had been doing. The user’s account was “the client lost the server, the host said the member dropped, the member was fine, after a restart the member wasn’t detected and a port was still taken, I freed it, started again, and the host died”.

• The journal (core::report, ~/.local/share/openphalanx/events.jsonl): the app writes a line when the server’s state changes (with the message, so a pause names its reason), when Start or Stop is pressed, when a start fails, when the cluster service can’t start or comes back, when members come and go (by number, not by name) and when the ports are freed. Written as it happens, so it survives a machine that freezes. Cut back to its latest 400 lines when it passes 256 KB.
• Did the last run end cleanly? (running.json, report::begin_run / heartbeat / end_run): written at launch, refreshed every 10 s with the server’s state, removed on a normal exit. One found at the next launch whose process is gone means the app was killed, it crashed, or the machine froze or lost power. The app then shows an alert on every page (“Openphalanx didn’t shut down cleanly last time: it was last seen at 14:10 with the server starting…”) with Report it…; closing it, or the report dialog, dismisses it (dismiss_unclean). A second copy that is still running is not counted as a crash.
• Report a problem (sidebar, under GitHub; ReportDialog.svelte, make_report, report::gather): the user says what happened, the app gathers the rest and shows the whole report before anything leaves the machine: version and how it was installed, the server’s state and message, the settings that matter, the cluster (role, strategy, members by number and whether they are online), the machine (OS, kernel, CPU, memory and swap, GPUs, driver, Docker), the backend container (image, status, exit code, killed for memory or not), who holds the server’s ports, the kernel log’s signs of trouble from this boot and the one before (an Oops, the out-of-memory killer, an NVIDIA Xid, a stuck CPU; kernel_faults), the journal’s last 60 lines, and the last 80 lines of the backend log and 40 of the split worker’s. Open GitHub issue copies the report and opens github.com/chophilip21/openPhalanx/issues/new with the title, the bug label and the first 5,000 characters filled in (a link can’t hold more); the rest is pasted. It is also saved under ~/.local/share/openphalanx/reports/ (the latest 20 kept). Nothing is sent by the app.
• What a report never holds (report::redact): the home folder’s name (it becomes ~), the host name, IPv4 addresses (<ip>), and long hexadecimal strings (tokens, fingerprints: the first 8 characters stay). Members appear by number. Prompts are never in it: the backend doesn’t log them.
• Every probe has a time limit (6 s): a report is most wanted when something on the machine has stopped answering.
• Headless: oppxs report [what happened] prints the same report and the link.
• Each 2 s snapshot asks the GPU and Docker together (build_snapshot: gpu::query and docker::inspect under tokio::join!; the admin call needs the container’s answer, so it follows). The gateway’s /admin/status likewise gathers SGLang’s readiness, metrics and inputs.
• nvidia-smi has a time limit too (gpu::query, 8 s, the process killed): the app asks every 2 s from its monitor and from every page that shows VRAM. Without a limit a driver that stops answering blocks the monitor (nothing feeds the backend’s watchdog; a member stops reporting and the host pauses) and leaves a stuck process behind with every question from a page. Whether that happened in the incident above is not known: the machine’s own recording showed kernel CPU time climbing from 0.2 to 2.8 cores over ten minutes with memory free, after a kernel Oops earlier in that boot, and had no per-process data.
• The port conflict is an alert (App.svelte, the cluster-service condition): when the cluster service can’t start because another server process holds its ports, the top-right stack shows the reason on every page with a Free the ports button. It was a small underlined “Free the ports…” at the bottom of the cluster panel, easy to miss and not recognisable as a button.
• Tested: core unit tests (the journal and its trimming, the run marker through a kill and a clean exit, redaction, the kernel-log search, the report’s layout and the link’s length). oppxs report on the 3090: a real report, which found the previous boot’s kernel Oops and the exited backend (exit code 255). The dialog, the two alerts and the sidebar entry are type-checked, not yet seen in the running app.

Settings page: GPUs, ports, privacy, storage

Part of the developer notes; the index is in CLAUDE.md.

The Settings page (last in the sidebar; pages/Settings.svelte, CHO-5) groups what isn’t about starting a model, so the Server page only has the model, its context window and the pre-flight list. Every change is saved to settings.json at once.

• Server: the GPUs (below; locked while a backend runs) and Web search for clients, moved here from the Server page.
• Several GPUs in one machine (CHO-24; experimental: written and unit-tested on a machine with one card, never run on one with several):
  • Setting: settings.gpus is the list of GPU indices the server may use, null for every GPU in the machine (the default on a new install). A settings.json from before this, which names one card in gpu_index, keeps that one card (Settings::from_json), so an update doesn’t move a working setup. gpu_index is still written (the first selected card) for older versions.
  • Choosing: Settings shows one checkbox per card when there are several (index, name, total and free memory; at least one stays ticked; ticking them all saves null, so a card added later is used too). set_gpus; headless, oppxs gpus lists them and oppxs gpus all / oppxs gpus 0,2 chooses. Refused while a backend runs.
  • A model that fits one card runs on one card: of the selected ones, the one with the most free memory (server::best_gpu). That is the single-GPU path everything else here was tested on, so a machine with several cards changes nothing for a model that fits one, except that the fullest card is no longer picked by index.
  • A model that doesn’t fit one card is spread over the selected ones (server::local_split): until now that was “Won’t fit”. The layers are shared out by the cluster split’s planner (split::plan_gpus, one stage per card in index order, by free memory), and the container gets --gpus "device=0,2", SGLANG_PP_LAYER_PARTITION and --pp-size N with the two settings every rank must share (--chunked-prefill-size, --max-mamba-cache-size). No --nnodes: read in SGLang 0.5.21 (engine._calculate_rank_ranges), with one node every pipeline rank runs on this machine, rank r on the r-th visible card; its parser accepts these flags (prepare_server_args in the image: pp_size 2, nnodes 1). --mem-fraction-static is one value for all ranks, the smallest any stage allows.
  • What shows it: the pre-flight VRAM row turns amber with “Doesn’t fit one GPU, so it is spread over GPU 0, GPU 1 (experimental, not yet tried on real hardware): GPU 0: 40 layers (…) · GPU 1: 24 layers (…)”, or says why it can’t be. The Models page counts the selected cards together (the pool donut, titled “This machine · N GPUs (experimental)”), each card paying its own runtime memory (vram::split_across), and the context slider’s maximum follows (server::max_context_over in oppxs).
  • With a cluster: a member offers and uses only its selected cards (its report lists those; its split worker takes the one with the most free memory). A host that splits across the cluster gives one card, as before; spreading one cluster stage over a machine’s cards is not done.
  • Not done: tensor parallel (--tp-size, faster on identical cards); Force load over several cards (it forces the best single card); per-card load and temperature in the charts for the cards in use; and any run at all on real hardware. A first run should check that both ranks come up, the KV cache size SGLang reports, and a long prompt.
  • The single-card start still works on this code: the lifecycle example on the 3090 (Qwen2.5-Coder-14B at 32k: image rebuilt, ready in 226 s, pair, token checks, chat, revoke). A cluster split start, where the member now picks its card through the selection, hasn’t been run.
  • Tested: core unit tests (settings: all by default, a chosen list, the old file’s one card, an empty list refused; server: the card with the most free memory, nothing spread when the model fits one, the plan over two cards and its fraction, the reason when it doesn’t fit, the longest context over several; docker: the container arguments), and oppxs gpus on the 3090 with scratch settings (the list, and the refusal while the backend runs).
• Privacy: Hide IP addresses (settings.hide_ips): addresses show with their first five digits hidden (192.168.1.74 → xxx.xx8.1.74; the dashed form 100-125-91-43 too; IPv6 is left alone, since its hex looks like the fingerprints beside it). It works on the screen, not on the data (lib/privacy.ts): a MutationObserver masks text and title tooltips as they appear or change, so server labels, Rust-built messages, log lines, chart labels and the pairing command are all covered without each component knowing. Copy buttons read the data and copy the real address; anything inside data-real is left as is. Turning it off puts the real text back. Checked in a harness page: on, off, and text Svelte updates after masking; versions (0.6.1), dates, model names and fingerprints are untouched.
• Ports: each port the server uses, what it’s for, who reaches it (laptops, this machine only, other servers), when it’s open, and who listens on it now (ports::overview, from ss -H -ltnup: “this app”, “Docker (a container’s published port)”, another process by name and pid, or “not visible” for another user’s, which is the backend’s own process on a split model’s host network), refreshed every 10 s. Every row has a Change button (set_port, edited in place), greyed out with the reason as its tooltip when the port can’t move:
  • Movable (ports::Movable, all used on this machine only or by clients): the client API (settings.agent_port, default 9090: published by Docker and passed as AGENT_PORT, shown in the endpoint and pairing command, read by oppx pair self from the container; laptops pair again with the new address), the admin API (settings.admin_port, 9091: published on loopback and passed as ADMIN_PORT; the app’s AdminClient takes the port from the running container’s environment, ContainerInfo.admin_port, so a backend adopted after a restart or started by hand is reached where it really listens), and the oppxs control API (settings.control_port, 9094: read by oppxs run and its commands at start). Client and admin are locked while a backend is starting or running; control is refused while something holds it (a running oppxs).
  • Checked (ports::port_problem): from 1024 up, not a fixed port (9092, 9093, 9096, 9098, 9100–9116), and not where another movable port already is.
  • Fixed: the cluster API (9092) and discovery (9093/udp): every server broadcasts to 9093 and announces its cluster address with 9092, so moving either on one machine would hide it from the others; it would need a cluster-wide setting. The split model’s ports (9096, 9098, 9100–9116) stay fixed for now; the host sends the rendezvous address in its orders, so they could become host-movable later.
• Storage: where settings, downloaded models, the Hugging Face cache, backend state and cluster state live, with a copy button.
• Appearance: theme System / Light / Dark (settings.theme; the sidebar’s switch saves there too).
• Settings stick: settings.json (~/.config/openphalanx/) is never touched by an update or by closing the app, and only a change on this page (or an oppxs command) writes it. Loading is per field (Settings::from_json): a value that doesn’t read (a wrong type, written by hand or by another version) falls back alone instead of resetting every setting, which the old all-or-nothing parse did on the next save; a file that isn’t JSON is copied to settings.json.unreadable before anything overwrites it; keys from a newer version are kept and written back (extra). The theme moved from the webview’s localStorage into settings.json; localStorage only paints the first frame before the backend answers.
• Headless: oppxs port (the three movable ports), oppxs port <n> / port admin <n> / port control <n>, oppxs ports, oppxs web-search on|off, with the same rules (tried on scratch settings).
• Tested: core unit tests for the port overview, the port checks (fixed ports, one movable port onto another, a moved control port being the one watched) and the per-field loading; the page in a Vite harness with stubbed Tauri calls (light, dark, 460 px, server running: client and admin greyed, a clash refused, a move saved, masking on and off, the theme saved). Not yet seen in the running app, and a backend hasn’t been started on moved ports yet.

Trace page and the gateway’s request trace

Part of the developer notes; the index is in CLAUDE.md.

The Trace page (under Logs) shows what each request cost and how requests relate, live while the server runs. It reads the gateway’s in-memory trace (docker/server/reqtrace.py, the latest 2,000 requests; nothing is written to disk).

• What a record holds: device, timings (wait for a slot, first token when streamed, total), prompt, cached and written tokens, status, and the labels the client attached. No text: the conversation’s shape is kept as one hash per message prefix. oppx labels each request in the body (oppx_trace: session, turn, step, kind of call, the tool the previous action chose and how long the client ran it, context estimate, events such as tool output cut or a repeat); the gateway strips them, like oppx_utility. /v1/info says "trace": true and oppx sends labels only then (OPPX_TRACE), since an older gateway would pass them on to SGLang. Other clients show up as single requests without turns.
• Capture content (a checkbox, off by default; PUT /admin/trace/content): also keeps the end of the latest message, the reply and tool titles (they name files and patterns), 6,000 characters each, in memory. Switching it off drops them. This is the one place the server holds prompt text, and only while its owner asks for it.
• Views: turns (newest first, with stuck / failed / context-cut badges); where the time went for a turn, as a flame-style chart (a time axis, the turn’s own span, then one span per request placed on the turn’s clock so they step down and to the right, each named beside it and coloured by phase: the model reading the prompt, the model writing, the client running the tool, waiting; under it the phases as a table with their share of the turn, including the time between requests; for calls that aren’t streamed the read/write split is estimated from the tokens written at the streamed replies’ speed, and says so); context per request (prompt tokens stacked as from-cache and computed against the window, ▾ where the client cut context or the cache held under half of what had been sent before); the requests as a table (the same numbers without hovering); prefix cache.
• One colour per meaning across the page: strong blue is the prompt the model had to read (time in the first chart, computed tokens in the second), pale blue the part that came from the cache, orange is writing, green the client. The first version used blue and orange for different things in the two charts. The palette was checked with the dataviz validator on both themes’ surfaces. The page lays itself out by its own width (container queries): under 860 px the turn list moves on top and the tiles go two by two; it was rendered at 1280, 760 and 460 px.
• Prefix cache (RadixAttention): SGLang 0.5.21 has no endpoint for its tree (only pretty_print to its own log), so the page shows two things. SGLang’s own counters from /metrics: tokens cached for reuse (kv_evictable_tokens), in use by running requests, evicted so far, and the recurrent-state pool. And the tree of shared prefixes rebuilt from the requests (Trace.tree: a node is a run of messages every request through it shares; it ends where requests part or one ends), sized in tokens with each request’s own characters-per-token ratio and coloured by how recently it was used. It shows what could be reused; expected_cached (the part of a prompt an earlier request already sent) against SGLang’s reported cached_tokens shows what was. On a real turn they agreed within a few tokens once warm (2,370 expected, 2,368 cached).
• Admin API: GET /admin/trace?after=<seq> (new records and unfinished older ones, plus the pool), GET /admin/trace/tree, PUT /admin/trace/content, DELETE /admin/trace. App commands get_trace, get_trace_tree, set_trace_content, clear_trace.
• Tested: scripts/test_reqtrace.py (7 tests, no dependencies, in CI); the gateway in the backend image against a fake model (streamed, plain, a refusal, content on and off, no admin token: 401); and a second gateway with the new code in front of the running split Qwen3.8-27B with a real oppx -p question (route, action, answer, reply: all labelled, the tool attributed, no text kept). The page itself was checked with made-up data in both themes; not yet seen against a live backend in the app (that needs the server restarted on the new image).
• A new build file: reqtrace.py is in docker::BUILD_FILES and the Dockerfile, so the image tag changes: every cluster member must have the same code, or its split worker can’t build the image the host asks for.

Web search: SearXNG, the router, the search guard

Part of the developer notes; the index is in CLAUDE.md.

Web search

• SearXNG: the openphalanx-searxng container is pinned by digest and runs on the private openphalanx Docker network with no published ports. Its settings live in crates/openphalanx-core/searxng/settings.yml and are written to ~/.local/share/openphalanx/searxng/settings.yml (0644, read-only mount, FORCE_OWNERSHIP=false). The secret is passed as SEARXNG_SECRET, and --log-driver none keeps queries out of logs. The GUI starts it before the backend when settings.web_search is on (the default; toggle on the Server page). The backend gets SEARXNG_URL=http://openphalanx-searxng:8080.

• Gateway:

  • POST /v1/search {query, max_results} returns {results: [{title, url, snippet}]}.
  • A chat request with X-Oppx-Web-Search: auto (sent by oppx unless --no-web) goes through the router, in two steps:
    1. Decide: a yes/no call to the same model (gateway.classify: one regex-constrained token, or a short free answer on reasoning models; see ask/edit routing in the oppx-chat-frontend topic). It reads the user’s whole latest turn, every user message since the last assistant message, minus any paragraph that repeats the system prompt: Aider appends its editing rules to each message, and the router read those instead of the question. The few-shot examples sit in a fixed system prompt, so they’re cached. Median about 50 ms (about 260 ms on gpt-oss); 16/16 correct on mixed phrasings for both.
    2. Query: only on yes, a call writes the search query (constrained JSON, or free text on reasoning models). It’s told today’s date, and drop_invented_dates removes dates and years the model added that the user didn’t write. gpt-oss stamped queries with “2024-10-04” or today’s date and found stale pages.
    3. Results say when they were retrieved, that they beat training data for things that change (versions, releases), and that links can’t be opened. gpt-oss otherwise tried to browse and returned nothing.
  • Speculative routing: the answer starts at the same time as the router, and its response is held unread until the router decides. With no search, the router costs about 27 ms of time-to-first-token (68 ms against 39 ms measured). With a search, the speculative answer is closed (SGLang aborts it) and the request is resent with <web_search_results> appended to the end of the latest message, with an “untrusted” note.
  • Native tool calling was tried and doesn’t work with Qwen2.5-Coder-14B.

• Search guard (docker/server/searchguard.py, used by gateway.web_search, so by /v1/search and the router alike): public engines suspend an address that asks too often, and SearXNG then returns an empty list, which a client can’t tell from “nothing found”. A day of agent evals did exactly that (Brave and Google CSE “Suspended: too many requests”, DuckDuckGo timing out), and one agent turn then reworded an empty search 41 times.

  • Cache: the same query within 15 minutes is answered from memory, 256 queries at most; nothing on disk. “The same” ignores case, punctuation, word order and filler words, and from four words up also covers nearly the same words (NEAR, 0.8 of all the words either query has): agents reword a search far more often than they ask something new. Shorter queries must match word for word (“vite latest release” is not “react latest release”).
  • Pace: a burst of 6 searches, then one every 20 s, for the whole server. Over that the search is refused with 503 and retry-after, not queued.
  • Pause: engines refusing (no results and every engine in SearXNG’s unresponsive_engines) twice in a row pauses search for 5 minutes, doubling up to an hour while the probe after each pause is refused too. During a pause nothing is sent, the 503 says why (“web search is paused for 4 min: the search engines are refusing requests (brave: Suspended: too many requests; …)”), and the router isn’t asked either. An empty answer from engines that did respond is a result, and is cached.
  • Settings: SEARCH_CACHE_S, SEARCH_BURST, SEARCH_REFILL_S, SEARCH_TRIP_AFTER, SEARCH_PAUSE_S, SEARCH_PAUSE_MAX_S. GET /admin/status has a search block (paused, for how long, the reason, sent, from cache, paced); the app doesn’t show it yet.
  • Client: a search with nearly the words of an earlier one in the session (0.6 of them shared; Agent._searched, the latest 30) gets the earlier results back with a note, without a request. The agent stops offering web_search until the server’s retry-after has passed, in this turn and the next ones (Agent._search_off_until); three searches in a row with no results, or 8 in one turn (MAX_SEARCHES), take the tool away for the turn. oppx search prints the server’s reason.
  • Tested: scripts/test_searchguard.py (7 tests, no dependencies, in CI), and a second gateway from the working tree on spare ports in front of a fake SearXNG: a repeat from the cache (1 query reached the engine for 2 requests), the 7th search in a burst refused with retry-after: 5, two refusals then a pause (2 queries reached the engine for 4 requests), a cached query still answered while paused, and search back after the pause. Then on the real 3090 in the rebuilt image (the lifecycle example, Qwen2.5-Coder-14B at 32k): after two searches GET /admin/status showed paused: true with “brave: Suspended: too many requests; duckduckgo: CAPTCHA; google cse: Suspended: too many requests”. searchguard.py is a new build file, so the image tag changes and every cluster member needs the same code; a split start on the new image hasn’t been run yet.
  • When testing, don’t search in a loop. scripts/eval_app.py runs without web search unless --web is given.

• Client: oppx, oppx aider and oppx proxy send the header by default; --no-web opts out (--web is accepted as a hidden no-op). oppx search calls /v1/search.

• Manual run (without the GUI):

  docker network create openphalanx
  docker run -d --name openphalanx-searxng --network openphalanx --log-driver none \
    -e FORCE_OWNERSHIP=false -e SEARXNG_SECRET=$(openssl rand -hex 32) \
    -v $PWD/crates/openphalanx-core/searxng/settings.yml:/etc/searxng/settings.yml:ro \
    searxng/searxng@sha256:c642712fcedcdaa78fac44f71eada86aff510745826ba1bd1a368211fea2ce7f
  # then add to the backend's docker run: --network openphalanx -e SEARXNG_URL=http://openphalanx-searxng:8080
  

Changelog

Generated from merged pull request titles by scripts/bump_version.py.

v0.8.1 (2026-10-17)

Fixes

• Configure Renovate (#19)

v0.8.0 (2026-10-17)

Features

• Documents and pictures, an input guard, a regex guard, and faster downloads (#18)

v0.7.5 (2026-10-15)

Fixes

• Benchmarks on a 3090 and a 4090 Laptop, and a check at launch before the VRAM verdict (#16)

v0.7.4 (2026-10-12)

Fixes

• Problem reports after a crash, the port conflict as an alert, and a time limit on nvidia-smi (#15)

v0.7.3 (2026-10-12)

Fixes

• Follow-ups that made no edit, typing during a turn, and agent gaps (#14)

v0.7.2 (2026-10-12)

Fixes

• Permission gate audit: commands that ran unasked, questions that repeated, and connector recovery (#13)

v0.7.1 (2026-10-12)

Fixes

• Agent fixes from the end-to-end app audit, a search guard, and server notifications (#12)

v0.7.0 (2026-10-11)

Features

• Headless browser for the chat, and a Settings page for the server (#11)

v0.6.2 (2026-10-11)

Fixes

• Working links, oppx –update, and a permissions audit (#10)

v0.6.1 (2026-10-10)

Fixes

• Fix/setup docs (#9)

v0.6.0 (2026-10-10)

Features

• Trace feature added to the server (#8)

v0.5.1 (2026-10-07)

Fixes

• Fix split serving, agent context and client defects found in end-to-end testing (#6)

v0.5.0 (2026-10-07)

Features

• Agent engine that reads files on demand; backend image built locally (#5)

v0.4.1 (2026-10-05)

Fixes

• Release build: version every workspace crate in Cargo.lock (#4)

v0.4.0 (2026-10-05)

Features

• Multi-GPU cluster: discovery, host control and one model split across servers (#3)

v0.3.0 (2026-10-04)

Features

• OpenPhalanx proof of concept (#1)