Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

oppx chat frontend: modules, Aider hooks, context manager, sessions, prefix cache

Part of the developer notes; the index is in CLAUDE.md.

  • Chat frontend (client/oppx/frontend/: the oppx_chat package plus run.py, embedded with include_str! via agent::FRONTEND):

  • Modules, lowest layer first; each imports only from the ones above it (the map is in oppx_chat/__init__.py): config (environment from oppx), term (output, spinner, Esc), render (streamed answers, diffs), context (ContextManager), mcp (connectors: the MCP client and its tools; no UI imports), agent (the agent engine; no UI imports), oppx_io (Aider’s IO, prompt, keys), routing (ask/edit check), cache (prefix-cache patches), commands (slash commands, memory), app (Aider patches, turn loop, main). A new module must also be listed in agent::FRONTEND; a unit test checks this.

  • Engines (oppx --engine agent|aider, OPPX_ENGINE; default agent):

    • The agent engine is in the oppx-agent skill, its permission rules in oppx-permissions, and the app-building eval in agent-app-eval.
    • aider: Aider’s own loop, with everything below (context manager, cache patches, ask/edit routing). Slash commands, /undo, sessions, the status bar and Esc work in both.
    • Measured (RTX 3090, this repo at v0.4.1): a typical Aider request was ~25k tokens, ~60% whole files the model had mentioned and ~30% repo map, with 8–19 s to the first answer and a 42% cache hit rate. The agent’s questions use 2–14k-token requests with 76–93% served from cache, answering in 2–6 s. Edits (scripts/eval_edits.py, 6 tasks checked by compiling and testing, 3 runs): Qwen2.5-Coder-14B-AWQ agent 18/18, Aider 15/18 (Aider can’t do the multi-file rename); gpt-oss-20b agent 5/6 three times, Aider 4/6. Split Qwen3.6-27B-AWQ (3090 + 4090 laptop, v0.5.0): eval_edits.py agent 6/6 (24–52 s a task); a 10-turn bench_agent.py session that fills the window went from 3 failed turns and a 61% cache hit rate to none and 89% with the fixes above.

  • How it runs: oppx writes the package to ~/.cache/oppx/frontend-<version>-<hash>/ and runs run.py with the Python of the Aider venv (the python next to the real launcher, or its shebang), so aider, prompt_toolkit and rich are importable.

  • How it hooks into Aider: it sets aider.main.InputOutput to its own OppxIO, and base_coder.WaitingSpinner to its Thinking spinner. It calls aider.main.main(argv, return_coder=True), then loops over coder.run_one() itself, handling SwitchCoder the way Aider’s main loop does.

  • What it changes on screen:

    • It hides Aider’s announcements, URLs and Tokens: lines.
    • Edit blocks are hidden while they stream, and each turn ends with a Claude-style diff (⏺ Update(file) plus colored lines).
    • Edit-retry noise collapses to one line.
    • Routine confirmations are auto-accepted. Commands the model proposes, and going over the context window, are asked.

  • Aider version: it relies on Aider internals, so it’s tested with aider-chat 0.86.2 and warns on other versions; installs are pinned to that version. Crashes go to ~/.local/state/oppx/last-crash.txt with a hint to use --classic.

  • Claude Code parity:

    • Keys: Ctrl-C clears the line, twice on an empty line exits; Ctrl-D exits; Esc (or Ctrl-C) interrupts the model; Shift+Tab toggles plan mode (/ask, no edits); \+Enter or Esc-Enter adds a new line; ? shows shortcuts.
    • Input prefixes: !cmd becomes /run, @file adds the file, # note appends to OPENPHALANX.md. Shell commands run with PAGER/GIT_PAGER=cat and without the Esc watcher, so !git log doesn’t open less and a command that prompts gets the keys (before, both hung the chat).
    • Commands: /clear /compact /cost /context /status /model /init /memory /review /export /doctor /vim /resume /undo /permissions /exit, (/undo restores the files the last editing turn changed, once; nothing is committed), plus Aider’s /add /drop /run /test /lint /web, our /search, and /mcp (connectors: the connectors-and-browser topic). Other Claude-only commands report “not available”.

  • Esc interrupt: EscWatcher holds the terminal in cbreak mode during a turn and turns a lone Esc into SIGINT. It pauses whenever a question needs an answer. Keystrokes typed during a turn are kept: finished lines are queued as the next messages, and a partial line is pre-filled.

  • Typing during a turn is shown (term.ahead, announce_queued): the terminal doesn’t echo while a turn runs (cbreak mode), so type-ahead worked and showed nothing, which read as “nothing can be typed”. The spinner’s line now ends with what is being typed and a cursor block (✻ Thinking… (2s · esc to interrupt) > and which one is pub▌), cut from the left to fit the width since the line is redrawn in place; with nothing typed it says “type to queue a message”, and with lines waiting “2 queued”. Enter prints “Queued for when this is done: …” into the transcript (at the spinner’s next tick, never into a streaming answer) and the line is sent as the next message when the turn ends. Backspace, Ctrl-U (the line) and Ctrl-W (the last word) work; a pasted character split across two reads is kept. Not shown while the final answer streams (the spinner is off then): what is typed there is still kept and comes back in the next prompt. Seen in a real chat (tmux): the typed text on the spinner line, the queued note, the queued message run next, and a half-typed line pre-filled at the prompt after it.

  • Interrupts: Aider’s keyboard_interrupt (which exits on a double press) is replaced. A mid-stream interrupt is detected from the “I see that you interrupted…” note Aider records; check the coder returned after SwitchCoder too.

  • Ask/edit routing: each normal message goes through a one-word ask/edit decision by the server’s model through the proxy (routing.ask_choice). In the agent engine the message is read with the end of the assistant’s last reply and any change still owed (routing.intent_input), and a second one-word check asks whether a reply announced a change (routing.announces_change); see the oppx-agent topic. It’s one regex-constrained token on regular models (about 40 ms, 16/16 on probes, including text-only requests such as essays). On reasoning models (/v1/info reasoning, passed as OPPX_REASONING), SGLang applies the regex only after the reasoning, so it’s a short free answer at low effort (chat_template_kwargs reasoning_effort: low, enable_thinking: false) instead, and the first allowed word counts. That scores 16/16 on gpt-oss-20b in about 260 ms. These calls carry oppx_utility: true, so the gateway never routes or searches them. Questions run as /ask, so they can’t edit; a file named in an answer is added to the chat but doesn’t trigger Aider’s “I added these files” follow-up request (it made the model answer twice). Without it, Qwen-14B in diff mode deleted mul when asked “what does mul return?”. The gateway skips the web-search router for requests that carry regex or response_format.

  • Context manager (oppx_chat/context.py):

    • Hooks: it wraps Coder.format_messages, so every request Aider builds is fitted first.

    • Budget: (context − 4096 reserved for the answer) / factor. The factor covers the gap between Aider’s generic token count and the model’s real tokenizer: 1.10 until calibrated (26,065 tokens at 32k), then the measured ratio × 1.03. After each answer, ContextManager.calibrate counts the request it just sent with the server’s tokenizer (POST /v1/tokenize via the proxy, in a background thread) and keeps a running average. On Qwen2.5-Coder the ratio is about 1.01, giving 27,541 tokens.

    • Making room, cheapest loss first:

      1. summarize the conversation (summarizer.summarize_all);
      2. set aside files the model requested that are least recently used;
      3. halve the repo map down to 1,024 tokens;
      4. set aside older files you added or that were edited.

      Files used in the current turn are never set aside. Set-aside files stay visible as signatures in the repo map.

    • Repo map: capped at 8,192 tokens (map_mul_no_files=1). Aider alone lets it grow to about 28k when no files are in the chat, which caused a 46k-token request on this repo.

    • File requests: the model’s requests go through confirm_ask. A single file over 60% of the budget is refused with a reason.

    • Calibrating early: an uncalibrated request above 60% of the budget is measured with /v1/tokenize before it’s sent. The 1.10 guess was 20% short on Qwen3.6, so its first large request failed every time and nothing ever calibrated.

    • Server refusals: SGLang’s “Input length (N tokens) exceeds the maximum allowed length (M tokens)” (and “longer than the model’s context length”) is classified as a context overflow (_get_ex_info), not a connection error, which Aider retried for about a minute with nothing on screen. The numbers set the ratio and the server’s limit, and run_turn refits and resends once. --allow-auto-truncate is deliberately not used: SGLang would cut the prompt’s start (system prompt, repo map) without telling the agent.

    • Final guard: check_tokens is replaced, so a request is never sent over the limit, and Aider’s “proceed anyway / providers won’t charge” text is suppressed.

    • Visibility: /context shows the breakdown, and the status bar shows the percentage used.

  • Progress display: one TurnSpinner runs from Enter to the end of the turn. It’s paused by any output (out() and the spinner share OUT_LOCK) and restarted on each wait; Aider’s WaitingSpinner is mapped onto it. Coder.show_pretty is forced to True. Aider disables streaming and its spinner whenever it picks a non-``` fence (any Markdown file with code blocks in the chat), which looked like a freeze. The edit-block hider understands every fence Aider may pick: ```, four backticks, and <source>, <code> and <pre> tags.

  • Editing safeguards:

    • Aider’s --no-suggest-shell-commands is always set. Aider otherwise offers to run any bash block in a reply, even quoted file content.
    • A change request that yields no edit gets one model-agnostic retry in whole format (_whole_file_retry; small files only, output hidden, diff shown), then a “No changes were made” notice.
    • The content before an edit comes from OppxIO.write_text.

  • Questions on strict models: questions reuse the edit prompts (for the cache), so cache.QUESTION_RULE adds a question exception to the system prompt and reminder. gpt-oss ranks system rules over the note in the message and otherwise answered questions with edit blocks. Aider’s few-shot examples go in the system prompt (examples_as_sys_msg); sent as turns, gpt-oss took them for the real conversation.

  • Server errors (oppx_io.server_problem): litellm’s errors and Aider’s provider hints (“Check your API key”) become one plain message per turn: revoked (pair again), starting up or busy, unreachable, or certificate changed. A refused turn skips the whole-file retry, and Aider’s “Retrying in N s” lines are hidden.

  • Model info: oppx reads GET /v1/info at start-up and passes OPPX_MODEL_ID (shown in the welcome box, status bar and /status) and a --model-settings-file with the server’s edit_format and use_repo_map: true (Aider otherwise assumes whole and no repo map for our model name). Reasoning output is hidden by hide_reasoning, and the spinner keeps going while the model reasons. That covers <think> spans, a lone </think>, Aider’s “► THINKING / ► ANSWER” markers, and Aider’s internal <thinking-content-…> tag: with SGLang’s separate reasoning_content, Aider 0.86 opens with that tag but closes with the configured one. OppxIO.ai_output strips reasoning from the session file too, so -c/-r don’t resend it.

  • Memory: OPENPHALANX.md (and OPPX.md and AGENTS.md if present, matched in any letter case, so OPPX.MD works) in the repo root is loaded read-only every turn. /init asks the model to write it.

  • Sessions: a message is written to the session file once: the prompt writes what is typed (as Aider does), and the agent turn writes only what didn’t come from the prompt (-p, lines typed during a turn; OppxIO.input_logged). Both used to write, so every typed message was in the file twice, and -c/-r restored it twice. One file per conversation in ~/.local/state/oppx/history/<repo>-<id>/<YYYYmmdd-HHMMSS>.md, with a shared input.history. The old single per-repo file is migrated as session 00000000-000000. -c and -r pass --restore-chat-history.

  • Prefix-cache friendliness (oppx_chat/cache.py; measured with scripts/bench_session.py). SGLang only reuses an unchanged prompt prefix, and two things used to change near the start of every request, giving 3–7% cache hits over 10 turns:

    • Repo map: Aider re-ranked it around the chat files and the names each message mentions. It’s now ranked once for the whole repo and cached per (file list, map size). The context manager’s map size only shrinks within a session (reset by /clear).
    • System prompt: questions ran in Aider’s ask mode, which has its own prompts. AskCoder now gets the editing coder’s prompts, and the question carries ASK_NOTE; ask mode never applies edits. The map heading’s {other} word is fixed too.

    Result: 50% of prompt tokens served from cache over the same 10 turns, and follow-up questions start answering in 0.6 s instead of 4.8 s. The rest is inherent: new or edited files, files moving into the history after the first edit, and compaction. OPPX_DUMP_REQUESTS=<file> writes every request as JSON lines to find where consecutive requests diverge.

  • Testing: drive it in a pty (Python pty.fork, 120×40 via TIOCSWINSZ). The driver must answer cursor-position requests (ESC[6n → ESC[30;1R), or prompt_toolkit never draws the status bar. Render the raw bytes with pyte to see the real screen.