Skip to content

feat: add display-only Markdown transformers - #4952

Open
rumpl wants to merge 1 commit into
rumpl/markdown-trusted-sgrfrom
rumpl/markdown-transform-hooks
Open

rumpl wants to merge 1 commit into
rumpl/markdown-trusted-sgrfrom
rumpl/markdown-transform-hooks

Conversation

@rumpl

@rumpl rumpl commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

Adds settings.markdown_transformers: display-only rewriting of assistant Markdown in the TUI, modeled on Pi's registerMarkdownTransformer. Stored messages, model context, exports, and the copy-message action keep the original Markdown; mouse selection copies what is displayed.

Each transformer is a long-lived process speaking newline-delimited JSON-RPC 2.0 on stdio. The TUI sends markdown/transform with the complete Markdown snapshot, plus message_type, is_streaming, available_width, and message identity; the reply's result.markdown is rendered. Transformers run in order, and a failing one leaves the preceding Markdown in place.

Third of a stack, split to keep reviews focused. Replaces #4943, which has the earlier review history.

  1. fix(markdown): strip terminal controls from rendered text #4948: strip terminal controls from rendered text
  2. feat(markdown): render trusted SGR styling and replay it across wraps #4949: render trusted SGR styling and replay it across wraps
  3. This PR: display-only Markdown transformers, full TUI
  4. feat(leantui): apply display-only Markdown transformers #4953: Markdown transformers in the lean TUI

Design

pkg/markdowntransform has independent layers:

  • command.go: starts the process and exposes stdin plus stdout lines. It has no protocol knowledge.
  • rpc.go: JSON-RPC 2.0 over any writer and line stream. It has no process knowledge.
  • transformer.go: the markdown/transform method, the per-request timeout, and the restart policy. Error replies keep the process running. Malformed output, timeouts, and exits restart it, at most once per second. Transformers must exit when stdin closes.
  • display.go: one background worker for the program. Each rendered message owns a Surface.
    • A render returns immediately with the latest result, or with the source before the first result.
    • Only the latest pending snapshot per surface is transformed.
    • The most recently rendered surfaces go first, so live replies are not queued behind restored history.
    • A streaming reply is transformed at most once every 50 ms. Its first and completed snapshots are not delayed.

Sending the full Markdown keeps transformers stateless, as in Pi, where the transformer runs synchronously once per frame. The rate limit bounds the cost of resending long replies over a pipe.

TUI integration:

  • One display per program, owned by the TUI and stopped during cleanup. Views are refreshed through the existing all-tabs broadcast.
  • Replies report is_streaming until their saved message arrives or the last stream of their session stops, including segments split at tool or reasoning boundaries. A nested stream completes its own replies but not the parent's, even when it shares the parent's session.
  • Assistant replies lose every escape sequence before they are sent to a transformer. Transformer output is rendered with AllowSGR (feat(markdown): render trusted SGR styling and replay it across wraps #4949).
  • Transformed replies keep the incremental streaming renderer. Per-code-block copy is hidden while transformers are configured, because rewritten fences cannot be mapped back to the source code.
  • Results are versioned and batched per RefreshInterval. The live reply is spliced in, and other changes share one rebuild.

Known limitation: replies of a remote nested stream that shares its parent's session complete when that session's last stream stops, because runtime events carry no stream identity. Tracked in #4951.

A Go implementation of Pi Emphasis using this protocol is at https://github.com/rumpl/markdown-emphasis.

Validation

  • task build passes.
  • Race tests pass for pkg/markdowntransform, pkg/tui/..., and pkg/userconfig. They include real subprocess tests for process reuse, error replies, timeout and exit restarts, and ordered chains.
  • golangci-lint run --new-from-rev and the project cops report no issues. Windows go vet passes for the affected packages.
  • task test fails only on the existing TestACPRuntimeConfig_ModelSwitchingIsSessionOwned.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Status markers inside multiline Markdown code spans can be incorrectly replaced, and images added or rewritten by transformers fail to render inline.

2 open issues
What changed in this PR

Adds configurable, display-only Markdown transformers to the full TUI. Assistant replies can be rewritten for presentation without changing stored messages, model context, exports, or whole-message copying.

Changes:

  • Introduces persistent JSON-RPC transformer processes with timeouts, restart handling, ordered execution, and background scheduling.
  • Integrates transformed rendering, batched refreshes, and reply-completion tracking across TUI tabs.
  • Adds user settings, protocol documentation, a Python example, and lifecycle tests.
File Description
pkg/​userconfig/​userconfig.go Adds transformer settings.
pkg/​userconfig/​userconfig_test.go Tests settings serialization.
pkg/​tui/​tui.go Owns the shared display and its lifecycle.
pkg/​tui/​streamcontent/​streams.go Tracks active streams per session.
pkg/​tui/​streamcontent/​streams_test.go Tests nested-stream completion tracking.
pkg/​tui/​page/​chat/​runtime_events.go Completes replies when session streams stop.
pkg/​tui/​page/​chat/​chat.go Wires display configuration and stream resets.
pkg/​tui/​messages/​messages.go Adds transform-completion notifications.
pkg/​tui/​markdown_display.go Batches transform refreshes across tabs.
pkg/​tui/​image_support.go Reuses the shared chat broadcast helper.
pkg/​tui/​dispatch.go Extracts all-tab message dispatch.
pkg/​tui/​components/​messages/​transform_test.go Tests completion and transcript refresh behavior.
pkg/​tui/​components/​messages/​messages.go Refreshes transformed items and creates streaming replies.
pkg/​tui/​components/​messages/​assistant_content.go Completes logical replies and session segments.
pkg/​tui/​components/​message/​transform_test.go Tests display isolation, styling, and copy behavior.
pkg/​tui/​components/​message/​message.go Renders transformed snapshots with trusted styling.
pkg/​markdowntransform/​transformer.go Implements request timeouts and restart policy.
pkg/​markdowntransform/​transformer_test.go Tests process reuse, failures, and chains.
pkg/​markdowntransform/​rpc.go Implements newline-delimited JSON-RPC calls.
pkg/​markdowntransform/​rpc_test.go Tests responses, errors, and timeouts.
pkg/​markdowntransform/​markdowntransform.go Defines configuration, inputs, and ordered execution.
pkg/​markdowntransform/​main_test.go Provides subprocess test helpers.
pkg/​markdowntransform/​display.go Schedules snapshots and versions completed results.
pkg/​markdowntransform/​display_test.go Tests scheduling, throttling, and notifications.
pkg/​markdowntransform/​command.go Manages process pipes and diagnostic logging.
examples/​markdown-transform/​settings.yaml.example Shows transformer configuration.
examples/​markdown-transform/​emphasis.py Demonstrates status-marker styling.
docs/​configuration/​user-settings/​index.md Documents configuration, protocol, and display behavior.

Review effort: Balanced

Comment on lines +11 to +31
TOKEN = re.compile(r"(`+)(.*?)\1|\[\[(Done|Passed|Failed|Blocked|Needs you)\]\]")
FENCE = re.compile(r"^ {0,3}(`{3,}|~{3,})")


def decorate(markdown):
lines = []
fence = None
for line in markdown.splitlines(keepends=True):
match = FENCE.match(line)
if fence:
if match and match[1][0] == fence[0] and len(match[1]) >= len(fence):
fence = None
elif match:
fence = match[1]
else:
line = TOKEN.sub(
lambda m: m[0] if m[1] else f"\033[{COLORS[m[3]]};7m {m[3]} \033[27;39m",
line,
)
lines.append(line)
return "".join(lines)

innerRenderWidth := width - messageStyle.GetHorizontalFrameSize()
content, imagePlaceholders := mv.markdownImagePlaceholders(msg.Content, innerRenderWidth)
content, imagePlaceholders := mv.markdownImagePlaceholders(mv.displayMarkdown(innerRenderWidth), innerRenderWidth)
Add settings.markdown_transformers: external commands that rewrite
assistant Markdown before the TUI renders it, like Pi's
registerMarkdownTransformer. Stored messages, model context, copies and
exports keep the original Markdown.

Each command runs as one long-lived process speaking newline-delimited
JSON-RPC 2.0 (markdown/transform). Requests carry the full Markdown and
are served by one background worker, so rendering never waits on a
process: a message shows its source until its first result, then its
latest result. Newest renders are transformed first, streaming replies
at most once per 50 ms, and failing processes restart at most once per
second.

Replies report is_streaming until their saved message arrives or the
last stream of their session stops.

Signed-off-by: Djordje Lukic <[email protected]>
@rumpl
rumpl force-pushed the rumpl/markdown-transform-hooks branch from 334b07c to 210ef18 Compare November 17, 2026 22:58
Copilot AI balanced review requested due to automatic review settings November 17, 2026 22:58
@aheritier aheritier added area/docs Documentation changes area/tui For features/issues/fixes related to the TUI kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Nov 17, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Transformer-added or rewritten images are never loaded, and the example decorator incorrectly rewrites status tokens inside multiline inline-code spans.

3 open issues

Review effort: Balanced

Comment on lines +1004 to +1006
mv.markdownVersion = version
mv.renderCache.valid = false
return true

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Documentation changes area/tui For features/issues/fixes related to the TUI kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants