ex tool should return stderr/errors to MCP client

prittjam commented

2026-03-22 09:26:35 -07:00

(Migrated from github.com)

Problem

When using the ex tool via MCP (e.g. from Claude Code), errors from include() or other expressions go to stderr but are never returned to the MCP client. The tool returns (completed with no output) even when the REPL shows a full stacktrace.

This makes it impossible for AI agents to diagnose errors without the user manually copying and pasting the stacktrace from the REPL buffer.

Example

# Agent runs:
ex(e="include(\"/path/to/script.jl\")", q=false)
# Returns: (completed with no output)
# Meanwhile the REPL shows a full MethodError stacktrace

Suggestion

The ex tool should capture stderr and include it in the MCP response, at least for errors/exceptions. This is critical for autonomous agent workflows where the agent needs to see errors to fix them.

Environment

Kaimon v1.0.1 (main branch)
Julia 1.12.5
Linux 6.18.7-76061807-generic
Claude Code + Emacs (claude-code-ide.el MCP integration)

Note: I see this is likely addressed in the 1.1 branch (commits a160f985, 32b68fe8, 89d3acf5), but 1.1 currently fails to precompile due to a BoxTabs dependency issue with Tachikoma.

## Problem When using the `ex` tool via MCP (e.g. from Claude Code), errors from `include()` or other expressions go to stderr but are never returned to the MCP client. The tool returns `(completed with no output)` even when the REPL shows a full stacktrace. This makes it impossible for AI agents to diagnose errors without the user manually copying and pasting the stacktrace from the REPL buffer. ## Example ```julia # Agent runs: ex(e="include(\"/path/to/script.jl\")", q=false) # Returns: (completed with no output) # Meanwhile the REPL shows a full MethodError stacktrace ``` ## Suggestion The `ex` tool should capture stderr and include it in the MCP response, at least for errors/exceptions. This is critical for autonomous agent workflows where the agent needs to see errors to fix them. ## Environment - Kaimon v1.0.1 (main branch) - Julia 1.12.5 - Linux 6.18.7-76061807-generic - Claude Code + Emacs (claude-code-ide.el MCP integration) Note: I see this is likely addressed in the 1.1 branch (commits `a160f985`, `32b68fe8`, `89d3acf5`), but 1.1 currently fails to precompile due to a `BoxTabs` dependency issue with Tachikoma.

prittjam commented

2026-03-22 12:06:00 -07:00

(Migrated from github.com)

Update: Kaimon 1.1 + Claude Code SSE incompatibility

Installed Kaimon 1.1 successfully (needed Tachikoma from dev/v2.0-preview branch for BoxTabs). The MCP server starts and ping works fine. However, ex tool calls hang from Claude Code's perspective:

The eval runs correctly (user sees result in REPL)
But Claude Code never receives the response
Claude Code spins waiting, user has to interrupt
This happens with all ex calls, regardless of q, s, or ses parameters
ping works fine — only ex is affected

This looks like an SSE streaming change in 1.1 that Claude Code's MCP client can't handle. The eval result is never delivered back to the MCP client even though it completes on the REPL side.

Reverted to 1.0.1 for now.

## Update: Kaimon 1.1 + Claude Code SSE incompatibility Installed Kaimon 1.1 successfully (needed Tachikoma from `dev/v2.0-preview` branch for `BoxTabs`). The MCP server starts and `ping` works fine. However, `ex` tool calls hang from Claude Code's perspective: - The eval runs correctly (user sees result in REPL) - But Claude Code never receives the response - Claude Code spins waiting, user has to interrupt - This happens with all `ex` calls, regardless of `q`, `s`, or `ses` parameters - `ping` works fine — only `ex` is affected This looks like an SSE streaming change in 1.1 that Claude Code's MCP client can't handle. The eval result is never delivered back to the MCP client even though it completes on the REPL side. Reverted to 1.0.1 for now.

prittjam commented

2026-03-22 13:00:33 -07:00

(Migrated from github.com)

Also when I try to intially setup 1.1. there is a bunch of fancy text and animation and it causes problems with the julia REPL/terminal.

kahliburke commented

2026-03-22 13:23:23 -07:00

(Migrated from github.com)

@prittjam I am using it successfully with the latest Claude all the time, so perhaps something else is going on here. What version of Claude code are you using?

kahliburke commented

2026-03-22 13:28:36 -07:00

(Migrated from github.com)

Status: Fixed in 1.1 branch

The error capture and SSE streaming issues are both resolved in the current 1.1 branch. Here's verification of errors flowing back correctly to the MCP client (Claude Code):

Error examples

Explicit error:

ex(e="error(\"test error from agent\")", q=false)
→ ERROR: test error from agent
  Stacktrace:
   [1] error(s::String) @ Base ./error.jl:44
   [2] top-level scope @ none:1
   ...

UndefVarError:

ex(e="undefined_variable + 1", q=false)
→ ERROR: UndefVarError: `undefined_variable` not defined in `Main`
  Suggestion: check for spelling errors or missing imports.
  Stacktrace: ...

include() errors (original issue):

ex(e="include(\"/nonexistent/path.jl\")", q=false)
→ ERROR: SystemError: opening file "/nonexistent/path.jl": No such file or directory
  Stacktrace:
   [1] include(mapexpr::Function, mod::Module, _path::String) @ Base ./Base.jl:307
   ...

ArgumentError:

ex(e="parse(Int, \"not_a_number\")", q=false)
→ ERROR: ArgumentError: invalid base 10 digit 'n' in "not_a_number"
  Stacktrace: ...

All error types return full stacktraces to the MCP client.

SSE hang fix

The ex hang issue (eval runs but result never returns to Claude Code) was caused by REPL.call_on_backend deadlocking when the gate eval runs on a background thread while the REPL backend is occupied by the interactive session. This is now fixed — async evals bypass call_on_backend and eval directly.

Setup

Requires Tachikoma ≥ 1.1.6 (for BoxTabs export). Make sure your Tachikoma dependency resolves to 1.1.6+.
The setup wizard TUI issue in non-standard terminals (Emacs) is a known limitation — you can skip it by creating ~/.config/kaimon/config.json manually with {"mode": "lax"} before first run.

## Status: Fixed in 1.1 branch The error capture and SSE streaming issues are both resolved in the current `1.1` branch. Here's verification of errors flowing back correctly to the MCP client (Claude Code): ### Error examples **Explicit error:** ``` ex(e="error(\"test error from agent\")", q=false) → ERROR: test error from agent Stacktrace: [1] error(s::String) @ Base ./error.jl:44 [2] top-level scope @ none:1 ... ``` **UndefVarError:** ``` ex(e="undefined_variable + 1", q=false) → ERROR: UndefVarError: `undefined_variable` not defined in `Main` Suggestion: check for spelling errors or missing imports. Stacktrace: ... ``` **include() errors (original issue):** ``` ex(e="include(\"/nonexistent/path.jl\")", q=false) → ERROR: SystemError: opening file "/nonexistent/path.jl": No such file or directory Stacktrace: [1] include(mapexpr::Function, mod::Module, _path::String) @ Base ./Base.jl:307 ... ``` **ArgumentError:** ``` ex(e="parse(Int, \"not_a_number\")", q=false) → ERROR: ArgumentError: invalid base 10 digit 'n' in "not_a_number" Stacktrace: ... ``` All error types return full stacktraces to the MCP client. ### SSE hang fix The `ex` hang issue (eval runs but result never returns to Claude Code) was caused by `REPL.call_on_backend` deadlocking when the gate eval runs on a background thread while the REPL backend is occupied by the interactive session. This is now fixed — async evals bypass `call_on_backend` and eval directly. ### Setup - Requires **Tachikoma ≥ 1.1.6** (for `BoxTabs` export). Make sure your Tachikoma dependency resolves to 1.1.6+. - The setup wizard TUI issue in non-standard terminals (Emacs) is a known limitation — you can skip it by creating `~/.config/kaimon/config.json` manually with `{"mode": "lax"}` before first run.

kahliburke commented

2026-03-22 13:33:00 -07:00

(Migrated from github.com)

@prittjam Tested again and it is working here..Yes the 1.1 branch is in flux with other changes in Tachi. I will work on stabilizing and releasing it soon, but if you bypass the startup wizard with the workaround suggested above (I have not tried to do it in emacs) and you already have the dev/v2.0-preview of Tachi checked out, that should be a good combination until the official release Is there.

Let me know if I can assist in getting the setup worked out. It works very well for me but there are sometimes these environmental issues that are hard to predict.

@prittjam Tested again and it is working here..Yes the 1.1 branch is in flux with other changes in Tachi. I will work on stabilizing and releasing it soon, but if you bypass the startup wizard with the workaround suggested above (I have not tried to do it in emacs) and you already have the dev/v2.0-preview of Tachi checked out, that should be a good combination until the official release Is there. Let me know if I can assist in getting the setup worked out. It works very well for me but there are sometimes these environmental issues that are hard to predict.

prittjam commented

2026-03-22 13:46:27 -07:00

(Migrated from github.com)

Workaround: Disabling println stripping

For our workflow (CUDA kernel debugging with progress prints), the println stripping in the ex tool is a significant blocker. The agent can't see any stdout output, making it impossible to diagnose where long-running operations hang.

One-line fix that worked for us

In src/Kaimon.jl around line 766, commenting out the remove_println_calls line:

# Before:
expr = remove_println_calls(expr, true, quiet, was_stripped)

# After:
# expr = remove_println_calls(expr, true, quiet, was_stripped)

Feature request

Could the ex tool get an option to disable println stripping? For example:

ex(e="...", q=false, strip_println=false)

Or perhaps only strip when q=true (quiet mode), and pass through stdout when q=false? The current behavior where println is "ALWAYS stripped" (per the docs) means agents have zero visibility into long-running operations, compilation progress, or debugging output.

Context

We're developing CUDA kernels where JIT compilation takes 30+ seconds per kernel. Without println output, the agent can't tell which kernel is compiling, whether it's hung, or what errors occurred. We had to have the user manually paste REPL output back to the agent, which defeats the purpose of the MCP integration.

## Workaround: Disabling println stripping For our workflow (CUDA kernel debugging with progress prints), the `println` stripping in the `ex` tool is a significant blocker. The agent can't see any stdout output, making it impossible to diagnose where long-running operations hang. ### One-line fix that worked for us In `src/Kaimon.jl` around line 766, commenting out the `remove_println_calls` line: ```julia # Before: expr = remove_println_calls(expr, true, quiet, was_stripped) # After: # expr = remove_println_calls(expr, true, quiet, was_stripped) ``` ### Feature request Could the `ex` tool get an option to disable println stripping? For example: ``` ex(e="...", q=false, strip_println=false) ``` Or perhaps only strip when `q=true` (quiet mode), and pass through stdout when `q=false`? The current behavior where println is "ALWAYS stripped" (per the docs) means agents have zero visibility into long-running operations, compilation progress, or debugging output. ### Context We're developing CUDA kernels where JIT compilation takes 30+ seconds per kernel. Without println output, the agent can't tell which kernel is compiling, whether it's hung, or what errors occurred. We had to have the user manually paste REPL output back to the agent, which defeats the purpose of the MCP integration.

prittjam commented

2026-03-22 13:47:11 -07:00

(Migrated from github.com)

Update: Kaimon 1.1 SSE issue — more details

Tried Kaimon 1.1 again (with Tachikoma from dev/v2.0-preview for BoxTabs). The ex tool hangs from Claude Code's perspective:

ping works fine (returns immediately)
ex(e="1+1", q=false) — the eval runs, user sees 2 in REPL, but Claude Code never receives the response. Claude Code spins indefinitely until the user interrupts.
Tried with/without ses parameter, with s=true, all same behavior
This happens consistently, not intermittently

The MCP config:

{
  "mcpServers": {
    "kaimon": {
      "type": "http",
      "url": "http://127.0.0.1:2828/mcp",
      "headers": {
        "Authorization": "Bearer kaimon_..."
      }
    }
  }
}

Reverted to 1.0.1 where ex works (though println is stripped). Seems like the 1.1 SSE streaming path for ex results isn't completing/closing the response properly for HTTP MCP clients.

Environment

Claude Code CLI (latest)
Julia 1.12.5
Linux x86_64
HTTP transport (not stdio)

## Update: Kaimon 1.1 SSE issue — more details Tried Kaimon 1.1 again (with Tachikoma from `dev/v2.0-preview` for `BoxTabs`). The `ex` tool hangs from Claude Code's perspective: - `ping` works fine (returns immediately) - `ex(e="1+1", q=false)` — the eval runs, user sees `2` in REPL, but Claude Code never receives the response. Claude Code spins indefinitely until the user interrupts. - Tried with/without `ses` parameter, with `s=true`, all same behavior - This happens consistently, not intermittently The MCP config: ```json { "mcpServers": { "kaimon": { "type": "http", "url": "http://127.0.0.1:2828/mcp", "headers": { "Authorization": "Bearer kaimon_..." } } } } ``` Reverted to 1.0.1 where `ex` works (though println is stripped). Seems like the 1.1 SSE streaming path for `ex` results isn't completing/closing the response properly for HTTP MCP clients. ### Environment - Claude Code CLI (latest) - Julia 1.12.5 - Linux x86_64 - HTTP transport (not stdio)

kahliburke commented

2026-03-22 16:28:38 -07:00

(Migrated from github.com)

Diagnostic scripts and recommended patterns

We've been testing extensively and want to share diagnostic tools and findings. We're working to understand what's different in your environment — everything works reliably on our end, so we need your help isolating the issue.

How println stripping works

The ex tool strips println/print/printstyled from the agent's submitted code AST at all nesting levels. This is by design — agents should use q=false with return values, not println.

However, stdout from called functions is captured and returned. This is the key distinction:

What the agent sends	What happens
`println("debug"); 42`	println stripped, returns `42`
`my_function()` where my_function has println	println executes, stdout captured, returned with `q=false`
`include("script.jl")` where script has println	all println output returned with `q=false`

Tested examples from our session:

# Agent println stripped (by design):
ex(e='println("hello"); 42', q=false)
→ 42
  ⚠️ Note: println/print/logging calls were removed.

# Calling a function that prints — stdout returned:
ex(e='demo_compute_with_progress(3)', q=false)
→ Processing batch 1/3...
  Processing batch 2/3...
  Processing batch 3/3...
  Done! Processed 3 batches.
  1498.80...

# Simulated kernel compilation — all phases visible:
ex(e='demo_compile_kernel("matmul_fp16")', q=false)
→ Compiling kernel 'matmul_fp16'...
    [parsing] matmul_fp16
    [type inference] matmul_fp16
    [optimization] matmul_fp16
    [code generation] matmul_fp16
  Kernel 'matmul_fp16' compiled successfully.
  :matmul_fp16

# Errors with full stacktrace:
ex(e='include("/nonexistent.jl")', q=false)
→ ERROR: SystemError: opening file "/nonexistent.jl": No such file or directory
  Stacktrace: ...

Recommended pattern for long-running tasks: Gate.progress()

For CUDA kernel compilation and similar long-running work, define a custom GateTool that uses Gate.progress() to stream real-time updates to the agent:

using Kaimon.Gate: GateTool, progress

compile_tool = GateTool("compile_kernel", function(name::String)
    progress("Compiling $name: parsing...")
    # ... actual compilation ...
    progress("Compiling $name: type inference...")
    # ...
    progress("Compiling $name: code generation...")
    # ...
    return "Kernel $name compiled successfully"
end)

Gate.serve(tools=[compile_tool])

The agent calls compile_kernel(name="matmul") and each Gate.progress() call sends an SSE progress notification during execution. These are visible in the Kaimon TUI Activity tab and delivered to the MCP client.

Diagnostic scripts

Attached are test scripts to verify behavior in your environment — include demo_println_behavior.jl to get test functions, then call them with q=false to verify stdout flows back:

https://gist.github.com/kahliburke/44b0053a1715951419409c8a23219d23

Isolating the issue

Since the fixes above work reliably in our testing, we'd like to narrow down what's different in your setup. You mentioned using claude-code-ide.el for Emacs integration. While the MCP HTTP traffic between Claude Code and Kaimon isn't altered by the Emacs wrapper, the Claude Code CLI runs inside a terminal emulator buffer (vterm/eat), and the CLI may behave differently based on its terminal environment — TTY detection, output buffering, capability reporting, etc.

Could you try running Claude Code from a regular terminal (outside Emacs) and have the agent use the ex tool? If it works from a standard terminal but not from Emacs, that tells us the CLI's streaming behavior changes based on its terminal context, and we can investigate further from there.

## Diagnostic scripts and recommended patterns We've been testing extensively and want to share diagnostic tools and findings. We're working to understand what's different in your environment — everything works reliably on our end, so we need your help isolating the issue. ### How println stripping works The `ex` tool strips `println`/`print`/`printstyled` from the **agent's submitted code AST** at all nesting levels. This is by design — agents should use `q=false` with return values, not println. However, **stdout from called functions is captured and returned**. This is the key distinction: | What the agent sends | What happens | |---|---| | `println("debug"); 42` | println stripped, returns `42` | | `my_function()` where my_function has println | println executes, stdout captured, returned with `q=false` | | `include("script.jl")` where script has println | all println output returned with `q=false` | Tested examples from our session: ``` # Agent println stripped (by design): ex(e='println("hello"); 42', q=false) → 42 ⚠️ Note: println/print/logging calls were removed. # Calling a function that prints — stdout returned: ex(e='demo_compute_with_progress(3)', q=false) → Processing batch 1/3... Processing batch 2/3... Processing batch 3/3... Done! Processed 3 batches. 1498.80... # Simulated kernel compilation — all phases visible: ex(e='demo_compile_kernel("matmul_fp16")', q=false) → Compiling kernel 'matmul_fp16'... [parsing] matmul_fp16 [type inference] matmul_fp16 [optimization] matmul_fp16 [code generation] matmul_fp16 Kernel 'matmul_fp16' compiled successfully. :matmul_fp16 # Errors with full stacktrace: ex(e='include("/nonexistent.jl")', q=false) → ERROR: SystemError: opening file "/nonexistent.jl": No such file or directory Stacktrace: ... ``` ### Recommended pattern for long-running tasks: Gate.progress() For CUDA kernel compilation and similar long-running work, define a custom GateTool that uses `Gate.progress()` to stream real-time updates to the agent: ```julia using Kaimon.Gate: GateTool, progress compile_tool = GateTool("compile_kernel", function(name::String) progress("Compiling $name: parsing...") # ... actual compilation ... progress("Compiling $name: type inference...") # ... progress("Compiling $name: code generation...") # ... return "Kernel $name compiled successfully" end) Gate.serve(tools=[compile_tool]) ``` The agent calls `compile_kernel(name="matmul")` and each `Gate.progress()` call sends an SSE progress notification during execution. These are visible in the Kaimon TUI Activity tab and delivered to the MCP client. ### Diagnostic scripts Attached are test scripts to verify behavior in your environment — include `demo_println_behavior.jl` to get test functions, then call them with `q=false` to verify stdout flows back: https://gist.github.com/kahliburke/44b0053a1715951419409c8a23219d23 ### Isolating the issue Since the fixes above work reliably in our testing, we'd like to narrow down what's different in your setup. You mentioned using `claude-code-ide.el` for Emacs integration. While the MCP HTTP traffic between Claude Code and Kaimon isn't altered by the Emacs wrapper, the Claude Code CLI runs inside a terminal emulator buffer (vterm/eat), and the CLI may behave differently based on its terminal environment — TTY detection, output buffering, capability reporting, etc. Could you try running Claude Code from a regular terminal (outside Emacs) and have the agent use the `ex` tool? If it works from a standard terminal but not from Emacs, that tells us the CLI's streaming behavior changes based on its terminal context, and we can investigate further from there.

prittjam commented

2026-03-22 18:07:55 -07:00

(Migrated from github.com)

Thanks, I'll try this a bit later. Maybe I'll wait for 1.1 official release and revisit

kahliburke commented

2026-03-22 20:27:35 -07:00

(Migrated from github.com)

@prittjam Thank you, I will close as I am seeing the correct behavior and even have unit tests for this stuff. There is some other factor which we might be missing or some bug that you're hitting which I'm not. Let's revisit when you have the time to do a little investigation, as I'd really love for you to be able to use this, it is in my opinion a HUGE leap forward from what MCPRepl provided. 1.1 is actually getting pretty powerful.

Rows
Columns

ex tool should return stderr/errors to MCP client #14

Problem

Example

Suggestion

Environment

Update: Kaimon 1.1 + Claude Code SSE incompatibility

Status: Fixed in 1.1 branch

Error examples

SSE hang fix

Setup

Workaround: Disabling println stripping

One-line fix that worked for us

Feature request

Context

Update: Kaimon 1.1 SSE issue — more details

Environment

Diagnostic scripts and recommended patterns

How println stripping works

Recommended pattern for long-running tasks: Gate.progress()

Diagnostic scripts

Isolating the issue