Eben60 commented

(Migrated from github.com)

I have started to play around with your PendulumSim.jl, tried to run it through Kaimon, and ran into some problems. These might have been resolved - at least they were resolved for this specific situation. I am not even trying to understand what was happening, so below I am reproducing the report the authors of the fixes, i.e. the AI agents. See also my forks of PendulumSim.jl - https://github.com/Eben60/PendulumSim.jl , and of Kaimon.jl https://github.com/Eben60/Kaimon.jl/tree/2.0-integration_makie-fix (forked yesterday from https://github.com/kahliburke/Kaimon.jl/tree/2.0-integration ), where these fixes have been implemented. I hope this helps.

Title: [Julia 1.12 + macOS] Deadlocks, FieldErrors, and display stealing in gate_eval.jl with VSCodeServer

Description

While testing PendulumSim.jl via the Kaimon MCP on Julia 1.12 and macOS (with the VSCode extension active), we ran into a series of interconnected bugs that completely broke GUI evaluations (e.g., GLMakie).

We have identified three distinct issues and developed fixes for all of them.

1. `FieldError` on Julia 1.12

The Bug: In Julia 1.12, Base.active_repl_backend is a REPLBackendRef, which does not have the .ast_transforms field. Accessing it directly in gate_eval.jl and gate_stream.jl throws a FieldError, crashing Kaimon's evaluation loop.
The Fix: Wrap all accesses to ast_transforms with a hasproperty check:

if hasproperty(backend, :ast_transforms)
    # perform AST manipulation
end

2. macOS GUI Deadlock via VSCodeServer Background Thread

The Bug: When Kaimon pushes an AST to the REPL's repl_channel, VSCodeServer intercepts it and schedules the evaluation on its own backend task, which runs on a background thread. When evaluating a GUI function (like DoublePendulum.run_double_pendulum_gui()), GLMakie attempts to open a window using GLFW. On macOS, calling Cocoa framework functions from a background thread instantly and permanently freezes the event loop, deadlocking the VSCode backend task entirely.
The Fix: We had to hard-bypass the REPL channel in gate_eval.jl by forcing Kaimon to always evaluate directly on the interactive thread:

# Previously: if has_repl && on_interactive
if false 
    # push to repl_channel
else
    _eval_with_capture(expr)
end

3. VSCode Display Stealing closes interactive windows

The Bug: Once we successfully bypassed the REPL channel (fixing the deadlock), calling display(fig) caused the Makie window to flash and disappear immediately. This happens because VSCodeServer installs a global display hook. It intercepts the plot, renders it as a static image for the VSCode Plot Pane, and then GLMakie cleans up the native window.
The Fix: Temporarily remove VSCodeServer from Base.Multimedia.displays during the evaluation, and restore it in a finally block:

vscode_displays = Any[]
for (i, d) in enumerate(Base.Multimedia.displays)
    if occursin("VSCodeServer", string(typeof(d)))
        push!(vscode_displays, (i, d))
    end
end
for (i, d) in reverse(vscode_displays)
    deleteat!(Base.Multimedia.displays, i)
end

try
    val = _eval_with_capture(expr)
    _maybe_echo_result(val)
    return val
finally
    for (i, d) in vscode_displays
        insert!(Base.Multimedia.displays, i, d)
    end
end

Context

This was thoroughly debugged and tested using the run_double_pendulum_gui() from PendulumSim.jl. Applying these three fixes allows Kaimon to successfully launch and maintain interactive GLMakie windows on macOS without deadlocking the VSCode extension.

I have started to play around with your [PendulumSim.jl](https://github.com/kahliburke/PendulumSim.jl), tried to run it through Kaimon, and ran into some problems. These might have been resolved - at least they were resolved for this specific situation. I am not even trying to understand what was happening, so below I am reproducing the report the authors of the fixes, i.e. the AI agents. See also my forks of PendulumSim.jl - https://github.com/Eben60/PendulumSim.jl , and of Kaimon.jl https://github.com/Eben60/Kaimon.jl/tree/2.0-integration_makie-fix (forked yesterday from https://github.com/kahliburke/Kaimon.jl/tree/2.0-integration ), where these fixes have been implemented. I hope this helps. ---- # Title: [Julia 1.12 + macOS] Deadlocks, FieldErrors, and display stealing in gate_eval.jl with VSCodeServer ## Description While testing `PendulumSim.jl` via the Kaimon MCP on Julia 1.12 and macOS (with the VSCode extension active), we ran into a series of interconnected bugs that completely broke GUI evaluations (e.g., `GLMakie`). We have identified three distinct issues and developed fixes for all of them. ### 1. `FieldError` on Julia 1.12 **The Bug:** In Julia 1.12, `Base.active_repl_backend` is a `REPLBackendRef`, which does not have the `.ast_transforms` field. Accessing it directly in `gate_eval.jl` and `gate_stream.jl` throws a `FieldError`, crashing Kaimon's evaluation loop. **The Fix:** Wrap all accesses to `ast_transforms` with a `hasproperty` check: ```julia if hasproperty(backend, :ast_transforms) # perform AST manipulation end ``` ### 2. macOS GUI Deadlock via VSCodeServer Background Thread **The Bug:** When Kaimon pushes an AST to the REPL's `repl_channel`, `VSCodeServer` intercepts it and schedules the evaluation on its own backend task, which runs on a **background thread**. When evaluating a GUI function (like `DoublePendulum.run_double_pendulum_gui()`), `GLMakie` attempts to open a window using GLFW. On macOS, calling Cocoa framework functions from a background thread instantly and permanently freezes the event loop, deadlocking the VSCode backend task entirely. **The Fix:** We had to hard-bypass the REPL channel in `gate_eval.jl` by forcing Kaimon to always evaluate directly on the interactive thread: ```julia # Previously: if has_repl && on_interactive if false # push to repl_channel else _eval_with_capture(expr) end ``` ### 3. VSCode Display Stealing closes interactive windows **The Bug:** Once we successfully bypassed the REPL channel (fixing the deadlock), calling `display(fig)` caused the Makie window to flash and disappear immediately. This happens because `VSCodeServer` installs a global display hook. It intercepts the plot, renders it as a static image for the VSCode Plot Pane, and then GLMakie cleans up the native window. **The Fix:** Temporarily remove `VSCodeServer` from `Base.Multimedia.displays` during the evaluation, and restore it in a `finally` block: ```julia vscode_displays = Any[] for (i, d) in enumerate(Base.Multimedia.displays) if occursin("VSCodeServer", string(typeof(d))) push!(vscode_displays, (i, d)) end end for (i, d) in reverse(vscode_displays) deleteat!(Base.Multimedia.displays, i) end try val = _eval_with_capture(expr) _maybe_echo_result(val) return val finally for (i, d) in vscode_displays insert!(Base.Multimedia.displays, i, d) end end ``` ### Context This was thoroughly debugged and tested using the `run_double_pendulum_gui()` from `PendulumSim.jl`. Applying these three fixes allows Kaimon to successfully launch and maintain interactive GLMakie windows on macOS without deadlocking the VSCode extension.

👀 1

kahliburke commented

2026-06-28 01:05:55 -07:00

(Migrated from github.com)

@Eben60 , this is specific to VSCode interactions with the REPL when using the Julia extension for VSCode?

Eben60 commented

2026-06-28 02:37:24 -07:00

(Migrated from github.com)

Sorry, I forgot to supply an important detail: Antigravity IDE was used. I only tested it in the VSCode REPL, i.e. with the active VSCode extension.

Sorry, I forgot to supply an important detail: Antigravity IDE was used. I _only_ tested it in the VSCode REPL, i.e. with the active VSCode extension.

kahliburke commented

2026-06-28 13:11:16 -07:00

(Migrated from github.com)

Thanks for the detailed writeup @Eben60, and for doing the work in your fork. I ran all three down using your PendulumSim run_double_pendulum_gui() across a few setups.

Short version: the GUI works fine as long as the gate isn't sharing the IDE extension's REPL.

Environment	VSCodeServer	active_repl_backend	inline display	GUI
Standalone REPL	no	REPLBackend (has ast_transforms)	none	works
Kaimon-spawned session	no	REPLBackend	none	works
Standard VSCode REPL (1.12.6)	yes	REPLBackend	InlineDisplay	works, window stayed open
Antigravity REPL (your setup)	yes	REPLBackendRef (no ast_transforms)	InlineDisplay	all three bugs

I couldn't reproduce 1 or 2 even in standard VSCode. There the backend is a real REPLBackend and mt=true evals land on thread 1, so no FieldError and no background-thread deadlock. They look specific to how Antigravity wraps the backend (REPLBackendRef + eval on a background thread).

The path I'd recommend: for GUI/GLMakie work, let Kaimon spawn its own session instead of attaching the gate to the extension's REPL. A spawned process never loads VSCodeServer, so there's no backend hijacking, no display stealing, no FieldError, and it works while you keep editing in your IDE. I ran the full double pendulum that way and the window opens on thread 1, animates, and the gate stays responsive.

What I'm taking from your fork: fix 1, as a hasproperty(backend, :ast_transforms) guard so that if a gate ever does land on a REPLBackendRef it skips the transforms (softscope/auto-Revise, which that backend can't do anyway) instead of crashing.

What I'm holding off on: the call_on_backend bypass (2) and pulling VSCodeServer out of the display stack (3). To make them safe in general they'd regress things that currently work. 2's hard bypass loses the thread-1 pinning that keeps GLMakie reliable standalone, and 3 would stop inline plots showing up in the VSCode plot pane for everyone (e.g. CairoMakie users who want that). The spawned-session path sidesteps all of it, so I'd rather point there than special-case the embedded REPL. Happy to revisit if running inside the IDE REPL matters for your workflow.

Thanks for the detailed writeup @Eben60, and for doing the work in your fork. I ran all three down using your PendulumSim `run_double_pendulum_gui()` across a few setups. Short version: the GUI works fine as long as the gate isn't sharing the IDE extension's REPL. | Environment | VSCodeServer | active_repl_backend | inline display | GUI | |---|---|---|---|---| | Standalone REPL | no | REPLBackend (has ast_transforms) | none | works | | Kaimon-spawned session | no | REPLBackend | none | works | | Standard VSCode REPL (1.12.6) | yes | REPLBackend | InlineDisplay | works, window stayed open | | Antigravity REPL (your setup) | yes | REPLBackendRef (no ast_transforms) | InlineDisplay | all three bugs | I couldn't reproduce 1 or 2 even in standard VSCode. There the backend is a real REPLBackend and `mt=true` evals land on thread 1, so no FieldError and no background-thread deadlock. They look specific to how Antigravity wraps the backend (REPLBackendRef + eval on a background thread). The path I'd recommend: for GUI/GLMakie work, let Kaimon spawn its own session instead of attaching the gate to the extension's REPL. A spawned process never loads VSCodeServer, so there's no backend hijacking, no display stealing, no FieldError, and it works while you keep editing in your IDE. I ran the full double pendulum that way and the window opens on thread 1, animates, and the gate stays responsive. What I'm taking from your fork: fix 1, as a `hasproperty(backend, :ast_transforms)` guard so that if a gate ever does land on a REPLBackendRef it skips the transforms (softscope/auto-Revise, which that backend can't do anyway) instead of crashing. What I'm holding off on: the call_on_backend bypass (2) and pulling VSCodeServer out of the display stack (3). To make them safe in general they'd regress things that currently work. 2's hard bypass loses the thread-1 pinning that keeps GLMakie reliable standalone, and 3 would stop inline plots showing up in the VSCode plot pane for everyone (e.g. CairoMakie users who want that). The spawned-session path sidesteps all of it, so I'd rather point there than special-case the embedded REPL. Happy to revisit if running inside the IDE REPL matters for your workflow.

kahliburke commented

2026-07-08 22:17:49 -07:00

(Migrated from github.com)

Closing this out since 2.0 has shipped, and addresses at least some of the surface described here. But please open specific issues if you run into them.

Rows
Columns

Kaimon / GLMakie interaction problems #56

Title: [Julia 1.12 + macOS] Deadlocks, FieldErrors, and display stealing in gate_eval.jl with VSCodeServer

Description

1. FieldError on Julia 1.12

2. macOS GUI Deadlock via VSCodeServer Background Thread

3. VSCode Display Stealing closes interactive windows

Context

1. `FieldError` on Julia 1.12