Provide Qdrant via JLL? (lazily?)

bdklahn commented

2026-04-13 13:07:28 -07:00

(Migrated from github.com)

Hello,
Kaimon.jl looks really cool! Thanks for creating this. I think it opens possibilities which I don't think many fully grasp, yet!

A while back, I threw this thing up on Yggdrasil.
https://github.com/JuliaPackaging/Yggdrasil/blob/master/Q/Qdrant/build_tarballs.jl
(I'm not affiliated with Qdrant; just though it was a good implementation)

It is kinda quick 'n dirty/lazy, because I maybe could/should actually do the/a Rust build, rather than simply pull the precompiled upstream binaries. Also, it kinda requires any updater to grab the new upstream hashes, etc., rather than maybe just being able to update the version string.

I just wondered if that could be useful to be able to dynamically pull Qdrant . . . maybe lazily, on first use.

I had meant to engage the upstream Qdrant folks, to see if they'd have any interest in maintaining that build_tarballs.jl file. For them, it could be a(n alternative) binary source. I just haven't had time.

Thanks

Hello, Kaimon.jl looks really cool! Thanks for creating this. I think it opens possibilities which I don't think many fully grasp, yet! A while back, I threw this thing up on Yggdrasil. https://github.com/JuliaPackaging/Yggdrasil/blob/master/Q/Qdrant/build_tarballs.jl (I'm not affiliated with Qdrant; just though it was a good implementation) It is kinda quick 'n dirty/lazy, because I maybe could/should actually do the/a Rust build, rather than simply pull the precompiled upstream binaries. Also, it kinda requires any updater to grab the new upstream hashes, etc., rather than maybe just being able to update the version string. I just wondered if that could be useful to be able to dynamically pull Qdrant . . . maybe lazily, on first use. I had meant to engage the upstream Qdrant folks, to see if they'd have any interest in maintaining that build_tarballs.jl file. For them, it could be a(n alternative) binary source. I just haven't had time. Thanks

kahliburke commented

2026-04-14 19:55:52 -07:00

(Migrated from github.com)

@bdklahn Thanks for the feedback. Hmm it's an interesting idea, although it's also really simple to run the docker container. So I wonder if this path might add more complexity than it's solving? The docker container is maintained and is documented as the recommended installation path.

So, I have mixed feelings about this approach. Did you find installing and running through docker to be too cumbersome?

@bdklahn Thanks for the feedback. Hmm it's an interesting idea, although it's also really simple to run the docker container. So I wonder if this path might add more complexity than it's solving? The docker container is maintained and is documented as the recommended installation path. So, I have mixed feelings about this approach. Did you find installing and running through docker to be too cumbersome?

bdklahn commented

2026-04-15 12:47:50 -07:00

(Migrated from github.com)

I guess I was considering this as a way to add less complexity for users. I'm a software engineer with a lot of sysadmin experience. So I, myself, have no problem installing and running Docker, or Podman, or Apptainer, etc. I know how they work and have used and modified them a lot.
But requiring users to run two separate extra services to get the most benefit out of a package . . .

I'm not 100% sure why/how the Qdrant folks are using Docker. Sometimes people use it to bootstrap pods for scalable services (E.g. with Kubernetes, etc.), for their paid hosted services. Some use it to mitigate their glibc deps not always matching hosts. But, theoretically, they could be doing static musl for glibc and making their Rust binary very portable (not needing any chroot "containment" work around).

BinaryBuilder is a way to get something which works and is published in an easy way to leverage Julia Pkg Artifacts as a separate "module" (package).

I work with software developers, engineers, domain experts, and "users". All of them would prefer anything outside their scope of interest to be more "declarative" vs. "imperative".

] add ThisPackage
vs.
Adding a bunch of documentation lines to say . . .
Go figure out how to . . .
Install Docker (Desktop)
Start some daemon
Run (pull) a container.
Install Ollama
etc.

Idk . . . Re: containers: sometimes it feels a bit overkill to stack a bunch binary blobs into an overlay filesystem (or flatten into a squashfs), and then run a VM on non-native systems, just to run a single server binary.

Sometimes that is the best/only thing you can do. But I think things like the Julia Artifacts system and BinaryBuilder at least offer the possibility of not having to brute force it like that.

But maybe I'm missing something. I thought the Rust toolchain was capable of creating portable binaries. Did Qdrant not do that?

I guess I was considering this as a way to add less complexity for users. I'm a software engineer with a lot of sysadmin experience. So I, myself, have no problem installing and running Docker, or Podman, or Apptainer, etc. I know how they work and have used and modified them a lot. But requiring users to run two separate extra services to get the most benefit out of a package . . . I'm not 100% sure why/how the Qdrant folks are using Docker. Sometimes people use it to bootstrap pods for scalable services (E.g. with Kubernetes, etc.), for their paid hosted services. Some use it to mitigate their glibc deps not always matching hosts. But, theoretically, they could be doing static musl for glibc and making their Rust binary very portable (not needing any chroot "containment" work around). BinaryBuilder is a way to get something which works and is published in an easy way to leverage Julia Pkg Artifacts as a separate "module" (package). I work with software developers, engineers, domain experts, and "users". All of them would prefer anything outside their scope of interest to be more "declarative" vs. "imperative". ] add ThisPackage vs. Adding a bunch of documentation lines to say . . . Go figure out how to . . . Install Docker (Desktop) Start some daemon Run (pull) a container. Install Ollama etc. Idk . . . Re: containers: sometimes it feels a bit overkill to stack a bunch binary blobs into an overlay filesystem (or flatten into a squashfs), and then run a VM on non-native systems, just to run a single server binary. Sometimes that is the best/only thing you can do. But I think things like the Julia Artifacts system and BinaryBuilder at least offer the possibility of not having to brute force it like that. But maybe I'm missing something. I thought the Rust toolchain was capable of creating portable binaries. Did Qdrant not do that?

kahliburke commented

2026-04-15 18:24:10 -07:00

(Migrated from github.com)

@bdklahn, yeah I don't know the full story, I'm just working off of their docs:

https://qdrant.tech/documentation/quickstart/#how-to-get-started-with-qdrant-locally

This just recommends running via docker. I'm game to try an alternative method, in the end all Kaimon cares about is an API to call at a known port. Currently it's actually hardcoded to 6333 which is the default. Alternative paths to this end are welcome. Do you want to take a stab at a PR?

@bdklahn, yeah I don't know the full story, I'm just working off of their docs: https://qdrant.tech/documentation/quickstart/#how-to-get-started-with-qdrant-locally This just recommends running via docker. I'm game to try an alternative method, in the end all Kaimon cares about is an API to call at a known port. Currently it's actually hardcoded to 6333 which is the default. Alternative paths to this end are welcome. Do you want to take a stab at a PR?

kahliburke commented

2026-07-08 22:20:18 -07:00

(Migrated from github.com)

@bdklahn Closing this out due to lack of response, but if it remains something of interest we can look at it again. I've been focused on releasing 2.0 and encourage you to try it.

bdklahn commented

2026-07-09 06:22:55 -07:00

(Migrated from github.com)

@bdklahn Closing this out due to lack of response, but if it remains something of interest we can look at it again. I've been focused on releasing 2.0 and encourage you to try it.

Sorry I didn't even respond. I've been very busy with work (and my mom's failing health) and have barely had any time.

If you close as complete a feature request Github Issue with question marks in the title, that seems to imply to users that you have zero interest in even considering it.

I did do a quick test with the Quarto_jll, with having the julia process follow the spawned quarto process, and kill when the parent exited. It seemed to work fine. The Quarto server ran, created the db files, responded to request, and shutdown after Juila did.
I've got about a dozen other concurrent softwares that I'm working on, so I'd need to go find where I put that. But it took seconds for an LLM response to produce something which worked. It was then a matter of asking you if/what (serve?) function flag you might want for that.

I have no stake in Quarto (other than a quick contrib to make the binaries easy to get via Juila), nor this, since I haven't even had time to really explore Kaimon. I was just passing through because this looked like a good pattern for AI use. I wondered, since Quarto is already available via jll, why the obvious wasn't done: use the jll as a runtime start option, vs. expect users to install/use Docker separately. Rust binaries are particularly useful as small, single-file (static-compiled), Artifacts. But if you don't understand how Pkg Artifacts and JLLs work, you might not be able to maintain that, anyway. In that case it's probably better, to just leave this closed.

> [@bdklahn](https://github.com/bdklahn) Closing this out due to lack of response, but if it remains something of interest we can look at it again. I've been focused on releasing 2.0 and encourage you to try it. Sorry I didn't even respond. I've been very busy with work (and my mom's failing health) and have barely had any time. If you close as complete a feature request Github Issue with question marks in the title, that seems to imply to users that you have zero interest in even considering it. I did do a quick test with the Quarto_jll, with having the julia process follow the spawned quarto process, and kill when the parent exited. It seemed to work fine. The Quarto server ran, created the db files, responded to request, and shutdown after Juila did. I've got about a dozen other concurrent softwares that I'm working on, so I'd need to go find where I put that. But it took seconds for an LLM response to produce something which worked. It was then a matter of asking you if/what (serve?) function flag you might want for that. I have no stake in Quarto (other than a quick contrib to make the binaries easy to get via Juila), nor this, since I haven't even had time to really explore Kaimon. I was just passing through because this looked like a good pattern for AI use. I wondered, since Quarto is already available via jll, why the obvious wasn't done: use the jll as a runtime start option, vs. expect users to install/use Docker separately. Rust binaries are particularly useful as small, single-file (static-compiled), Artifacts. But if you don't understand how Pkg Artifacts and JLLs work, you might not be able to maintain that, anyway. In that case it's probably better, to just leave this closed.

kahliburke commented

2026-07-09 10:36:30 -07:00

(Migrated from github.com)

@bdklahn I just hadn't heard back on this and was trying to tidy up, we can certainly reopen the issue if there's interest in it. Not saying it wouldn't work.

kahliburke commented

2026-07-19 19:37:55 -07:00

(Migrated from github.com)

Shipped in Kaimon v2.1.0. 🎉

@bdklahn — this is exactly what you suggested. Kaimon now runs Qdrant with zero Docker: it lazily installs Qdrant_jll into a small Kaimon-owned environment and launches the binary itself, tied to Kaimon's lifecycle (parent-follows-child, so it shuts down cleanly on exit) — the same pattern you prototyped with Quarto_jll. There are TUI controls to start/stop it, and it can autostart on launch.

Thanks for pushing on this, and for the nudge toward the JLL/Artifacts route — you were right that it's the more declarative, ]add-and-go path. Closing as implemented; hope things have eased up on your end.

Shipped in Kaimon v2.1.0. 🎉 @bdklahn — this is exactly what you suggested. Kaimon now runs Qdrant with zero Docker: it lazily installs `Qdrant_jll` into a small Kaimon-owned environment and launches the binary itself, tied to Kaimon's lifecycle (parent-follows-child, so it shuts down cleanly on exit) — the same pattern you prototyped with `Quarto_jll`. There are TUI controls to start/stop it, and it can autostart on launch. Thanks for pushing on this, and for the nudge toward the JLL/Artifacts route — you were right that it's the more declarative, `]add`-and-go path. Closing as implemented; hope things have eased up on your end.

bdklahn commented

2026-07-20 04:39:42 -07:00

(Migrated from github.com)

Very cool. Thanks!
I see your package as something very next level. Whenever I watch LLM traces calling the bash shell commands, I think about things like giving them the exact introspection of code and objects, like you can do with julia, and letting them use (or build?) custom fast code. Lately I've been playing with Goose. It provides some of it's own modern tools via things like ripgrep, etc. But I'd consider the pattern here as a potential way to build up custom (fast) tools. And where there are good (fast) modern tools already available, as seems to be the case with Rust binaries, I like how Artifacts were designed to make that reproducible and declarative.
Anyway, I'd often rather work on stuff like this than my real job. I'm trying to find a way to combine.

Very cool. Thanks! I see your package as something very next level. Whenever I watch LLM traces calling the bash shell commands, I think about things like giving them the exact introspection of code and objects, like you can do with julia, and letting them use (or build?) custom fast code. Lately I've been playing with [Goose](https://goose-docs.ai/). It provides some of it's own modern tools via things like [ripgrep](https://github.com/burntsushi/ripgrep), etc. But I'd consider the pattern here as a potential way to build up custom (fast) tools. And where there are good (fast) modern tools already available, as seems to be the case with Rust binaries, I like how Artifacts were designed to make that reproducible and declarative. Anyway, I'd often rather work on stuff like this than my real job. I'm trying to find a way to combine.

❤️ 1

kahliburke commented

2026-07-20 04:44:56 -07:00

(Migrated from github.com)

@bdklahn Please check out https://github.com/kahliburke/KaimonSlate.jl

bdklahn commented

2026-07-20 05:50:45 -07:00

(Migrated from github.com)

@bdklahn Please check out https://github.com/kahliburke/KaimonSlate.jl

That's very interesting. I was just now looking at Bonito. I've tried BonitoBook.jl. It still a little rough, but I like the "Julia first" idea, and having the flexibility to layout in a CSS grid, etc. (vs. Pluto.jl). I like leveraging WGL and a fast communication path. I agree with both the opinions about Pluto.jl. E.g. nice but not for me, yet I really like their connected cell deps. It looks like your're dealing with that part with KaimonSlate.jl. I love DAGs . . . for so much.

> [@bdklahn](https://github.com/bdklahn) Please check out https://github.com/kahliburke/KaimonSlate.jl That's very interesting. I was just now looking at [Bonito](https://bonito.sh/stable/index.html). I've tried [BonitoBook.jl](https://github.com/SimonDanisch/BonitoBook.jl). It still a little rough, but I like the "Julia first" idea, and having the flexibility to layout in a CSS grid, etc. (vs. [Pluto.jl](https://github.com/JuliaPluto/Pluto.jl)). I like leveraging WGL and a fast communication path. I agree with both the opinions about Pluto.jl. E.g. nice but not for me, yet I really like their connected cell deps. It looks like your're dealing with that part with KaimonSlate.jl. I love DAGs . . . for so much.

Rows
Columns

Provide Qdrant via JLL? (lazily?) #24