Repository navigation
docs: add Rapid-MLX provider guide - #4660
raullenchai wants to merge 3 commits into
Conversation
|
👋 Some commits in this PR are not signed and verified by GitHub. Please sign your commits with a GPG or SSH key registered in your GitHub account, then force-push. Commits that are not verified: See GitHub's guide on signing commits for setup instructions. I've added |
ab26ff5 to
6280f4a
Compare
|
Thanks @aheritier — the commit is now signed and verified, and rebased on the latest main (only change: nav entry placed alphabetically before Runcrate). |
There was a problem hiding this comment.
🟡 Changes recommended
The documentation needs Python 3.12+ as the prerequisite for the current Rapid-MLX release and a trusted-host setting for host.docker.internal to prevent HTTP 400 errors in the container setup.
2 open issues
What changed in this PR
This PR documents how to connect Docker Agent to Rapid-MLX’s local inference server on Apple Silicon using a custom provider definition. It adds documentation and an example without changing Go code.
Changes:
- Adds a guide covering setup, authentication, Chat and Responses APIs, tool calling, and troubleshooting.
- Provides a runnable configuration using
qwen3.5-4b-4bit. - Adds navigation and links from the provider overview and local-model guide.
| File | Description |
|---|---|
examples/rapid-mlx.yaml |
Adds a local chat configuration with optional authentication. |
docs/providers/rapid-mlx/index.md |
Documents Rapid-MLX setup and integration. |
docs/providers/overview/index.md |
Links to the new provider guide. |
docs/providers/local/index.md |
Includes Rapid-MLX among local model servers. |
docs/data/nav.yml |
Adds the Rapid-MLX navigation entry. |
Review effort: Balanced
aheritier
left a comment
There was a problem hiding this comment.
Reviewed all five changed files against the custom-provider implementation and Rapid-MLX release sources.
Prior threads:
- 4552493046: still open [should-fix] (high confidence) — Rapid-MLX 1.0.3 requires Python >=3.12; change the Python 3.10 prerequisite to 3.12+ to avoid installing an older release.
- 4552493144: still open [blocking] (high confidence) — The documented container URL receives HTTP 400 because host.docker.internal is absent from the default Host allowlist; add it with --trusted-hosts alongside the bind address and API-key configuration.
|
Thanks for the careful review. Both findings are addressed:
I checked this against the Rapid-MLX source and |
eb3f0c4 to
a7db056
Compare
There was a problem hiding this comment.
🟡 Changes recommended
The Rapid-MLX guide still has unreliable setup and container-authentication instructions, inaccurate model and tool-support assumptions, and an API-key exposure risk.
6 open issues
Use an isolated Python 3.12+ environment for pip installation · New Document model memory requirements and an 8 GB alternative · New Stop the existing server before starting this one · New Qualify automatic parser support and link to the tool-calling guide · New Pass the API key through the environment, not command-line arguments · New Pass the API key into the Docker Agent container · New
2 resolved since last review
Review effort: Balanced
| ```bash | ||
| $ brew install rapid-mlx | ||
| # or | ||
| $ pip install rapid-mlx |
| 2. Start the server with a model. For a small first download, use the | ||
| `qwen3.5-4b-4bit` alias | ||
| ([`mlx-community/Qwen3.5-4B-MLX-4bit`](https://huggingface.co/mlx-community/Qwen3.5-4B-MLX-4bit)): |
| $ export RAPID_MLX_API_KEY=your-server-api-key | ||
| $ rapid-mlx serve qwen3.5-4b-4bit --port 8000 | ||
| $ curl http://127.0.0.1:8000/v1/models -H "Authorization: Bearer $RAPID_MLX_API_KEY" | ||
| $ docker agent run rapid-mlx.yaml --exec "Hello" |
| The example is chat-only. Rapid-MLX parses tool calls for the models it | ||
| serves, and the parser is selected automatically from the model alias, but | ||
| reliability still depends on the model, instructions, and tool schemas. A 4B | ||
| model is useful for testing the connection; choose a larger tool-capable model | ||
| for agentic coding work. |
| Host allowlist: | ||
|
|
||
| ```bash | ||
| $ rapid-mlx serve qwen3.5-4b-4bit --port 8000 --host 0.0.0.0 --api-key your-server-api-key --trusted-hosts host.docker.internal |
| Configure Docker Agent with the same API key as described in | ||
| [Authentication](#authentication). `--trusted-hosts` replaces the default | ||
| loopback allowlist; add `localhost 127.0.0.1` if you also need local access. |
Signed-off-by: raullenchai <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Raullen <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Raullen <[email protected]>
a7db056 to
974feb8
Compare
|
Thanks for the review — addressed all six points in 974feb8 (rebased on main):
|

Rapid-MLX is an open-source (Apache-2.0) local inference server for Apple Silicon that serves MLX models behind OpenAI-compatible Chat Completions and Responses endpoints. It isn't covered in the provider guides yet, so users have no documented way to configure it.
This adds a provider guide at
docs/providers/rapid-mlx/index.md, following the same structure as the existing local provider guides. It covers:extra_bodyis only forwarded for Chat;host.docker.internal.A runnable example at
examples/rapid-mlx.yamluses the smallqwen3.5-4b-4bitmodel. The provider is added todocs/data/nav.ymland linked from the local and overview provider pages.Docs and example only; no Go code changes.
go test ./pkg/configpasses, including parsing and schema validation of the new example, and markdownlint is clean.I maintain Rapid-MLX and am happy to keep this page current.