How to Run Any Model in DeepSeek Harness ?

DeepSeek Harness (dsh) ships with DeepSeek’s own models wired in, but you’re not locked to them. The harness treats model providers as configuration: point a provider block at any OpenAI-compatible endpoint, hand it a credential reference, and your agent sessions run on whatever model sits behind that URL. A local Ollama instance, a company gateway, Qwen through DashScope’s compatible mode, or the big catalog providers like Anthropic and OpenAI all plug into the same block.

This guide walks through that block key by key, then builds three working recipes: a local model, a hosted OpenAI-compatible endpoint, and the built-in catalog providers. Everything quoted here comes from the official providers guide on the master branch, fetched August 20, 2026. One caveat up front: dsh is a developer preview, and the README warns in all caps that there will be compatibility-breaking changes. Check the docs against your installed version before you copy anything into production.

If you’re new to the harness itself, start with what DeepSeek Harness is and how it works, then come back here for the provider plumbing.

Why swap models in an agent harness at all

An agent harness is a loop: the model plans, calls tools, reads results, and repeats. The harness owns the loop; the model is an ingredient. Three reasons you’d change the ingredient:

Cost. Agent sessions burn tokens fast because every tool result gets fed back into context. Routing routine sessions to a cheaper model, or to DeepSeek V4-Flash instead of V4-Pro, changes your bill without changing your workflow. You can keep an expensive frontier model configured for the sessions that need it.

Data locality. Some codebases can’t leave the building. A provider block pointed at a model running on your own hardware means prompts, file contents, and tool outputs never cross the network. Same harness, same UI, zero egress.

Local development. When you’re building plugins or testing agent behavior, you don’t want every iteration to cost API credits or depend on your network. A small local model answers fast enough to test the loop, and you swap the real model back in when behavior matters.

The design follows from dsh’s architecture: everything in the harness is a plugin, and the model adapter is one of the replaceable pieces. Provider routes are owned by the dsh-llm-pi-ai plugin, which is documented in the repo’s plugin config catalog as holding “provider routes this instance owns.” That’s the machinery. The user-facing surface is one YAML block.

The provider block, key by key

Custom providers live in $DSH_HOME/settings.yaml, and you can also create them from the web UI under Settings → Models. Here’s the example straight from the official docs:

llm-pi-ai:
  providers:
    my-gateway:
      apiKeyEnv: GATEWAY_API_KEY
      api: openai-completions
      baseURL: https://gateway.example/v1
      models:
        - id: legacy-chat
        - id: vision-preview
          input: [text, image]

What each key does:

• my-gateway is the provider ID. It’s a permanent identifier, so pick a name you can live with; the display name shown in the UI is set separately.
• apiKeyEnv names the environment variable that holds your API key. The settings file never contains the secret itself, only this reference. More on where the actual key lives below.
• api declares the wire protocol. openai-completions is the documented value for OpenAI-compatible endpoints, which is what makes the “any model” promise work: most gateways, local runtimes, and hosted providers speak this protocol.
• baseURL is the endpoint root the harness sends requests to.
• models lists the model IDs available through this provider. Each entry needs at least an id, which must match what the endpoint expects in the request body.
• input declares modalities per model. Custom models default to text-only, so a vision model must explicitly declare input: [text, image] or image attachments won’t reach it. There’s also a route-level defaultInput that sets a fallback for every model in the provider; a model-level input overrides it.
• compat holds compatibility switches for endpoints that deviate from stock OpenAI behavior. The docs call out two: supportsDeveloperRole: false for backends that reject the developer role, and maxTokensField: max_tokens for backends that want the older output-cap field name. Compat can be set at the route level or per model.

One convenience worth knowing: when you add a custom provider through the web UI, a “Fetch available models” option queries the endpoint’s OpenAI-compatible GET /models route and fills the model list for you. If your endpoint implements that route, you skip the manual typing.

Where the actual API key lives

Secrets are stored write-only in $DSH_HOME/.credentials.yaml. After you save a key through the UI, dsh returns only a redacted descriptor; the literal value is never shown again. settings.yaml holds references (apiKeyEnv names, credential descriptors), never the keys themselves. That split means you can commit or share a settings file without leaking anything, and rotate a key without touching provider config.

Recipe 1: run a local model through Ollama

Ollama exposes an OpenAI-compatible API at http://localhost:11434/v1, which Ollama documents in its own OpenAI compatibility guide. Since dsh speaks openai-completions to any base URL, the pairing is straightforward.

[VERIFY: the dsh docs don’t show an Ollama-specific example; this recipe applies the documented custom-provider schema to Ollama’s documented OpenAI-compatible endpoint. Test on your install before publishing internally.]

llm-pi-ai:
  providers:
    ollama-local:
      apiKeyEnv: OLLAMA_API_KEY
      api: openai-completions
      baseURL: http://localhost:11434/v1
      models:
        - id: gpt-oss:20b
        - id: qwen3

Notes on this one:

• Ollama doesn’t require an API key locally, but the schema expects a credential reference, so set a dummy value: export OLLAMA_API_KEY=ollama. Ollama ignores whatever you send.
• The model id must match the tag Ollama serves. Run ollama list and copy the names exactly, tag included.
• Pull the model first (ollama pull gpt-oss:20b) and confirm the server responds before wiring it into dsh. We covered the full local setup in how to run GPT-OSS using Ollama, and the same pattern works for other open-weight models like Kimi K3 if your hardware is up to it.

A quick sanity check saves you a confusing agent session: hit http://localhost:11434/v1/models in Apidog before touching dsh config. If that request returns your model list, the base URL is right, the server is up, and “Fetch available models” in the dsh UI will work too. If it doesn’t, no amount of harness configuration will fix it.

Expectation management: agent harnesses lean hard on tool calling and long context. Small local models handle the loop for testing, but they’ll plan worse and drop tool calls more often than the frontier models the harness was built around. That’s fine for plugin development; it’s frustrating for real work.

Recipe 2: a hosted OpenAI-compatible endpoint (Qwen via DashScope)

For a hosted example, pick a vendor that documents its OpenAI compatibility rather than one you assume has it. Alibaba Cloud Model Studio (DashScope) does: its OpenAI compatibility page documents a /compatible-mode/v1 endpoint for Qwen models, with regional, workspace-specific domains (for Singapore: https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1) and authentication via the DASHSCOPE_API_KEY environment variable.

Mapped onto the dsh schema:

llm-pi-ai:
  providers:
    qwen-dashscope:
      apiKeyEnv: DASHSCOPE_API_KEY
      api: openai-completions
      baseURL: https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1
      models:
        - id: qwen3-max

Replace {WorkspaceId} with your actual workspace domain from the Model Studio console, and check the vendor’s model list for current IDs; we keep a rundown of the flagship tier in our Qwen 3.8 API guide. The same pattern extends to any vendor with documented OpenAI compatibility: Moonshot’s Kimi API, OpenRouter, a vLLM deployment, or your company’s internal gateway. The only parts that change are baseURL, the env var name, and the model IDs. If you’ve configured open-source models in Codex, this will feel familiar; dsh’s YAML block plays the same role as Codex’s model_providers config.

Two hosted-endpoint specifics:

• If the vendor’s endpoint rejects requests with odd errors about roles or token fields, that’s what the compat switches exist for. Try supportsDeveloperRole: false first; older OpenAI-compatible implementations predate the developer role.
• Vision models must declare input: [text, image] explicitly, even if the hosted model supports images. dsh assumes text-only for custom models unless told otherwise.

Recipe 3: the built-in catalog providers

You don’t need a custom block for the mainstream clouds. dsh ships catalog providers for DeepSeek, Anthropic, and OpenAI, where setup is mostly “paste an API key.” Specialty catalog entries carry their own native auth flows: Bedrock uses AWS credentials, Vertex wants an ADC project, Azure needs its api-version, and Codex authenticates via OAuth.

Catalog providers are the low-friction path when you just want Claude or GPT behind the harness, and they’re how most people will run DeepSeek V4-Pro, whose API launch in August 2026 arrived alongside the harness itself (details at api-docs.deepseek.com). Custom providers are for everything the catalog doesn’t cover: local runtimes, gateways, regional vendors, and OpenAI-compatible aggregators.

Selecting the model and what sessions remember

Adding a provider makes its models available; selecting a model in Settings → Models makes it the default for new sessions. Two behaviors from the docs worth internalizing:

1. Existing sessions keep the model they were started with. Sessions log their original model, so changing the default mid-project doesn’t silently rewrite history or change what an in-flight session uses.
2. If you delete the provider that owns the current default, the composer blocks input until you pick a new model. The harness fails loud rather than guessing.

That session-pinning matters for reproducibility: when you’re comparing dsh against other harnesses (we did exactly that in DeepSeek Harness vs Claude Code), you can trust that a session’s transcript reflects one model, not a mid-run swap.

Troubleshooting the usual failures

Wrong or unreachable baseURL. The most common failure is the least exotic. Confirm the URL ends where the protocol expects (usually /v1 for OpenAI-compatible endpoints, /compatible-mode/v1 for DashScope) and that a plain GET {baseURL}/models succeeds outside the harness. This is the checkpoint where Download Apidog pays for itself in five minutes: send the request with the same header (Authorization: Bearer $KEY) the harness will send, and read the actual status code and body instead of a wrapped harness error. If you’re developing offline or the vendor is flaky, mock the provider’s /models and /chat/completions responses in Apidog and point baseURL at the mock while you build.

Missing or empty env var. apiKeyEnv names a variable; it doesn’t create one. If the variable isn’t set in the environment dsh actually runs in, requests go out unauthenticated and come back 401. Remember that a process launched from a GUI or a service manager may not inherit your shell profile. echo $GATEWAY_API_KEY in the same context that launches dsh web, not just in a random terminal.

Input modality mismatch. You attach an image, and the model never sees it, or the request errors. Custom models are text-only by default. Add input: [text, image] on the model entry, or set defaultInput at the route level if every model on the provider handles images.

Protocol quirks. Errors mentioning an unsupported role or a rejected token parameter point at compat switches: supportsDeveloperRole: false and maxTokensField: max_tokens are the two documented ones.

Everything worked yesterday. Developer preview. Pin the version you deploy, read the release notes before upgrading, and expect the settings schema to move. The deepseek-harness repo is the source of truth, not any blog post, this one included.

One more integration note: model providers are only half the customization story. The other half is what tools the agent can call, and you can wire your API workflows in directly; we cover that in using Apidog CLI inside DeepSeek Harness.

FAQ

Does DeepSeek Harness support Ollama officially?

The official providers doc doesn’t mention Ollama by name. What it supports is any endpoint speaking the openai-completions protocol, and Ollama documents an OpenAI-compatible API at http://localhost:11434/v1. The recipe above combines the two documented halves; test it on your install, since dsh is a developer preview and schemas can shift between releases.

Where does dsh store my API keys?

In $DSH_HOME/.credentials.yaml, write-only. The UI shows a redacted descriptor after saving, and settings.yaml holds only references like apiKeyEnv names. You never end up with a plaintext key inside your provider config.

Can I run different models for different sessions?

Yes. Selecting a model sets the default for new sessions only; every existing session keeps the model it started with. So you can run a cheap model like DeepSeek V4-Flash for routine sessions, switch the default to a heavier model for a hard problem, and your earlier sessions stay untouched.

My custom endpoint returns errors that the same request doesn’t produce in curl. What now?

Compare the exact payloads. The harness may send a developer role or a newer token-cap field your backend doesn’t accept; the documented fixes are supportsDeveloperRole: false and maxTokensField: max_tokens under compat. Replaying the harness-shaped request in an API client shows you which field the backend chokes on.