Skip to content

Using llm with OpenCode Zen and GWDG Models🔗

llm is a CLI tool by Simon Willison for interacting with large language models from the terminal. Since both OpenCode Zen and GWDG's SAIA expose OpenAI-compatible APIs, llm can talk to them with a small YAML configuration.

Installing llm🔗

Pick one of the following methods:

$ pip install llm
$ brew install llm
$ uv tool install llm

Connecting to OpenCode Zen🔗

OpenCode Zen is an AI gateway that provides access to a curated list of tested models through a single API key. Get an API key at opencode.ai/zen/settings/api-keys.

Store the key:

$ llm keys set opencode-zen

Find the configuration directory and create or edit extra-openai-models.yaml:

$ dirname "$(llm logs path)"

Add model entries to extra-openai-models.yaml:

- model_id: opencode-zen-claude-sonnet-4
  model_name: claude-sonnet-4
  api_base: "https://opencode.ai/zen/v1"
  api_key_name: opencode-zen

- model_id: opencode-zen-gpt-5
  model_name: gpt-5
  api_base: "https://opencode.ai/zen/v1"
  api_key_name: opencode-zen

- model_id: opencode-zen-deepseek-v4-flash-free
  model_name: deepseek-v4-flash-free
  api_base: "https://opencode.ai/zen/v1"
  api_key_name: opencode-zen

Run a prompt:

$ llm -m opencode-zen-claude-sonnet-4 'Explain SAIA in one paragraph'

Connecting to GWDG Models🔗

GWDG SAIA hosts open-weight models on GWDG infrastructure with an OpenAI-compatible API. An Academic Cloud account and a SAIA API key are required.

Store the key:

$ llm keys set gwdg

Add model entries to the same extra-openai-models.yaml:

- model_id: gwdg-llama-3.1-8b
  model_name: meta-llama-3.1-8b-instruct
  api_base: "https://chat-ai.academiccloud.de/v1"
  api_key_name: gwdg

- model_id: gwdg-qwen3.6-27b
  model_name: qwen3.6-27b
  api_base: "https://chat-ai.academiccloud.de/v1"
  api_key_name: gwdg

- model_id: gwdg-deepseek-v4-flash
  model_name: deepseek-v4-flash
  api_base: "https://chat-ai.academiccloud.de/v1"
  api_key_name: gwdg

Run a prompt:

$ llm -m gwdg-qwen3.6-27b 'What is the capital of Lower Saxony?'

The full list of available GWDG models can be retrieved with:

$ curl -s -H "Authorization: Bearer $(llm keys get gwdg)" \
    https://chat-ai.academiccloud.de/v1/models | python -m json.tool

Useful Commands🔗

$ llm models                          # list all configured models
$ llm models default gwdg-llama-3.1-8b  # set a default model
$ llm logs -n 5                       # show last 5 prompts/responses
$ llm logs -n 1 --json                # last response as JSON