# DSPy

[DSPy](https://dspy.ai) replaces hand-written prompts with declarative modules
and optimisers. Configure it once with `dspy.LM` and every module in your
program uses Hyphen.

```bash
pip install -U dspy
export HYPHEN_API_KEY="sk-..."
```

*Setup guide: this config comes from DSPy's official documentation and has
not been run end to end against the gateway. Corrections to [support@hyphen-solution.com](mailto:support@hyphen-solution.com).*

## The model

```python
import os
import dspy

lm = dspy.LM(
    "openai/minimax-m3",
    api_base="https://api.hyphen-solution.com/v1",
    api_key=os.environ["HYPHEN_API_KEY"],
    model_type="chat",
    max_tokens=4000,
)
dspy.configure(lm=lm)
```

Two naming details, both easy to get wrong:

- **The `openai/` prefix is required.** DSPy routes through LiteLLM, and the
  prefix is what tells it to treat this as a generic OpenAI-shaped endpoint.
  DSPy's own docs say to add it for any OpenAI-compatible provider. It is
  stripped before the request, so Hyphen receives `minimax-m3`.
- **It is `api_base`, not `base_url`.** That is LiteLLM's name. `api_base` and
  `api_key` are not named parameters on `dspy.LM`; they ride through `**kwargs`
  into LiteLLM, which is by design.

`model_type="chat"` is the default and can be omitted. Passing it explicitly is
cheap insurance.

## Predict and ChainOfThought

```python
classify = dspy.Predict("sentence -> sentiment: bool")
print(classify(sentence="it's a charming and often affecting journey.").sentiment)

qa = dspy.ChainOfThought("question -> answer")
result = qa(question="Why is a Bloom filter probabilistic?")
print(result.reasoning)
print(result.answer)
```

`ChainOfThought` injects a `reasoning` field before your declared outputs. Note
that this is DSPy's own prompted reasoning, which is separate from the
model's internal reasoning. Both consume tokens from the same `max_tokens`
budget, which is why 4000 is the floor here.

## A typed signature

```python
from typing import Literal
import dspy


class ClassifyIssue(dspy.Signature):
    """Classify a bug report by severity and component."""

    report: str = dspy.InputField()
    severity: Literal["low", "medium", "high", "critical"] = dspy.OutputField()
    component: str = dspy.OutputField()


classify = dspy.ChainOfThought(ClassifyIssue)
out = classify(report="The CSV exporter drops the last row on 2.4.1. Blocking reporting.")
print(out.severity, out.component)
```

DSPy builds structured output by prompting and parsing, not by
`response_format`, so it works on the M-series without any special
configuration.

## A module with tools

```python
def multiply(a: float, b: float) -> float:
    """Multiply two numbers."""
    return a * b


agent = dspy.ReAct("question -> answer", tools=[multiply], max_iters=6)
print(agent(question="What is 1234 * 4567?").answer)
```

Keep `max_iters` bounded. A ReAct loop that will not converge is the fastest
way to spend a monthly cap.

## Two models in one program

Use the cheap fast model for bulk classification and the flagship where quality
matters.

```python
fast = dspy.LM(
    "openai/minimax-m2.5",
    api_base="https://api.hyphen-solution.com/v1",
    api_key=os.environ["HYPHEN_API_KEY"],
    max_tokens=4000,
)

with dspy.context(lm=fast):
    result = classify(report="...")   # this call uses minimax-m2.5
```

## Optimisers and your budget

DSPy's optimisers (`MIPROv2`, `BootstrapFewShot` and friends) work by running
your program many times over a training set. That is dozens to hundreds of
requests per compile.

Do the optimisation runs on `minimax-m2.5`, check your remaining budget in the
[console](https://app.hyphen-solution.com) first, and keep the trainset small
while you are iterating. The hard cap means a runaway compile returns 429
rather than an invoice, but a spent cap still stops your day.

DSPy caches by default (`cache=True` on `dspy.LM`), so repeated identical calls
during development cost nothing.

## Gotchas

- **`openai/` prefix required.** Without it LiteLLM cannot route the call.
- **`api_base`, not `base_url`.**
- **`max_tokens` of 4000.** Prompted reasoning plus internal reasoning share
  the budget. See [Choosing a model](/choosing-a-model#the-max_tokens-gotcha).
- **No embedding models** on the gateway, so DSPy retrievers need an embedder
  from elsewhere.
- **`429`** means the monthly budget is spent. Optimisers reach it fast. See
  [Handling the 429 cap](/recipes/handling-429).

## Related

- [DSPy language models](https://dspy.ai/learn/programming/language_models/)
- [dspy.LM reference](https://dspy.ai/api/models/LM/)
- [Choosing a model](/choosing-a-model): which model for compile vs run.