# LlamaIndex

[LlamaIndex](https://developers.llamaindex.ai) is the retrieval and document
workflow framework. Use its `OpenAILike` LLM, which is the OpenAI client with
the model allowlist removed so arbitrary model IDs work.

```bash
pip install llama-index llama-index-llms-openai-like
export HYPHEN_API_KEY="sk-..."
```

*Setup guide: this config comes from LlamaIndex's official documentation and
has not been run end to end against the gateway. Corrections to [support@hyphen-solution.com](mailto:support@hyphen-solution.com).*

## The model

```python
import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="minimax-m3",
    api_base="https://api.hyphen-solution.com/v1",
    api_key=os.environ["HYPHEN_API_KEY"],
    context_window=200000,
    max_tokens=2000,
    is_chat_model=True,
    is_function_calling_model=True,
)

print(str(llm.complete("Explain a Bloom filter in three sentences.")))
```

Note it is `api_base`, not `base_url`. LlamaIndex kept the older name.

The last three kwargs are not optional in practice:

- **`is_chat_model=True`** routes to `/v1/chat/completions`. Leave it off and
  LlamaIndex uses the legacy completions path, which the gateway does not serve.
- **`is_function_calling_model=True`** unlocks `FunctionAgent` and tool calling.
  Hyphen supports tools, so this is correct.
- **`context_window`** tells LlamaIndex how much it can pack into a prompt.
  200000 suits the standard M-series. Use 4000000 for `minimax-text-01` and
  66000 for `minimax-m2-her`.

## Chat and streaming

```python
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="system", content="You are terse."),
    ChatMessage(role="user", content="Define idempotent."),
]

print(str(llm.chat(messages)))

for chunk in llm.stream_chat(messages):
    print(chunk.delta, end="", flush=True)
```

## An agent with tools

`FunctionAgent` is async only, and it requires
`is_function_calling_model=True` on the LLM.

```python
import asyncio
import os
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="minimax-m3",
    api_base="https://api.hyphen-solution.com/v1",
    api_key=os.environ["HYPHEN_API_KEY"],
    context_window=200000,
    max_tokens=4000,
    is_chat_model=True,
    is_function_calling_model=True,
)


def multiply(a: float, b: float) -> float:
    """Useful for multiplying two numbers."""
    return a * b


agent = FunctionAgent(tools=[multiply], llm=llm)


async def main():
    print(str(await agent.run("What is 1234 * 4567?")))


asyncio.run(main())
```

Swap `FunctionAgent` for `ReActAgent` (same import path, same `.run()`) if you
would rather drive tools through prompting than native tool calls.

## Using it as the global default

```python
from llama_index.core import Settings

Settings.llm = llm
```

Every index, query engine and agent then uses Hyphen without being passed the
LLM explicitly.

## Embeddings are not on the gateway

This is the easy one to miss. A `VectorStoreIndex` needs an
embedding model, and the Hyphen catalog has none. Set `Settings.embed_model` to
a local model or another provider:

```python
# pip install llama-index-embeddings-huggingface
from llama_index.core import Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
Settings.llm = llm  # generation still goes to Hyphen
```

Retrieval runs locally, generation runs on Hyphen. That split is usually what
you want anyway, since embedding a corpus is cheap locally and repeated.

## Gotchas

- **`api_base`, not `base_url`.**
- **`max_tokens` of 2000 or more.** 4000 for agents. See
  [Choosing a model](/choosing-a-model#the-max_tokens-gotcha).
- **Long documents:** switch the model to `minimax-text-01` and
  `context_window` to 4000000. See [Long-context work](/recipes/long-context).
- **`429`** means the monthly budget is spent. See
  [Handling the 429 cap](/recipes/handling-429).

## Related

- [OpenAILike reference](https://developers.llamaindex.ai/python/framework-api-reference/llms/openai_like/)
- [LlamaIndex agents](https://developers.llamaindex.ai/python/framework/module_guides/deploying/agents/)
- [Long-context work](/recipes/long-context): feeding it a whole book.