Copy page

Build with Hyphen

LlamaIndex

LlamaIndex is the retrieval and document workflow framework. Use its OpenAILike LLM, which is the OpenAI client with the model allowlist removed so arbitrary model IDs work.

Code
pip install llama-index llama-index-llms-openai-like
export HYPHEN_API_KEY="sk-..."

Setup guide: this config comes from LlamaIndex's official documentation and has not been run end to end against the gateway. Corrections to support@hyphen-solution.com.

The model

Code
import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="minimax-m3",
    api_base="https://api.hyphen-solution.com/v1",
    api_key=os.environ["HYPHEN_API_KEY"],
    context_window=200000,
    max_tokens=2000,
    is_chat_model=True,
    is_function_calling_model=True,
)

print(str(llm.complete("Explain a Bloom filter in three sentences.")))

Note it is api_base, not base_url. LlamaIndex kept the older name.

The last three kwargs are not optional in practice:

• is_chat_model=True routes to /v1/chat/completions. Leave it off and LlamaIndex uses the legacy completions path, which the gateway does not serve.
• is_function_calling_model=True unlocks FunctionAgent and tool calling. Hyphen supports tools, so this is correct.
• context_window tells LlamaIndex how much it can pack into a prompt. 200000 suits the standard M-series. Use 4000000 for minimax-text-01 and 66000 for minimax-m2-her.

Chat and streaming

Code
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="system", content="You are terse."),
    ChatMessage(role="user", content="Define idempotent."),
]

print(str(llm.chat(messages)))

for chunk in llm.stream_chat(messages):
    print(chunk.delta, end="", flush=True)

An agent with tools

FunctionAgent is async only, and it requires is_function_calling_model=True on the LLM.

Code
import asyncio
import os
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="minimax-m3",
    api_base="https://api.hyphen-solution.com/v1",
    api_key=os.environ["HYPHEN_API_KEY"],
    context_window=200000,
    max_tokens=4000,
    is_chat_model=True,
    is_function_calling_model=True,
)


def multiply(a: float, b: float) -> float:
    """Useful for multiplying two numbers."""
    return a * b


agent = FunctionAgent(tools=[multiply], llm=llm)


async def main():
    print(str(await agent.run("What is 1234 * 4567?")))


asyncio.run(main())

Swap FunctionAgent for ReActAgent (same import path, same .run()) if you would rather drive tools through prompting than native tool calls.

Using it as the global default

Code
from llama_index.core import Settings

Settings.llm = llm

Every index, query engine and agent then uses Hyphen without being passed the LLM explicitly.

Embeddings are not on the gateway

This is the easy one to miss. A VectorStoreIndex needs an embedding model, and the Hyphen catalog has none. Set Settings.embed_model to a local model or another provider:

Code
# pip install llama-index-embeddings-huggingface
from llama_index.core import Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
Settings.llm = llm  # generation still goes to Hyphen

Retrieval runs locally, generation runs on Hyphen. That split is usually what you want anyway, since embedding a corpus is cheap locally and repeated.

Gotchas

• api_base, not base_url.
• max_tokens of 2000 or more. 4000 for agents. See Choosing a model.
• Long documents: switch the model to minimax-text-01 and context_window to 4000000. See Long-context work.
• 429 means the monthly budget is spent. See Handling the 429 cap.

Related

• OpenAILike reference
• LlamaIndex agents
• Long-context work: feeding it a whole book.