Platform

Open-source building blocks

Modular open-source libraries that compose into the infrastructure layer around the LLM. Each independently versioned and licensed under Apache 2.0 on GitHub.

appinfra

Reliable infrastructure: config, logging, scheduling, lifecycle.

llm-saia

Clean LLM communication: structured requests, typed responses, guards.

llm-infer

Inference across backends: same code across providers.

llm-kelt

Feedback collection, context management, fine-tuning.

llm-gent

Trait-based agents: explicit capabilities and configuration.

To GitHub →

Example

An appinfra CLI creates a SAIA object, connects to an Anthropic backend via llm-infer, and sends one request.

import sys, asyncio
from appinfra.app import AppBuilder
from llm_infer.client import Factory, SAIAAdapter
from llm_saia import SAIA

app = AppBuilder("hello-llm").build()

@app.tool(name="run")
@app.argument("prompt")
def run(self):
    asyncio.run(_send(self.lg, self.args.prompt))

async def _send(lg, prompt):
    async with Factory(lg).anthropic() as client:
        saia = SAIA.builder().backend(SAIAAdapter(client)).build()
        print((await saia.instruct(prompt)).value)

if __name__ == "__main__":
    sys.exit(app.main())

Usage:

$ pip install appinfra llm-saia "llm-infer[anthropic,saia]"
$ export ANTHROPIC_API_KEY=sk-ant-...
$ python app.py run "What is systems engineering?"