Documentation

poot traces decoder models into a backend-neutral primitive graph, fuses it into GPU kernels, and runs inference with async capture/replay.

Pick a path

• Develop: build the workspace, run models as a library, author #[kernel] functions, embed crates.
• Serve: OpenAI-compatible HTTP server. It starts, bounds requests and loads any registered model and serves completions, chat, embeddings, rerank and LoRA administration from one engine thread.
• Architecture: graph IR, fusion, backends, capture/replay, serving design.

Develop

• Build and test
• Compute engine
• Authoring kernels
• Run a model
• Embedding poot (LLM)
• Supported models

Serve

• Serving your first model
• Picking a backend
• OpenAI-compatible API
• Configuration reference
• Continuous batching
• Speculative decoding
• LoRA adapters
• Quantized checkpoints
• Guided decoding
• Embeddings and reranking
• Multimodal
• Tensor parallelism

Architecture

• Architecture overview
• Graph IR
• Fusion and planning
• Backends
• Tracing
• Kernel import
• Execution
• Megakernel
• Attention
• Serving design
• Implementation
• NPU offload
• Glossary

Reference

• Feature matrix
• Performance
• FAQ

Backends and devices: see Develop: Backends.