Crates and implementation details
Which crate owns which layer, and the few constraints that span several pages.
Crate map
| Layer | Crate(s) | Role |
|---|---|---|
| Graph IR | poot-graph-ir | Primitive graph, Builder, ops compositions (attention, norms, MoE, ...), and the transform passes |
| CPU oracle | poot-eval | Reference evaluator; test-side correctness ground truth |
| Planning | poot-graph-plan | compile: passes, plans, storage, slots and numerics; typed refusals |
| Kernel import | poot-kernel-attr, pootc | #[kernel] attribute; rustc driver that imports kernels from Stable MIR |
| Kernel IR, synth | poot-kernel-ir, poot-kernelgen | Kernel body IR; synthesized bodies (elementwise, reduction, tiled GEMM, flash, dequant) |
| Codegen | poot-codegen | Kernel IR to LLVM IR to SPIR-V, PTX, AMDGCN code objects or AIE |
| Runtimes | poot-wgpu-runtime, poot-rocm-runtime, poot-ptx-runtime, poot-vulkan-runtime, poot-runtime-common | Devices, buffers, transfers; mechanism only |
| Executors | poot-wgpu-device (wgpu), poot-rocm-device, poot-ptx-device, poot-vulkan-device | Load, bind, capture, replay and read back a plan |
| NPU device | poot-npu | Experimental Devices for the XDNA2 and Intel NPU4 NPUs; see NPU Offload |
| Models | poot-models | The Model trait, the family registry, per-family tracers over shared components |
| Loading | poot-load, poot-quant, poot-weights | Safetensors and GGUF readers; packed quantization descriptors shared by loader, IR and planner; the weight roles, ids and map |
| Text | poot-text | Tokenizer, chat templates and guided-decoding constraints; needs no model and no device |
| Generation | poot-llm | ModelHandle and the driver: token loop, sampling suffix, KV slots |
| Serving | poot-serve | OpenAI-compatible HTTP server: one scheduling loop over the driver's step |
| Tooling | poot-orchestrator, poot-profile, poot-test-util | Pod runs, profiling, hardware smoke checks, test helpers |
Cross-cutting notes
- AIE compilation and NPU execution are separate stages. The compiler produces a typed AIE core object,
link recipe and linked artifact with a shape ABI for the planner's serialized schedule (the ordinary
kernel body with its thread grid turned into loops);
poot-npu'sDeviceloads and runs them through the shared engine. See NPU Offload. - The CPU oracle is test-side.
poot-evalis the reference that tests and device checks compare against; generation never calls it. See Execution. - Attention takes its own value width, split keys and paged keys. The fused attention op types a value head narrower than the key head, a key split into per-head and shared parts (MLA), and a key and value read in place from paged KV pools through a slot map. See Attention.
- ROCm AQL publication. Replay writes AQL packets to a host-coherent ring, then rings the doorbell. A discrete-GPU device-memory ring needs an explicit store fence first; integrated APUs refuse that constructor.
- Synthetic versus real checkpoints. Some families are verified only on bounded synthetic checkpoints against the CPU oracle. That is not a real-checkpoint claim; the feature matrix tracks the difference.