Skip to main content

Feature matrix

The poot binary and poot-serve select a GPU runtime with --backend. Three production backends run the same graph IR today: wgpu (Vulkan/SPIR-V, default), rocm (AMD raw HSA), and ptx (NVIDIA). They are not at parity. A fourth backend, raw Vulkan, lowers to the same SPIR-V as wgpu but runs on its own runtime and plans against the limits its own device reports, so it is a column in the tables below and has its own section. This page answers "does X work on backend Y?". The model and weight-format tables are generated from what the planner says; the other tables are written by hand.

The AMD XDNA2 and Intel NPU4 NPUs are not columns here. What each NPU runs is on NPU offload; the XDNA2 table below is generated from what the planner says about each op.

For lower-level kernel/dispatch capability (elementwise, GEMV, flash attention, capture/replay, ...), see the Backends architecture page, this page is the serving-level view: what poot-serve and the poot-llm library provide, with the paths no device lane runs marked as such.

Legend: colored badges use the same vocabulary on this page and on Backends: capability comparison. Yes = the indicated path has test coverage. Partial = works, with scope limits noted. Plans = the planner compiled the graph for that backend. Refused = the planner refused the graph with a typed error; the cell carries the message. Required = the backend's device lane requires cases that run the path, with their count (a requirement of the lane; its run on the device turns it into a result); No case = the lane requires none, which says nothing about whether the planner accepts it. No = not implemented for that backend. N/A = the concept does not apply to that backend.

poot-serve serves generation (/v1/completions, /v1/chat/completions), embeddings, reranking and LoRA administration from one engine thread over the driver on wgpu and ROCm. PTX and raw Vulkan serve generation too. See Serving.

Models and weight formats​

Whether a model runs on a backend is decided by planning: a model runs when compiling its graph for that backend succeeds, and a refusal is a typed error naming the op or constant and the capability or limit the backend lacks. So these tables are not written by hand. A test compiles the fixture graphs of every registered model family, the graphs of reference models at their real dimensions, and a dense model whose projections are stored in every weight format, for each backend, and writes the result between the markers below. The same region lists the library and server features the device lanes run, from the lanes' required-case lists.

This region is generated by just feature-matrix-regen from what compile says about each graph and from the required cases of the device lanes, never edited by hand. See Reading the matrix.

Model families​

Every registered family's fixture graphs (decode and prefill for a decoder, one forward for an encoder) compiled for each target at the fixture's small dimensions. A family joins this table by having a row in the registry's SHIPPED slice and a checkpoint in the fixture table.

FamilywgpuVulkanROCmPTX
bertPlansPlansPlansPlans
bert/cross_encoderPlansPlansPlansPlans
bloomPlansPlansPlansPlans
deepseek2PlansPlansPlansPlans
deepseek3PlansPlansPlansPlans
deepseek32PlansPlansPlansPlans
deepseek4PlansPlansPlansPlans
deepseek4/finetunePlansPlansPlansPlans
gemma2PlansPlansPlansPlans
gemma3PlansPlansPlansPlans
gemma4PlansPlansPlansPlans
gemma4/moePlansPlansPlansPlans
glm53_flashPlansPlansPlansPlans
glm53_flash/finetunePlansPlansPlansPlans
gpt_ossPlansPlansPlansPlans
granitePlansPlansPlansPlans
granite/granitemoePlansPlansPlansPlans
idefics3PlansPlansPlansPlans
idefics3/visionPlansPlansPlansPlans
llamaPlansPlansPlansPlans
llama/mixtralPlansPlansPlansPlans
minimax_m2PlansPlansPlansPlans
minimax_m2/finetunePlansPlansPlansPlans
mptPlansPlansPlansPlans
nemotron_hPlansPlansPlansPlans
olmo2PlansPlansPlansPlans
olmoePlansPlansPlansPlans
phi3PlansPlansPlansPlans
qwen2PlansPlansPlansPlans
qwen2_5_vlPlansPlansPlansPlans
qwen2_5_vl/visionPlansPlansPlansPlans
qwen3PlansPlansPlansPlans
qwen35PlansPlansPlansPlans
qwen35/finetunePlansPlansPlansPlans
qwen35/fp8PlansPlansPlansPlans
qwen35/textPlansPlansPlansPlans
qwen3moePlansPlansPlansPlans
qwen3nextPlansPlansPlansPlans
qwen4expPlansPlansPlansPlans
qwen4exp/finetunePlansPlansPlansPlans
qwen4exp/packedPlansPlansPlansPlans
smollm3PlansPlansPlansPlans

Real dimensions​

The graphs of reference models at their real dimensions, from the model's config.json or published sizes (decode and prefill over the model's whole context, and prefill into a full-context cache), compiled for each target's preset device limits. The weights are zeros: the planner reads shapes, types and limits, not values.

ModelwgpuVulkanROCmPTX
Qwen2.5-0.5BPlansPlansPlansPlans
Qwen3-0.6BPlansPlansPlansPlans
Qwen3.8-27BPlansPlansPlansPlans
Mixtral-8x7BPlansPlansPlansPlans
Mixtral-8x22BRefused decode: constant w.l0.moe.stack.gate a buffer of 1610612736 bytes (the device's single-buffer limit is 1342177280)Refused decode: constant w.l0.moe.stack.gate a buffer of 1610612736 bytes (the device's single-buffer limit is 1342177280)PlansPlans

Refusals​

One refusal on every backend, from a real model against a device limit it exceeds: the preset limits with one value lowered, as the row says. The cell is the planner's own message.

Model and limitwgpuVulkanROCmPTX
Mixtral-8x22B on a device whose largest buffer is the wgpu preset'sRefused decode: constant w.l0.moe.stack.gate a buffer of 1610612736 bytes (the device's single-buffer limit is 1342177280)Refused decode: constant w.l0.moe.stack.gate a buffer of 1610612736 bytes (the device's single-buffer limit is 1342177280)Refused decode: constant w.l0.moe.stack.gate a buffer of 1610612736 bytes (the device's single-buffer limit is 1342177280)Refused decode: constant w.l0.moe.stack.gate a buffer of 1610612736 bytes (the device's single-buffer limit is 1342177280)

Weight formats​

A one-layer dense decoder whose projections are stored in each format (a random admitted payload, bound as the stored bytes), compiled for each target.

FormatwgpuVulkanROCmPTX
AWQ int4PlansPlansPlansPlans
BF16 (dense)PlansPlansPlansPlans
E2M1 (FP4), per-32 E8M0 scalePlansPlansPlansPlans
FP8 E4M3, 128x128 block BF16 scalePlansPlansPlansPlans
FP8 E4M3, 128x128 block F32 scalePlansPlansPlansPlans
FP8 E4M3, per-channel BF16 scalePlansPlansPlansPlans
FP8 E4M3, per-channel F32 scalePlansPlansPlansPlans
GGUF IQ2_XSPlansPlansPlansPlans
GGUF IQ3_SPlansPlansPlansPlans
GGUF IQ3_XXSPlansPlansPlansPlans
GGUF IQ4_NLPlansPlansPlansPlans
GGUF IQ4_XSPlansPlansPlansPlans
GGUF MXFP4PlansPlansPlansPlans
GGUF Q2_KPlansPlansPlansPlans
GGUF Q3_KPlansPlansPlansPlans
GGUF Q4_0PlansPlansPlansPlans
GGUF Q4_1PlansPlansPlansPlans
GGUF Q4_KPlansPlansPlansPlans
GGUF Q5_0PlansPlansPlansPlans
GGUF Q5_1PlansPlansPlansPlans
GGUF Q5_KPlansPlansPlansPlans
GGUF Q6_KPlansPlansPlansPlans
GGUF Q8_0PlansPlansPlansPlans
GPTQ int4PlansPlansPlansPlans
GPTQ int4, act-orderPlansPlansPlansPlans

Device lane cases​

Library and server features the planner cannot decide, listed from the cases each device lane requires to run and pass (scripts/required-device-cases/). Required carries the number of cases that exercise the feature on that backend; it is a requirement of the lane, and the lane's run on the device is what turns it into a result. No case says the lane requires none, which is not a refusal.

FeaturewgpuVulkanROCmPTX
Served models and server startup (poot-serve)Required 51 casesRequired 2 casesRequired 49 casesRequired 43 cases
Paged KV cacheRequired 9 casesRequired 1 caseRequired 9 casesRequired 9 cases
Batched decodeRequired 5 casesRequired 1 caseRequired 7 casesRequired 5 cases
Prefix caching (reuse across requests)Required 5 casesNo caseRequired 5 casesRequired 4 cases
Speculative decoding: prompt lookup and draft modelRequired 24 casesNo caseRequired 24 casesRequired 24 cases
Speculative decoding: the model's own MTP layerRequired 9 casesNo caseRequired 9 casesRequired 9 cases
Sampled decodingRequired 11 casesNo caseRequired 11 casesRequired 10 cases
LoRA adaptersRequired 1 caseNo caseRequired 1 caseRequired 1 case
Guided decoding (regex, JSON schema, grammar)Required 3 casesNo caseRequired 3 casesRequired 2 cases
Embeddings and prompt embedsRequired 8 casesNo caseRequired 6 casesRequired 6 cases
Planning against the attached device's own limitsRequired 1 caseRequired 1 caseRequired 1 caseRequired 1 case

Measured​

The latest benchmark row per family and backend, from the stored results. A refusal carries the reason the tool reported.

FamilyBackendResultRun
qwen2rocmMeasured2026-10-21-amd-radeon-8060s-graphics-qwen2-5-0-5b-rocm-003809-r5
qwen2wgpuMeasured2026-10-21-amd-radeon-8060s-graphics-qwen2-5-0-5b-wgpu-003736-r5

XDNA2 NPU​

The AMD XDNA2 NPU is not a column above because no model compiles for it. This table compiles one F32 pointwise equation per row for the XDNA2 target with the limits measured on the hardware. NPU offload states what is verified on the hardware and each op's numerical contract.

OpXDNA2
addPlans
subPlans
mulPlans
divPlans
maxPlans
gePlans
negPlans
recipPlans
expPlans
logPlans
sqrtPlans
tanhPlans
erfPlans

Reading the matrix​

  • Plans means every graph of the row compiled for that backend: decode and prefill for a decoder, the one forward for an encoder. It says the planner can lower the graph. It does not say a real checkpoint of that family has run there; the Measured table lists the stored benchmark rows that did. A fixture row is the family's graphs at a few dozen channels; a Real dimensions row is the graphs a real checkpoint traces (its width, layer count and full context), over zero weights, so a limit that only a real width reaches shows there. A model that plans despite a weight over a backend's single-buffer limit (Qwen3.8-27B's 2.5 GB embedding table on wgpu) plans because the planner splits that weight into row chunks.
  • Refused carries the shape being compiled, the op or constant, and the capability or limit the backend lacks, exactly as the planner reports it. A row is a family key, or family/variant for one of the family's option fixtures (a Mixtral or Granite MoE checkpoint is the llama or granite family with experts). The Refusals table gives every backend one refusal: a real model against a device limit it exceeds, with the limit named in the row. Mixtral-8x22B is the real-dimension refusal: its stacked expert table is larger than the wgpu preset's single-buffer limit, so wgpu and raw Vulkan (which share that RADV limit) refuse it on their own caps; ROCm and PTX plan it and refuse it only when given that limit. Qwen3.8-27B plans on every backend because the planner splits its large weights.
  • Rejected is a compile failure that is not a missing capability, for example a graph over the compile limits.
  • Required and No case come from the required-case lists of the wgpu, raw Vulkan, ROCm and PTX device lanes, not from the planner. A feature is No case on a backend when its lane requires no case that exercises it; that is a gap in the lane, not a refusal.
  • The weight-format table binds a random admitted payload of each format as the stored bytes; it checks the lowering, not the numbers. A GGUF Mixtral names the llama architecture and loads as the llama family with experts when its metadata has llama.expert_count; a DeepSeek-V3 GGUF names deepseek2.
  • MXFP4 weights stay packed at load time. The routed-expert projection is one indexed packed contraction that reads only the selected experts' blocks; no expert table is materialized. Dense BF16 and F16 weights are kept as stored words with no host f32 copy. Packed checkpoint size is not a bound on total peak memory.
  • Real checkpoints: Qwen3.8-27B-FP8 ran on PTX (one H100 80 GB) and generates "Paris" for "The capital of France is", with the first token's top-five log-probabilities equal to the CPU oracle's within 1e-3. On ROCm (gfx1151) it also generates "Paris" with coherent text (about 5 decode tok/s, 12 prefill tok/s); its top-five comparison against the CPU oracle has not been run yet. It has not run on wgpu. DeepSeek-V4, MiniMax-M2, GLM-5.3-Flash and Qwen3.8-Flash-Next have only synthetic fixtures that match an independent f64 reference; no real public checkpoint of them has run.
  • The bert family runs on the driver's encode step (poot_llm::encoders). poot-serve runs encode steps for a decoder's embeddings and reranking on wgpu and ROCm, and starts on a BERT checkpoint (embeddings and cross-encoder rerank); the encode shapes are fitted and prepared before it serves.
  • Per-channel E4M3 compressed-tensors projections load packed and use the shared generated lowering. See Using quantized checkpoints.

Load limitations by family​

The tables above say which graphs plan. These are the checkpoints and configs a family's loader refuses, each with a typed error naming the field, tensor or type.

  • DeepSeek-V4 is loaded from a Hugging Face checkpoint only; a GGUF is refused as unsupported. A real quantized checkpoint is refused at load: its E4M3 projections carry .scale siblings, no loader builds the packed layout from them, and an F8_E4M3 weight without a .weight_scale or .weight_scale_inv sibling is a layout error rather than being read as unscaled dense weights. The packed layouts exist only in the family's test fixtures. A rope_scaling entry other than plain rotary is refused, as is a scoring_func other than sqrtsoftplus, equal compression rates for the two compressed attention kinds, and a compress_ratios entry that is neither 0 nor one of the two rates.
  • GLM-5.3-Flash is loaded from a Hugging Face checkpoint only; a GGUF is refused as unsupported. The loader refuses a hidden_act other than silu, a scoring_func other than sigmoid, a topk_method other than noaux_tc, attention bias, and the mla_use_nope, index_kpool_compress and index_kpool_always_select_tail flags when they are explicitly set the other way; an absent key takes the supported value. A layer is a linear_attention or deepseek_sparse_attention mixer with a dense or sparse feed-forward; any other layer_types or mlp_layer_types entry is refused. A sparse-attention layer with a nonzero qk_rope_head_dim is refused, and so is any indexer_types entry other than full.
  • Any GGUF is refused at load, by tensor name and type, when a quantized tensor has a ggml type with no packed format: Q8_1, Q8_K, IQ2_XXS, IQ2_S, IQ1_S, IQ1_M, TQ1_0 and TQ2_0. It is never loaded as dense weights silently. This holds for Qwen3.8-Flash-Next and every other GGUF family.

Expert pooling​

Expert pooling keeps a fixed number of resident expert rows per MoE layer and makes the experts a step selects resident before they are read, least recently used rows making room. It is a serving option (--expert-pool ROWS, see Configuration) for a model with routed experts whose stacked tables the planner can page, with the pool smaller than the expert count and at least the experts one token selects. A prefill longer than the pool serves is split into smaller chunks by the driver; a decode step whose rows times experts per token exceed the pool is refused at prepare with a typed capacity error, never split. Without the option every expert's weights are resident. Pooled decode is verified token for token against the resident run on Mixtral's fixture on wgpu and ROCm; the PTX run is pending, and the matrix cells above describe the resident paths.

Serving features​

The server's and the library's features that the planner cannot decide (prefix caching, speculative decoding, sampling, LoRA, guided decoding, embeddings, paged KV, batched decode) are in the generated table Device lane cases. These are not lane cases:

  • HTTP server and metrics. poot-serve serves the OpenAI-compatible routes and /metrics/prometheus on wgpu, ROCm, PTX and raw Vulkan.
  • KV preemption and swap. Not needed: a request's blocks are reserved at open, and a request that finds no room waits.
  • Tensor parallelism (multi-GPU). PTX serves it on a two-GPU NVLink pair (four-GPU decode only). wgpu, ROCm and raw Vulkan do not support it.
  • On-device sampling. A poot-llm library capability (the driver), verified by test on real hardware for every backend: a sampling step is appended to the decode graph itself, and the host reads back only the chosen token id (and a non-finite flag and any top-n logprob alternatives), not the full logits, for every request: logit bias, penalties, logprobs and guided masks run in that step too. The driver applies it to every row it steps, including the rows poot-serve schedules.
  • LoRA scope. Adapters hot-load, select and unload through the driver on every backend. The adapter scope is dense qwen2/llama-shaped models and the seven attention and MLP projections (q_proj/k_proj/v_proj/o_proj/gate_proj/up_proj/down_proj). See Using LoRA adapters.
  • Serving on PTX. The scheduling loop and the driver are backend-neutral. A PTX count in the table is what the PTX lane requires, and the lane runs on a rented NVIDIA pod, so a count is a requirement until that run is recorded (the MTP rows, for one, are listed as required before their pod run). The lane requires served cases for the families counted in the first row. See Serving.

Precision and execution model​

wgpuVulkanROCmPTX
Compute dtypef32f32f32, opt-in bf16 tensor-core matmulf32, opt-in bf16 tensor-core matmul
Tensor-core / matrix-unit matmulNo1Yes (cooperative matrix; needs VK_KHR_cooperative_matrix, verified RDNA3.5)Yes (RDNA WMMA, verified gfx1151)Yes (NVIDIA WMMA)
Execution model per tokenCached command-buffer re-encode + submitCommand buffers recorded once, replayed with a submit and a fence waitRaw HSA AQL packet record/replayCUDA graph capture/replay
Megakernel (on-device token loop, no host round-trip)No (no grid-wide sync primitive)No (no grid-wide sync primitive)No (planned)2No (planned)2
Requires a vendor SDK / driver beyond VulkanNoNoYes (ROCm/HSA userspace)Yes (CUDA)
On by defaultYesNo (--backend vulkan)Yes (--backend rocm)No (--backend ptx)

Raw Vulkan​

Raw Vulkan (--backend vulkan) runs the same SPIR-V kernels as wgpu through ash, with no wgpu. It implements the same executor contract as the other backends: a program is recorded once as native Vulkan command buffers and replayed, so a step costs a submit and a fence wait. It is lower priority than PTX and ROCm and is verified on one device family.

ItemStatus
Executor parity tableYes (the parity table and the backend-neutral coverage fixtures)
Dense decoder, greedy decodeYes (the registry's qwen2 fixture generates the same tokens as wgpu)
Tensor-core matmul (cooperative matrix)Yes (needs VK_KHR_cooperative_matrix and subgroup-size control; verified RDNA3.5)
Device-time profilingNo (no typed timing mode constructs a raw Vulkan device yet)
Multi-GPU / collectivesNo
ServingYes (greedy dense decoder verified; a driver without VK_EXT_memory_budget is refused at startup)
Hardware verifiedAMD Strix Halo iGPU under RADV only. NVIDIA and Intel Vulkan drivers are untested.

The device reports its own limits (largest buffer, grid, workgroup memory, subgroup sizes and matrix hardware) from the physical device, and the planner refuses what they rule out with a typed error instead of failing in the driver.

How to read this if you are choosing hardware​

  • NVIDIA GPU: PTX has the most mature single-GPU library path. poot-serve serves the dense fixture on PTX; see the serving table above for what has a served test.
  • AMD GPU: ROCm is the primary backend on AMD hardware for single-sequence library decode. wgpu also runs on the same AMD GPU as a fully portable fallback.
  • Anything else (Intel iGPU/dGPU, other Vulkan-capable hardware): wgpu is the supported choice. Raw Vulkan runs there in principle but is verified on AMD (RADV) only. Pick either through the poot-llm API or the poot-serve --backend option.
  • NPU: experimental, no model runs on one yet. See Architecture: NPU Offload.

See Picking a backend for the decision walkthrough.

Footnotes​

  1. A SPIR-V cooperative-matrix matmul exists and is hardware-verified on RDNA3.5 (f16 operands, f32 output, 16-aligned M/N/K, any batch; the accumulator is carried across K blocks), but no model's matmul feeds it yet, so it is not on the hot path for any model today. ↩

  2. Not implemented. The strategy is planned as a recorder behind the one compiled program on PTX and ROCm; see The Megakernel. ↩ ↩2