Description
Using an LLM for classification, routing or moderation means writing a prompt, begging for JSON, then writing the parser and the retry — and still guarding against a label it invented on the spot. Laya skips text generation entirely: hand it a state and a few typed questions and one forward pass returns the answers, a probability per option and a confidence for the question. Nothing is generated, so there is nothing to parse and nothing to hallucinate.
There are three question types. choice picks one of the options you supplied, score places the input on a rubric you defined, and noul answers how likely a statement is to be true. Where the threshold sits, and below what confidence a human takes over, is a line in your own code — not something the model decides. It is an Apache 2.0 Python package; pip install laya and both the weights and the data stay on your machine.
One forward pass per answer: a non-autoregressive encoder architecture answers a single question in roughly 33 ms on a T4, or 7.2 ms per question when batched. Those are local GPU numbers — on CPU the same call takes 193 to 464 ms, which is worth knowing before you pick hardware.
Three checkpoints to choose from: laya is ModernBERT-large, 421M parameters and a 512-token context, English only; laya-multilingual is mmBERT-base, 322M parameters and 1024 tokens, covering 100+ languages at twice the speed; laya-typed-decisions targets structured multi-question decision workflows.
Routing by script, before inference: the Router spends under 0.5 ms of pure Python identifying the writing system, then picks the checkpoint. That step is not optional — the English checkpoint scores 0.000 accuracy on Khmer while reporting 0.952 confidence, so the model will not warn you that it cannot read the input.
Preloading decides your latency: Router(preload=True) loads all three checkpoints into memory once and every call after that costs 32.8 ms. Lazy loading instead pays 7 to 10 seconds each time the language changes.
Self-hosted, priced per machine not per token: Apache 2.0 with open weights on Hugging Face. Run it on your own hardware and volume costs nothing extra, with no request ever leaving the network.
The author documents the weak spots too: on high-cardinality label spaces past 20 options it trails hosted decision APIs clearly (0.425 against 0.870 on Banking77); ordinal score is the weakest of the three types; and the base checkpoints sit near chance zero-shot — in the author's own words, a fast base to specialise, not a zero-shot decision engine.
There are three question types. choice picks one of the options you supplied, score places the input on a rubric you defined, and noul answers how likely a statement is to be true. Where the threshold sits, and below what confidence a human takes over, is a line in your own code — not something the model decides. It is an Apache 2.0 Python package; pip install laya and both the weights and the data stay on your machine.
Features
One forward pass per answer: a non-autoregressive encoder architecture answers a single question in roughly 33 ms on a T4, or 7.2 ms per question when batched. Those are local GPU numbers — on CPU the same call takes 193 to 464 ms, which is worth knowing before you pick hardware.
Three checkpoints to choose from: laya is ModernBERT-large, 421M parameters and a 512-token context, English only; laya-multilingual is mmBERT-base, 322M parameters and 1024 tokens, covering 100+ languages at twice the speed; laya-typed-decisions targets structured multi-question decision workflows.
Routing by script, before inference: the Router spends under 0.5 ms of pure Python identifying the writing system, then picks the checkpoint. That step is not optional — the English checkpoint scores 0.000 accuracy on Khmer while reporting 0.952 confidence, so the model will not warn you that it cannot read the input.
Preloading decides your latency: Router(preload=True) loads all three checkpoints into memory once and every call after that costs 32.8 ms. Lazy loading instead pays 7 to 10 seconds each time the language changes.
Self-hosted, priced per machine not per token: Apache 2.0 with open weights on Hugging Face. Run it on your own hardware and volume costs nothing extra, with no request ever leaving the network.
The author documents the weak spots too: on high-cardinality label spaces past 20 options it trails hosted decision APIs clearly (0.425 against 0.870 on Banking77); ordinal score is the weakest of the three types; and the base checkpoints sit near chance zero-shot — in the author's own words, a fast base to specialise, not a zero-shot decision engine.
