Description
The hard part of running vision AI on a phone is size and latency: one model to spot objects, another to read text, a third to cut out outlines, and none of them fit or respond in time. Mirage packs all of it into a model of about 22 MB, with 51.3 million parameters and dedicated 2-bit quantization, finishing a full analysis in 74 ms on a Samsung Galaxy S26 NPU.
Give it a photo and in one pass it describes the scene, outlines each object with a mask, reads signs and labels, works out the spatial relations between things, and makes a decision from options you supply, such as picking stop, yield or speed limit for a road sign. The result comes back as tidy structured data your app can use directly.
Scene descriptions: describes the setting, light and background in English, German, French, Spanish, Italian, Dutch and Polish.
Detection and segmentation: returns object names, boxes and scores, with a segmentation mask for each object.
Text recognition: reads the text in an image and marks where it sits.
Spatial relations: works out how objects relate, such as a sign attached to a pole.
Decisions from your labels: chooses from the labels you provide, and can be paired with a text model for deeper reasoning.
Runs locally: fits phones, wearables and robots, takes 107 ms on the S26 CPU, and the web demo runs entirely in your browser.
Getting started: after `pip install -U cactus-compute` one line of Python runs it, and the engine and weights download on first use. Apache-2.0 license.
Give it a photo and in one pass it describes the scene, outlines each object with a mask, reads signs and labels, works out the spatial relations between things, and makes a decision from options you supply, such as picking stop, yield or speed limit for a road sign. The result comes back as tidy structured data your app can use directly.
Features
Scene descriptions: describes the setting, light and background in English, German, French, Spanish, Italian, Dutch and Polish.
Detection and segmentation: returns object names, boxes and scores, with a segmentation mask for each object.
Text recognition: reads the text in an image and marks where it sits.
Spatial relations: works out how objects relate, such as a sign attached to a pole.
Decisions from your labels: chooses from the labels you provide, and can be paired with a text model for deeper reasoning.
Runs locally: fits phones, wearables and robots, takes 107 ms on the S26 CPU, and the web demo runs entirely in your browser.
Getting started: after `pip install -U cactus-compute` one line of Python runs it, and the engine and weights download on first use. Apache-2.0 license.
