Description
You watch an Excel tutorial, the captions hear "some" while the screen shows =SUM(B3:B10). An AI that learns from the transcript alone learns that mistake: commands misheard, steps paraphrased past recognition. video-to-skill reads the screen, not just the narration; every step cites the frame it came from, and on-screen text outranks the voice-over.
Give it a YouTube URL or a local file; it extracts the transcript and keyframes on your machine and compiles an installable agent skill, then a fresh sub-agent that sees only the package, never the video, executes each step in a sandbox before the verified badge is written. One command installs it, with no account or API key, under the MIT license.
Commands read off the pixels: Formulas, commands and extension ids come from the frame, each step carries a [t=MM:SS] timestamp and the frame, so a misheard narration does not corrupt it
Verification by execution: A sub-agent runs each step in a sandbox with no network, writes confined to a throwaway workspace and credentials unreadable; failures get repaired and re-verified, and without a sandbox it refuses and labels honestly
Runs locally: A prebuilt Rust extractor bootstraps ffmpeg, yt-dlp and whisper weights, all checksum-pinned; whisper is Metal-accelerated on Apple Silicon
Installed for every agent: The skill goes to each detected agent: Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode and Amp, as one real copy plus relative symlinks, after printing the exact plan and waiting for consent
Skills that grow: Feed a second video of the same task and it folds in: agreement raises confidence, conflicts become cited variants, nothing is overwritten
No fabrication: Non-procedural videos are refused with an analysis report; anything not visually confirmed ships flagged low-confidence
Give it a YouTube URL or a local file; it extracts the transcript and keyframes on your machine and compiles an installable agent skill, then a fresh sub-agent that sees only the package, never the video, executes each step in a sandbox before the verified badge is written. One command installs it, with no account or API key, under the MIT license.
Features
Commands read off the pixels: Formulas, commands and extension ids come from the frame, each step carries a [t=MM:SS] timestamp and the frame, so a misheard narration does not corrupt it
Verification by execution: A sub-agent runs each step in a sandbox with no network, writes confined to a throwaway workspace and credentials unreadable; failures get repaired and re-verified, and without a sandbox it refuses and labels honestly
Runs locally: A prebuilt Rust extractor bootstraps ffmpeg, yt-dlp and whisper weights, all checksum-pinned; whisper is Metal-accelerated on Apple Silicon
Installed for every agent: The skill goes to each detected agent: Claude Code, Codex, Cursor, Gemini CLI, GitHub Copilot, OpenCode and Amp, as one real copy plus relative symlinks, after printing the exact plan and waiting for consent
Skills that grow: Feed a second video of the same task and it folds in: agreement raises confidence, conflicts become cited variants, nothing is overwritten
No fabrication: Non-procedural videos are refused with an analysis report; anything not visually confirmed ships flagged low-confidence
