Skip to content
VMAX

VMAX

Every cut reviewable. Every post accountable.

Created on 8th August 2026

•

VMAX

VMAX

Every cut reviewable. Every post accountable.

What is the problem your project solves?

A video edit is a document, not a pile of pixels. Every cut reviewable, every render reproducible, every post accountable.

Agentic video tools give you a different video every time you ask. Programmatic render APIs give you the same video every time but need hand-written JSON. Nobody bridges the two.

For an agency running video for multiple client brands, that gap is the whole job. A client approved a cut three weeks ago and wants one change — now you are re-exporting everything and hoping it looks the same. There is no diff, no approval trail, and no way to prove the thing you delivered is the thing they signed off on.

That is a version-control problem wearing an editing problem's clothes.

And it does not stop at the export. The real agency loop is brief → shoot → analyse → edit → approve → post → learn, and today every one of those handoffs is a human copying a file into another tool. The approval lives in WhatsApp. The reason for a cut lives in someone's head. The published post has no traceable link back to the version that was approved. When a client asks "why is this frame here," the honest answer is usually that nobody remembers.

The impact of fixing it:

  • Revisions stop being re-exports.
  • Approvals become auditable artifacts.
  • Publishing inherits the approval instead of re-deciding it.
  • "The AI changed something and nobody knows what" stops being an acceptable answer in a client relationship.

How you are solving it?

Natural language compiles to OTF — a versioned, diffable JSON timeline document. An agent proposes changes to that document. A human reads a diff and approves it. The document then renders reproducibly, and the approved version is the only thing that can be published.

Three properties everything else falls out of:

  • Deterministic — same document plus same pinned toolchain produces byte-identical video. Revisions are cheap because unchanged segments are cached.
  • Diffable — any two versions produce a readable change list. That is the approval trail, the audit log, and the answer to unexplained AI edits — one feature doing three jobs.
  • Auditable — every cut carries the reason it exists.

The one rule that makes it work

The agent never invents a timecode. Deterministic signal processing produces a ranked candidate set of legal cut points — beat, shot_cut, sentence_end, silence — and the agent selects an index.

This is not a stylistic choice. At hour scale every video LLM scores 0.003–0.115 mIoU on temporal grounding while a 2021 CLIP baseline beats all of them at 0.269 (arXiv:2606.12300). Asking a model to find the moment is asking it to do the thing it is measurably worst at.

So we do not ask the model to find the moment. We ask it to choose between moments signal processing already found.

One product, five surfaces

The pipeline is one document moving through five stages. Every stage reads and writes the same OTF document — which is why an approval at stage four can still be trusted at stage five.

#SurfaceWhat it does
1Ingest & analysisFrame-accurate extraction: shot boundaries, beat grid, VFR normalisation, transcript with word-level timing, per-frame retrieval. This is what produces the candidate sets.
2GenerationBrand-locked image and video generation. Prompt construction from a brand twin, routed to local open-weight models first.
3EditClaude proposes operations over MCP; the desktop app edits the same document directly. Both write OTF.
4ApproveVersion diff, reason-carrying operations, human gate. Nothing renders as a deliverable without passing it.
5PublishThe approved version hands off to OSPYR's social MCP — Meta, Instagram, YouTube — carrying its approval with it.

Surface 1 — Ingest & analysis ✅ WORKING

Video comes in; structure comes out. ffprobe for container truth, VFR normalisation to a rational frame clock, shot-boundary detection, beat detection, WhisperX for word-level transcript timing, and frame-wise extraction with retrieval so a specific frame can be fetched and reasoned about rather than guessed at.

Everything is emitted as candidate sets — scored, sourced, and ranked. They are the only legal source of a timecode anywhere in the system. This surface exists precisely so the no-invented-timecode rule can hold: analysis finds the moments, the model only chooses among them.

→ 186 tests.

Surface 2 — Generation 🔶 BUILT, NO LIVE VENDOR CALL

Two agency capabilities, ported in as product features rather than invoked as external skills:

  • Brand-locked image prompting. A brand twin — palette, lighting grammar, framing rules, product handling, the things a brand gets wrong when a generic model guesses — compiles into image prompts. The point is that the same brief produces on-brand output twice, which generic prompting does not.
  • Reel construction. Beat-mapped plate sequencing: shots resolved against a cut sequence, timed to a rational frame clock, assembled into an OTF document rather than an opaque render job.

Local models are the free tier. On a user's own GPU, generation has zero marginal cost to us — no per-second API bill, no vendor able to discontinue an API underneath us.

  • ✅ Wan 2.2 and FLUX.2-klein-4B — Apache-2.0 and clean.
  • ❌ LTX-2, HunyuanVideo, MiniMax H3 — refused in code: a non-compete aimed squarely at AI video editors, a territorial exclusion that follows the output, and a mandatory attribution that would make any white-label claim false.

→ 134 tests on the generation router. Every provider is tested against an injected transport — no adapter has spoken to a live vendor yet.

Surface 3 — Edit ✅ MCP VERIFIED · 🔶 DESKTOP PARTIAL

  • Over MCP. 27 tools, spec revision 2026-07-28, stateless. Claude reads a token-budgeted timeline projection, requests a candidate set, and proposes trims, cuts and captions as document operations. What comes back is a version bump plus a human-readable diff — never a surprise video. Published as npx vmax-mcp.
  • In the desktop app. A native Windows editor: timeline, trim tools, snapping, and a prompt bar that speaks to the same MCP surface — so an instruction typed in the app and an instruction given to Claude produce the same document operations. The macOS app is a separate native Swift stack, deliberately, because the feel is the product.

Surface 4 — Approve ✅ WORKING

Two versions in, a readable change list out. Every operation carries the reason it exists, which is what makes the diff an approval trail rather than a changelog.

The desktop cannot produce a deliverable, by construction. libx264 is GPL and shipping a desktop app is distribution, so the desktop build has no deliverable-grade encoder. That licence constraint and the determinism constraint point at the same architecture: previews are local and marked role=proxy, and anything carrying an approval renders in a pinned cloud container.

A proxy cannot be mistaken for a master because the type system will not allow it.

Surface 5 — Publish ⭕ DESIGNED, HANDOFF NOT WIRED

The last mile is where agency work actually leaks. VMAX's answer: publishing consumes an approved version — not a file someone dragged out of a folder.

OSPYR already runs a live social MCP: brand-connected Meta and Instagram accounts, YouTube, scheduling, caption drafting against brand voice rules, comment triage, and performance read-back. Clients connect their own accounts; posting happens under their credentials, not ours.

The handoff — approved OTF version → rendered master → scheduled post, with the approval ID travelling all the way to the published asset — is specified and not yet built.

It is the single highest-value remaining integration, because it closes the loop: performance data returns against the version that produced it, so "which cut performed better" becomes a query instead of an argument.

What actually works today

Everything below was run, not asserted.

~1,500 tests passing across three languages

TestsComponent
382Rust workspace: document, ops, compositor, FFI, blast radius, variants
217MCP agent surface
186Perception: ingest, VFR, candidates
185Multi-tenant API, RLS, cross-tenant gate
170Export: FCP7 XML, OTIO, SRT, EDL, AAF
155Render: OTF to MP4
145Component library
134Generation router
129Editor UI
127Voice
114Provenance, C2PA, labelling

It renders real work. Imported a real 32-clip FCPXML from a production launch reel, resolved 44 plates, built an OTF document, validated it, and compiled it to video — 208 frames at exactly 8.6667s, frame-exact at 24fps, every in/out point preserved as rational time with no float rounding.

A real MCP client drove a real edit. 30/30 checks: read a projection, minted a candidate set, applied a trim, got back a diff and a version bump, retried the same idempotency key and got replayed: true with the document unchanged.

It is installable. npx vmax-mcp --seed-demo boots the server, seeds a 30-second reel fixture, and answers initialize with 27 tools. Verified from the published package, not from source.

One command proves the pipeline. python e2e/run.py — 10 stages green across Rust, Python and TypeScript, including a conformance check that the content hash agrees across all three.

What does NOT work yet

Stated plainly, because a submission that hides this is worth less than one that does not.

  • No video preview in the desktop editor. The wgpu NV12 path is proven on real hardware with zero channel error, but the Media Foundation import into it is unfinished. Timeline, trim tools and snapping work; the viewer is empty.
  • The macOS app has never compiled. ~7,150 lines of Swift written on a Windows machine; CI on a real Mac produced 127 errors, 104 of them one pointer-type mismatch.
  • The compositor is not wired to the renderer. Masks, transitions and motion graphics exist as a tested crate the render path does not consume yet. The reel above therefore has correct cuts and no mask reveal.
  • No adapter has spoken to a live model vendor. Every provider is tested against an injected transport.
  • The publish handoff is not built. The social MCP is live and the approval model is live; the bridge between them is specified only.

Notable engineering decisions

  • Rational time everywhere. {v, r} integers, never a float. A float in a time field is rejected at parse with an error explaining why. 100 frames at 29.97fps accumulate zero drift.
  • Position-independent cache keys. A chunk is keyed on its own content, never its timeline position — so a shifted-but-unchanged chunk is a cache hit, and the most common edit in existence stops being the most expensive one.
  • Undo publishes where it stops working. A restore returns an enumerated list of what it did not cover — already-rendered outputs, consumed stock licences, generation spend, anything published. An undo you cannot fully trust is worse than no undo unless you say exactly where it stops.

Challenges we ran into

The spec had a bug that would have killed the core value proposition. The design keyed the render cache on the whole-document hash, so any edit invalidated every chunk — segment caching, described in our own architecture as the single biggest margin lever in the system, would have missed on every ripple edit. Fixed by keying on the chunk's own content, position-independent. There is now a test proving the naive version reports 20 of 20 chunks dirty where the correct one reports 1.

Six independent implementations of timeline arithmetic appeared. Rust, Python and TypeScript services each needed the document model before the language bindings existed, so each wrote its own. Every team that hit it said so rather than papering over it. Consolidated into one shared crate plus PyO3 and wasm bindings, with a conformance corpus proving the content hash agrees across all three.

An easing curve was silently broken. cubic_1d dropped a factor, flattening every curve toward zero. Dissolves and fades still worked — they just were not eased. It would have shipped, demoed fine, and been discovered months later as "our animations feel cheap."

A dlsym name mismatch that fails silently. Swift bound vmax_surface_viewport; the renderer exports vmax_surface_get_viewport. dlopen succeeds, every other symbol binds, and the viewport is silently nil — so the view just never updates.

An authorisation hole. may_read_client returned true for any UUID when all_clients was set. "All brands" was never meant to mean "all brands in existence." A foreign brand ID reached the INSERT and was stopped only by a composite foreign key — defence-in-depth doing the primary control's job.

The MCP server was spec-pure and unreachable. Pinned to revision 2026-07-28, which removed initialize — but every released client opens with initialize. The server was correct and no real client could connect to it. Fixed by answering the handshake truthfully from discover() without allocating a session — the thing the spec change actually cared about.

Licence traps that would have survived review.

  • WhisperX's default Hindi alignment model has no stated licence at all; its fr/de/es/it models are CC-BY-NC under a permissive package licence.
  • NVIDIA's most-cited Sortformer diarizer is CC-BY-NC while its streaming variant is CC-BY-4.0.
  • Depth Anything V2 Base and Large are restricted; Small is not.

Same families, different terms. All now enforced as executable checks that raise, not documentation that gets skimmed.

Technologies

  • Rust — timeline document, validator, canonical form, content hash, edit operations, compositor, C ABI, GPU timeline renderer (wgpu). One implementation, consumed by every other language.
  • Python 3.12 — perception (ffprobe, VFR normalisation, WhisperX, beat detection, candidate generation), render (FFmpeg, libass), export, provenance, generation routing, multi-tenant API (FastAPI, Postgres, row-level security).
  • TypeScript / React / Next.js — component library, editor surface, web app, marketing site.
  • Swift 6 — native macOS shell with an embedded MCP server.
  • MCP — 27 tools, spec revision 2026-07-28, stateless. Published as vmax-mcp on npm.
  • Claude Agent SDK — the planner loop.

How Did You Use Claude?

Claude is the intelligence layer of the product and the build.

In the product

  • The planner loop runs on the Claude Agent SDK. Claude takes a natural-language instruction and turns it into a proposed batch of edit operations against the OTF document.
  • Claude drives the edit through MCP. The engine is exposed as an MCP server with 27 tools (spec revision 2026-07-28, stateless). Claude reads a timeline projection, requests a candidate set, and proposes trims, cuts and captions as document operations — which come back as a version bump plus a human-readable diff.
  • Claude selects, it does not invent. Deterministic signal processing produces the ranked candidate set; Claude chooses an index. This is the core architectural decision of the project — it is what makes an agentic editor reproducible instead of merely plausible. Every proposal is idempotency-keyed: replaying the same key returns replayed: true with the document unchanged.
  • Claude reasons over frames, not vibes. The frame-wise extractor gives it addressable frames and word-level transcript timing to ground a decision in, instead of asking it to imagine what is at 00:42.
  • Claude writes the brand-locked prompts. Image and reel prompt construction runs against a brand twin, so the constraint is enforced in the prompt rather than hoped for in the output.

Every operation Claude proposes carries its reason — which is what makes the diff an approval trail rather than a changelog.

A real MCP client drove a real edit end to end: → 30/30 checks green.

In the build

Claude Code was the primary development surface across a Rust workspace, Python services, a TypeScript editor and a Swift macOS shell — writing and consolidating the shared timeline crate and its PyO3/wasm bindings, and building the conformance corpus that proves the content hash agrees across all three languages.

It also caught things we would have shipped:

  • the render-cache key bug that would have invalidated every chunk on any edit
  • the silently-flattened cubic_1d easing curve
  • the dlsym name mismatch that binds cleanly and returns nil
  • the may_read_client authorisation hole where "all brands" meant "all brands in existence"

Each of those was found by a security or correctness pass run against our own code, and each would have passed a demo.

What is the deployed URL for this project?

https://vmax-live.vercel.app

Discussion

Builders also viewed

See more projects on Devfolio