Architecture
How OpenFrame fits together: one project document, pure edits with snapshot undo, a command registry agents share, and a WebGL2 engine in the webview.
From docs/Architecture.md in the repository, rendered when the site is built.
How the pieces fit, and where each kind of code lives. Decisions behind the
shape are in decisions/; this page is the map.
The app is in openframe-application/: run every app command there, and
read src/, scripts/, src-tauri/ and fixtures/ paths in these docs
as inside it. The website is openframe-website/; docs/ at the root
serves both (ADR-081 The app and the website each have a folder, the docs stay at the root).
Layers
React UI (src/components) CapCut's four panels + overlays
│ reads slices via useEditor(), writes via actions/edit()
State (src/state) store (undo), actions (keymap handlers),
│ media import, library apply, drag, toasts
Pure core (src/lib) project types/ops/defaults, commands,
│ keyframes, catalogs, render plan, audio
│ math, timeline geometry, time, keymap
Engine (src/engine) media (mediabunny inputs), cursor, player,
│ compositor + shaders (WebGL2), text,
│ audio scheduling, thumbnails/peaks, export
Platform (src/platform) web (OPFS, File System Access) | desktop
(Tauri commands, media: protocol)
Rust (src-tauri) files, projects, export writes, FFmpeg
Arrows only point down. src/lib imports nothing from the layers above it
and has no DOM; src/engine never imports React.
A frame, from edit to pixels
- An action (
src/state/actions.ts) or command (src/lib/commands.ts) produces a newProjectvia pure ops;store.editsnapshots the old one for undo, andsettle()applies the magnet and linkage. - The player (
src/engine/player.ts) sees the change and requests the current frame; requests coalesce, latest wins. planFrame(project, f)lists the layers bottom-up with every parameter resolved (keyframes, animations, filter, transitions).resolveSourcesasks each video clip'sVideoCursorfor the sample at its source time (a quarter source-frame past the edge), images from a bitmap cache, text from the rasterizer.Compositor.renderruns the passes (layer → effects → blend; transitions mix two layer buffers; adjustment layers grade the accumulator) and presents over the project background.
Playback is the same loop driven by requestAnimationFrame, with the frame
derived from the audible audio clock; audio is scheduled separately by
scheduleAudio, which export reuses with an OfflineAudioContext.
Export (src/engine/export.ts) runs steps 3-5 once per output frame on a
private canvas, awaiting every source and the encoder instead of following
a clock, so no frame is ever dropped
(ADR-014 Export is a deterministic frame loop).
Module table
| Module | Owns |
|---|---|
lib/project/types.ts |
The document schema (also docs/Project format.md, planned) |
lib/project/ops.ts |
Every edit: place, split, trims, speed, freeze, detach, settle |
lib/project/compound.ts |
Compound clips: create, break apart, a compound's timeline as a view and back (ADR-037) |
lib/commands.ts |
Named commands + JSON Schemas; describeProject |
lib/analysis/* |
Silence, scene and beat detection, pure (AI tools) |
lib/render/plan.ts |
What is visible at a frame, fully resolved; a compound clip's layer carries its nested plan |
lib/export.ts |
Export settings to plan: sizes, rates, bitrates, estimate |
lib/textBox.ts |
Where the canvas text editor sits: a text raster's layout mapped into the viewer |
lib/keyframeEdit.ts |
Keyframes as a set: retime, move, ease presets, bezier handles, copy/paste |
lib/crop.ts |
Crop ratios, the largest crop of a ratio, where a dragged crop handle puts the edges (ADR-048) |
lib/mask.ts |
Mask shapes on the viewer: frame, outline, handles and drags, mirroring the shader's maskAlpha (ADR-049) |
lib/shapes.ts, lib/trackMatte.ts, lib/motionBlur.ts, lib/wiggle.ts |
Graphics: shape geometry and colors, which clips are track mattes, motion blur's shutter samples, wiggle's noise; the plan resolves them and the compositor draws them (ADR-056 to ADR-059, ADR-062) |
lib/blur.ts, lib/shadow.ts, lib/tilt.ts, lib/textAnim.ts |
Large blurs' pyramid plan and its CPU simulation; drop shadows; perspective tilt's homography; keyed title size, spacing and color (ADR-069 to ADR-071) |
lib/border.ts |
Borders on pictures: values, keys, the jump flood's steps; the compositor floods the picture's edge out and draws the border under it (ADR-099) |
lib/paramInfo.ts |
A parameter path's name and display units (Scale %, Position px, Volume dB) |
lib/catalog.ts |
Animations, effects, transitions, filters as data |
engine/compositor.ts |
WebGL2 passes, nested compound plans one framebuffer level deeper; shaders.ts holds the GLSL |
engine/cursor.ts |
One decoder per clip; scrub and play; reversed clips in backward windows |
engine/player.ts |
Preview loop, transport, shuttle, a silent bounded run (preview) |
engine/audio.ts |
Web Audio scheduling and the audible clock; retimed sound in blocks (lib/stretch.ts) |
lib/soundFx.ts, engine/soundFx.ts |
A clip's filters, echo, reverb and tape stop: values, impulse responses, time warp; the nodes (ADR-060) |
engine/mixdown.ts |
The mix rendered offline in pre-rolled windows, for export and measure_loudness |
lib/analysis/truePeak.ts, lib/analysis/delivery.ts, state/delivery.ts |
True peak, loudness range, delivery specs, the export limiter's plan, the loudness report (ADR-061) |
lib/music/*, engine/music.worker.ts, state/music.ts |
generate_music: theory, styles, the arranger (form, fit, chords, hook, events), instruments, the mix and master, measurements, the cue sheet; rendered in a worker, imported and placed (ADR-085, ADR-086) |
engine/thumbs.ts |
Filmstrips and waveform peaks |
engine/compoundThumbs.ts |
Compound clips' filmstrips (the nested timeline rendered offscreen) and mixed peaks |
engine/export.ts |
The export loop, codec choice, audio windows; stills (exportStills: the frames preview_frame and the contact sheets show, at the size asked) |
lib/brandKit.ts, lib/ads.ts, lib/versions.ts |
Ads: the brand kit and text roles, the readable text floor; ad elements laid out in the safe area; an ad's other shapes, cut-downs and textless master, groups fitted as one (ADR-063) |
engine/analyze.ts |
Decoding for analysis: mono PCM, per-frame scene scores |
engine/cutout.ts, engine/matte.worker.ts, lib/matte.ts |
Remove BG: the clip's frames through a segmentation model in a worker, steadied over time, stored as a grayscale matte video the compositor samples (ADR-032, ADR-033) |
lib/stickers.ts, engine/stickers.ts |
Sticker catalog and placement; the Canvas2D artwork and emoji rendering |
lib/templates.ts |
Word art and title templates as data, turned into text and box clips |
lib/markers.ts, lib/markerText.ts |
Marker edits/navigation/search and read-only notes/chapter formatting |
lib/mediaLibrary.ts |
Derived media filters, stable sorts and timeline reference counts; session preferences live in the store |
lib/overlays.ts |
Where a new overlay lands: above whatever shows at that time; what to play so it shows once added (ADR-047) |
lib/drop.ts |
Where a library drop lands (placeAsset, magnet cuts, overlays) and landing, which the drop preview draws from a dry run (ADR-046) |
engine/previews.ts |
Library card previews through an offscreen compositor |
state/store.ts |
App state, undo (whole-document snapshots), gestures; the open compound clips (nest) and document() |
state/project.ts |
Autosave drains changed revisions; one navigation transition at a time (ADR-039) |
state/projectWork.ts, lib/taskScope.ts |
Abortable document-session ownership for delayed imports, captions, stickers and analysis |
state/actions.ts |
Keymap action handlers, edit() |
state/export.ts |
The export job: dialog state, render() for automation |
state/textEdit.ts |
The typing session: live text edits from the canvas or the inspector, one undo step |
state/cropEdit.ts |
The crop mode's session: a copy of one clip's crop, the viewer showing it whole, Done as one undo step (ADR-048) |
state/ai.ts |
The AI tools: analyze, then one undoable edit |
automation.ts |
window.videoEditor |
platform/* |
Web and desktop implementations of Platform |
Decoder preparation (ADR-074)
The preview player uses lib/render/preroll.ts to prepare each upcoming
clip at its first visible frame, including transition handles. Keyframe
distance and measured decode throughput select 0.75-5 seconds of lead.
Admission estimates include decoder queue buildup within a 512 MiB budget
for additional cursors; oversized entries decode on demand. Export keeps
its own cursor pool and unchanged frame plan.
Music structure and voice balance (ADR-075)
analysis/beats.ts finds boundaries from adjacent bars supported by two-bar
novelty, allowing two-bar parts; smaller phrase-aligned timbral changes can
split variations. drops lists qualifying rises; drop stays the biggest.
analysis/balance.ts owns stem isolation, role conventions and attenuation
math. state/delivery.ts measures each requested stem with the offline
export mixer. state/ai.ts measures candidate ducking keyframes before one
undoable edit, rejecting results if the project changed during measurement.