Agent playbook

How an AI model should make an ad, an epic edit or a vertical short with OpenFrame's MCP tools, and the design rules it checks its work against.

From docs/Agent playbook.md in the repository, rendered when the site is built.

How an AI model should make a video with OpenFrame's MCP tools. A model gets the same text three ways: the help tool (topics ad, epic_edit, voiceover, social_short, design), the MCP prompts make_ad, epic_edit and social_short, and the resources guide://.... Every tool is in MCP tools; connecting is in Automation.

Design rules

  • Hook: the product or the most striking image in the first 1-2 s; the proposition in the first 3 s, the hook within 6 s, the brand within 5 s (TikTok's and YouTube's creative guides).
  • Pacing: ad shots average under 2 s (social ads 1-3 s, brand spots 2-5 s); epic edits cut on the beat, every 2-4 beats in a build and every beat after the drop.
  • Words: one idea per title, 2-6 words for a headline, on screen long enough to read twice (about 0.4 s per word, 1.5 s at least); lower thirds 3-6 s; captions at most 20 characters a second, two lines.
  • Type: at most two font families (a bold display face for headlines, a clean sans for the rest); style.size counts pixels at a 1080-tall canvas and scales with the height, so on 16:9 headlines are size 90-160 (8-12% of the frame height; an epic edit's one- or two-word slams up to 240), and on 9:16, where the frame is narrow, size 50-90 with 2-4 words a line; nothing smaller than 4.5% of the frame's short side, 5% on a tall frame or a short for the apps, since a phone shows every shape at its width (size 49 on 16:9 and 1:1, 43 on 4:5, 31 on 9:16, 54 for a 16:9 short; check_project names a smaller title with its share of the frame's height); inside the 90% title-safe area (align_clips margin 0.05); on 9:16 clear of the bottom 35% and the right 19%, where the apps' buttons sit (the viewer's TikTok, Reels, Shorts guide).
  • Contrast: text over footage gets an outline, a shadow or a box (style.stroke, style.shadow, style.background); aim for 4.5:1 against what is behind it. A soft shadow (blur 16 on small type) barely helps over a bright sky; a box or a 2-3 px outline does. check_project measures every title on the rendered picture.
  • Color: one grade over every shot (set_filter or apply_lut on an adjustment clip over everything), so the piece reads as one film. Titles and shapes added without track_id land above it and above the beat hits, so the grade and the punches leave them alone; one put under it by track_id is named in the answer's notes.
  • Motion: titles get an in and an out animation; push slowly into stills (frame_region); do not move everything at once; whip transitions follow the direction of motion.
  • Screen recordings (explainers, demos): push into the part that matters, then pan on (frame_region moves, one call; place_at holds a part at the picture's edge mid-frame, a band showing past the edge), ring it with add_callout on the word that names it, the rest dimmed, and glide the ring to the next part (to; sound clicks as it draws on and lands on each); check_project names a ring a pan carries off the frame; pin labels to the picture (add_text anchor), so a zoom never leaves them behind. export_versions crops each shape around what the callouts point at (else the region the push frames), a recording fitted with bands as well as one filling the frame, and keeps rings and spotlights on their parts; its check lists in callouts one that runs off a smaller frame, and in coverage footage left floating small.
  • Flashes: never more than three flashes in any one second (WCAG 2.3.1; UK ad rules test for it).
  • Sound: voice first, music ducked 10-15 dB under it (duck_under_speech; open_at ["Drop"] lets a drop between lines play), a whoosh on fast transitions, an impact on reveals with a riser whose peak lands on it, a beat of silence before the biggest hit; fade the music out over 1-2 s (set_fades). Silent footage gets its place's own sound under it (add_ambience: wind, water, rain, a city, a rocket's rumble, room tone), well under the voice. Listen before export: preview_audio plays a stretch of the mix or one stem (and measures it).
  • Loudness: online -14 LUFS with true peak at most -1 dBTP; EU broadcast -23 LUFS / -1 dBTP (EBU R128); US broadcast -24 LKFS / -2 dBTP (ATSC A/85).
  • End: logo, product, one call to action and where to go, held 2-3 s.
  • Check: preview_frames across the whole piece, preview_motion around cuts and moves, check_project for the rules above; fix, check again, then export.

A professional ad

Making an ad (15, 30 or 60 s) with these tools:

  1. new_project with a name; set_project for the shape: 1920x1080 for YouTube and TV, 1080x1920 for Reels, TikTok and Shorts, 1080x1080 or 1080x1350 for feeds.
  2. import_media: product shots, b-roll, the logo (PNG with transparency), music. describe_project for asset ids and durations; preview_media to see what is in each video. set_brand with the brand's colors, two fonts, the logo, url and tagline: every title role and ad element then agrees.
  3. Script in beats: hook (0-3 s), problem or desire, the product, proof (features, numbers, a demo), offer, call to action. Write the voice lines short. Put the grade's adjustment clip (step 7) on the timeline before any titles: new tracks stack on top (under the captions, which stay the top layer), and a grade added last would tint the titles and end card too (move_track below_track_id fixes the order later).
  4. add_voiceover with the script, directed (help topic voiceover: casting, pace, pauses, pronunciations; a line's caption can show other words than it says: "20% off" for "twenty percent off"): the voice and captions are placed at their real lengths, and it answers where each line landed. review_voiceover hears it back and names what to fix; retake_voiceover fixes a line; then cut the picture to the voice.
  5. Picture: add_media_clip a shot under each line; frame_region for slow push-ins on stills and product details; set_canvas_fill blur when footage does not fill the frame; set_speed for slow motion on hero shots.
  6. Words: a headline per idea with add_text and a role (headline, subhead, label...: sized and styled from the brand), animated per word or letter (set_animation wordPop, letterRise or typewriter; unit and stagger for any other) with an out; add_ad_element lower_third for a name, offer_badge for the offer, cta_button for the ask; text_behind_subject puts the hero title behind the person or product; set_caption_look for the captions (pop or boxed read well on social).
  7. Look: one grade for all shots (an adjustment layer with set_filter or apply_lut over everything), set_transition dissolve or a fast cut between shots. Graphics: add_shape for bars, boxes and lines that draw on (keyframe shape.draw), set_track_matte to show footage inside big title letters, set_border to frame a picture-in-picture shot (a rounded mask gets a rounded border), set_motion_blur on fast moves. A styled layer stack used again (a floating window: footage, mask, border) is a template: create_compound_clip it once, then duplicate_clips with unique true for each use and edit_compound replace_clip to put that use's footage in.
  8. Sound: music with add_media_clip on an audio track (or an original score with generate_music, its sections fitted to the cut and its final hit on the end card), duck_under_speech under the voice, set_fades out at the end, add_sound_effect on reveals, normalize_loudness on the voice.
  9. End card: add_ad_element end_card (logo, headline, the call-to-action button and the url, in the brand), held 2-3 s; or logo_reveal for a brand spot.
  10. preview_frames across the whole ad and check_project with kind ad; fix what they find (text cut off, too small, low in contrast, no end card); measure_loudness with spec online (or the broadcast spec) and loudness -14; then export_video (loudness -14). The other deliverables in one call: export_versions with check true first (a sheet of every shape and length), then without check: ratios 16:9, 9:16, 1:1, durations 30, 15, 6 and a textless master. Each version's crops keep what callouts point at and the regions the master pushes into, else follow the shots' subjects; the check sheet outlines in orange a subject a framing still cuts, or a callout off the frame (focus [{clip_id, x, y}] on that version places it by hand). Where a version needs more work by hand (an end card to lay out), export_versions with save_projects true saves it as a project in that shape: open_project it, fix it, export_video.

An epic edit

Making an epic edit (a hype video, montage or music-synced velocity edit):

  1. new_project; import_media the clips and one strong track (a clear beat and a drop), or compose one: generate_music (style launch or cinematic) with sections that fit the edit (a drop at the climax, the outro ending with the video; plan_music tries them without rendering) places the music and answers its cue sheet: tempo, bars, beats, sections, drops and the final hit.
  2. Put the music on an audio track first and analyze_music it with markers beats, bars and drop (generate_music's markers option does this for its own score): it answers the tempo, the beat grid, bars, phrases, named sections (intro, build, drop, breakdown, outro) and the drop, in timeline seconds (detail full adds every beat and hit); the markers follow the music clip through later trims, moves and speed changes. When the reading is wrong (the bars fall between the kicks in preview_motion, the drop is missed or early, a confidence is low), give the song's grid with set_music_grid (its tempo, a bar's first beat and the parts, in the file's seconds): the markers move onto it, and analyze_music, mark_beats and check_project use it from then on. Cut on downbeats and at phrase starts.
  3. fit_music drop_at to slide the music so its drop lands where the edit's climax should be (often 25-40% in for a short edit). A song too long for the cut loses whole bars with cut_music (the file's seconds, kept on its bar lines, the join crossfaded so it does not click), and one too short plays bars again with cut_music repeat (a final hit landing later); preview_audio's edges then measure the join, and a join into a silent breath before a hit is rolled a bar (roll_edit), not crossfaded. A flat loop has no drop (analyze_music answers drop null): make one at a bar about a third in by lifting the beat before it for a moment of silence (lift_range on the music), then add_marker label Drop at the bar; add_beat_hits marker_label Drop and the riser's end_at use it.
  4. preview_media each clip to find its best moments, then cut_to_beats with the shots in order (each with its in point): one call lays them end to end on the main track, every cut on the grid. Shape it to the music: intro (wide shots, bars, the title; pattern 4s), build (cuts every 2 beats), drop (every beat), breakdown (slow motion, a freeze with the title), final drop, logo. Run cut_to_beats per section with start and end, or one pattern for all.
  5. Speed: velocity edits run fast between beats and slow on the hit: set_speed_ramp lands a clip's slow (or fast) moment on a beat without moving its cuts.
  6. Hits: add_beat_hits flash, zoom, shake, rgbSplit, glow or zoomBlur on the drop (marker_label Drop), on bars, or at_cuts; it keeps flashes to 3 a second. Transitions whipLeft, whipRight, zoomPunch or spinBlur at a few cuts, in the direction of the motion.
  7. Look: one grade over everything (filter teal or a LUT on an adjustment clip), grain, and add_cinematic_bars (2.39) for a film feel. set_motion_blur on clips that move fast (punch-ins, whips) and set_edge_mode mirror on them so a shake or zoom never shows black; set_wiggle for a handheld shake on the drop.
  8. Titles: big, short, bold words (one display face, size 160-240) with pop or zoomIn; text behind the subject (remove_background on a copy above the title) for the hero shot.
  9. Sound design: add_sound_effect rise or reverseCymbal with end_at on the drop (it peaks there), impact, braam or subDrop on the drop's beat, whoosh on whip transitions; a set_audio_filter lowpass sweep that opens onto the drop, set_tape_stop for a stop before it; then fit_music end bar (or video) with fade_out to end the music cleanly.
  10. preview_motion around the drop and other key cuts (the beats are listed beside the frames), check_project with kind epic_edit (cuts on the beat, no more than 3 flashes a second, no black), measure_loudness, then export_video (loudness -14).

A directed voiceover

Voicing a video so it sounds read by a person (add_voiceover, review_voiceover, retake_voiceover):

  1. Cast: list_catalog section voices describes every voice (accent, gender, the model author's grade, character, what it suits). af_heart (grade A) is the most natural and the default; am_michael is the US man to cast first, bf_emma and bm_george the British pair. Two speakers in one call are lines with different voices: a question and its answer, a host and a product. A blend ("af_heart:70,af_bella:30") mixes two voices' styles.
  2. Write for the ear: one breath per line (8-16 words, a sentence or two); short words; no parentheses or symbols the voice cannot say; numbers as they are said, with caption for how they are written ("twenty percent off" shown as "20% off"). pronounce respells names and acronyms the voice gets wrong ({"MCP": "M C P", "OpenFrame": "Open Frame"}); captions keep the written words. Emphasis cannot be directed: the voice takes no mark that stresses a word (raising a word's stress was measured and changed nothing a listener would hear, so word stars are dropped with a note). To make a word stand out, write it last in its phrase, where English puts the main stress and the voice lengthens it ("The one I meant was her."), or [pause 0.2] before it.
  3. Pace: each line lands on its words a second over its speech (first word to last, the reader's own pauses at commas and sentence ends in, its pause marks out: a [pause] is the script's beat, so a line given one is not read faster to make up for it): read_as narration 2.3-2.8 (the default, a natural narrator), ad 2.6-3.1 or social 2.8-3.4; pace slow (an end-card tagline) or fast moves that band half a word a second, and a slow line is never read faster than the voice's own speed 1. words_per_second [lowest, highest] is held as given, whatever the pace; read_as and words_per_second go per call or per line (mission control's radio calls read as social among a narrator's lines). A band never rushes long words: one whose top would say a line's words faster than 5 syllables a second moves down until it does not ("Honestly? Mostly water.", seven syllables in three words, reads 1.64-2.14 words a second, not 2.3-2.8), and the line's answer says so in pace_note; a speed given reads it faster. Leave speed out: each line is read, measured and read again at the speed that lands it (up to three reads, never time-stretched), and its answer gives the speed, the words a second and the band. With Heart, narration lands at speeds 0.6-0.9 (short words need the slowest); the old fixed 0.9 read the same lines at 3.7-4.2 words a second. A speed given is kept as given. review_voiceover reports each line's words and syllables a second over its speech and over its clip, and the script's words a second with the gaps between lines in (lower: 2.3-2.8 inside lines reads about 1.8-2.4 across a script).
  4. Breathe: a comma rests 0.25-0.35 s, a sentence 0.5-0.8 s, a paragraph or section 1-1.5 s; each line's ending sets the gap after it (pause_after or paragraph: true for more). Inside a line, [pause 0.4] before a reveal or the product's name and [beat] before a punchline; not more than one every four to six words. at puts a line on a beat or a cut; the lines after it follow.
  5. Finish: natural (the default) gives every line a reader's pauses, a low cut and a little presence, one loudness, room tone under the pauses, the voice compressor and a touch of room; clean or raw when the voice is processed elsewhere. Captions: one a line by default; caption_words 1-3 shows a few words at a time, each word timed to the voice (a short's pop captions), and a retake captions its line the same way.
  6. Mix: the voice 12 LU over the music (10-15 for a promo, never under 8). How far over depends on the window, and every answer names it: program (the whole stretch, the music between lines counted: it reads lowest), speech (while the voice speaks, its lines together) and line (each line alone). duck_under_speech (track_id ducks every clip of a music track in one call, a song cut into pieces included) voice_above_music_lu aims the speech window; min_line_lu holds every line to a minimum, each dipping only as deep as it needs, and max_line_lu to a ceiling (15 LU unless given), so a line over a quiet intro or a last hit dips less instead of emptying it. Pauses between lines under min_gap (1.1 s) stay down so the music does not pump, and the answer lists them (held); open_at ["Drop"] brings the music up on a song's drops (or any marker or second) between lines. Volume keys made by hand outside the lines (a lift on a drop, a hold before the last hit) survive ducking again. A voice band-passed for a radio or a phone (set_audio_filter) loses 2-4 LU: keep_loudness true makes it up. The music keeps how it was ducked: retake_voiceover dips it again under the new take, and an edit that moves a line under it says so. review_voiceover measures each line; measure_loudness by_track gives all three windows, and kind film, ad, social or explainer judges the mix's loudness range (a film that measures 2 LU is very flat; a voice-led explainer over a ducked bed sits at 1.5-6 LU). A drop or a final hit inside a line plays ducked, and the answer says which: start the line after it and duck again with open_at. The voice at one loudness across lines (review_voiceover flags any line 1.5 LU off); export at loudness -14 for online.
  7. Listen back: review_voiceover after every voiceover and retake. Each line answers what was heard against the script, its pace against its band, silences, pitch range (under 3 semitones sounds flat), loudness and margin over the music, and a verdict: ok when nothing fails, otherwise fixes, the tool and arguments to call; advice is a note that does not fail the line (a pace a tenth off its band, short words drawn out). retake_voiceover fixes one line in place (new text, voice, words_per_second, speed or pronounce; ripple true when it grows); the voice reads the same input the same way, so change something. A difference heard again by the larger model and called likely recognizer needs no retake; one called likely voice comes with a respelling to try, and when a respelling did not help, reword the line. Review again until every line is ok, then cut the picture to the lines' final times (describe_project).

A short for TikTok, Reels or Shorts

Making a vertical short from a talking-head clip or screen recording:

  1. new_project; set_project 1080x1920; import_media.
  2. add_media_clip the clip; remove_silences and remove_filler_words tighten the talk.
  3. auto_reframe the clip to 9:16 (or frame_region onto the speaker or the part of the screen that matters); set_canvas_fill blur for anything that does not fill the frame.
  4. Captions a few words at a time (the pop look): a voiceover made here captions itself so, add_voiceover caption_words 2 or 3 (each word timed to the voice; a caption of one word keeps its color, the highlight lights the word being spoken among several); for recorded speech, get_transcript (words with times) and add_captions with a few words per caption and their word times; then set_caption_look pop. check_project reads such captions as the line they make. On a tall canvas captions sit at y 0.05, clear of the apps' captions and buttons; set_caption_look y, size and maxWidth move, size and narrow every caption at once.
  5. Hook title in the first second (add_text, size 120-160, with a box or outline), a few stickers or sound effects on punchlines (add_sticker, add_sound_effect).
  6. Music low under the voice (duck_under_speech), set_fades, normalize_loudness.
  7. preview_frames, then export_video with the social preset sizes (1080p).

Prompts

  • make_ad: Plan and edit a professional ad from the media given, following the ad playbook. Arguments: product (required), length, shape, media.
  • epic_edit: Cut a music-synced montage or hype edit from the clips and track given. Arguments: theme (required), length, media.
  • social_short: Turn a talking-head clip or screen recording into a captioned 9:16 short. Arguments: media (required), length.