01 Other projects · Quipcuts

Long videos in, captioned vertical shorts out.

Quipcuts (started as OpenClip) is an open-source take on tools like Opus Clip, built in October 2026. Upload a video or paste a YouTube link and it transcribes it word by word, asks an LLM for the strongest 30–60 second moments, reframes each one to 9:16 around the speaker's face and burns in captions that highlight the word being spoken. Every clip then opens in an editor: trim to the word, add a hook and a call to action, restyle the captions and add punch-in zooms on the loudest moments. The clip-picking model is an open-weights one (qwen3:8b), so it runs with no API key.

Free beta: sign in, then 3 videos a day up to 15 minutes each. Processing runs on a Modal GPU that sleeps when idle, so the first job waits about a minute and a half for it to start; a 4-minute video then takes about two minutes. Clips and uploads are deleted after 7 days.

fig. 01 · inputself-hosted Streamlit UI
OpenClip home screen: a heading reading Turn long videos into viral shorts, a YouTube link field with a Get clips button, and options for AI engine, local model, layout, clip length and number of clips.
The first, self-hosted version: one link field plus the options that matter (which model picks the clips, single or split-screen layout, clip length and count). The hosted app at quipcuts.app has the same options.
fig. 02 · a real runqwen3:8b · 78 s
Three generated vertical clips from Tim Urban's TED talk on procrastination, scored 87, 85 and 83 out of 100, each with burned-in captions and a highlighted word.
Three clips the local qwen3:8b model picked from Tim Urban's 14-minute TED talk Inside the Mind of a Master Procrastinator. The scores are the model's own guesses at how well each clip would do, not measured views. Video frames © TED, shown here as a test input.
78 sprocessing time logged for this run with qwen3:8b on a laptop (3 clips)
~30 sa run with llama3.2 (3B): faster, but only 1 clip
1080psource download, up from 720p after fixing the yt-dlp format string
12 / 12malformed-LLM-output test cases the JSON parser recovers

How it works

01 · download

yt-dlp, capped at 1080p

Separate MP4 video and M4A audio, merged by FFmpeg, plus a 16 kHz mono WAV for transcription. An FFmpeg check runs first and the UI shows setup steps if it is missing.

02 · transcribe

faster-whisper, word timestamps

Every word gets a start and end time, which later drive the captions. Falls back to CPU when the CUDA libraries are not installed.

03 · pick moments

Local LLM, structured output

Ollama by default (or any cloud model through LiteLLM). Small models often return broken JSON, so the parser repairs trailing commas, Python-style literals, unquoted keys and cut-off replies, then sends its errors back to the model for one retry.

04 · reframe

Face tracking per shot

OpenCV's YuNet detector samples 5 frames a second. Scene cuts are found from colour histograms, and the crop is smoothed within each shot so it doesn't drift across a cut.

05 · render

One FFmpeg pass

The crop path is fed to FFmpeg's sendcmd as a list of timed commands, and an .ass subtitle file highlights the active word in yellow.

06 · publish

YouTube upload, private first

Optional upload through the YouTube Data API with titles trimmed to fit and #Shorts added. Uploads start private and can be scheduled one a day over a week.

Two speakers: split screen

For interviews and podcasts, Quipcuts tracks two faces separately and stacks them: Face 1 in the top half, Face 2 in the bottom, each a 1080×960 crop with its own path. Faces are matched frame to frame by distance, and frames with more than four faces (audience shots) are ignored so a face in the crowd can't take over a panel.

In Auto it chooses split only when both faces are visible in at least 60% of the clip.

Tested so far on a synthetic video: two TED clips placed side by side. The figure shows its main weakness: when a face goes undetected for a whole shot, that panel can frame the wrong area. Here the bottom panel shows the stage logo instead of the second speaker's face.

fig. 03 · layout testsplit vs single
Two output clips from a layout test: on the left a split-screen clip with one speaker on top and a second, poorly framed panel below; on the right a single-layout clip following the main speaker.
Left: split layout. Right: single layout on the same input.

Limits

02 Other projects · ballsim

Physics simulations, rendered straight to YouTube Shorts.

ballsim renders the videos on GG Replay, a YouTube channel of "will the ball escape?" style simulations, started in October 2026. A scene is a short TOML file: the mode, the ring, the ball, the camera effects, the captions and the sound. ballsim simulates it at a fixed 600 steps a second, so the same seed always gives the same video, then draws every frame with Skia and encodes it with FFmpeg, adding a note of music on each bounce. A browser editor previews a scene live while you change it.

A personal tool: the code is private and it has one user, the channel's owner. ballsim uses YouTube API Services; using it means agreeing to the YouTube Terms of Service.

fig. 04 · long-formauto thumbnail
A thumbnail reading LEVEL 32 with a red IMPOSSIBLE? tag: a ball in a spinning ring, circled in red with an arrow, right at the edge of the ring's gap.
A thumbnail ballsim made by itself for a 32-level compilation. It simulates the level, finds the moment the ball comes closest to the gap, and marks it.
fig. 05 · 3DMuJoCo + Blender
A 3D maze seen from above at an angle, with four coloured marbles near a glowing green exit pad.
A marble race: MuJoCo simulates the marbles while the maze tilts, then Blender renders the frames.

What it does

01 · modes

20 scene types

Escape the ring, shrinking and spinning shapes, floor-is-lava, many-ball battles, and the colliding blocks that count the digits of π. Each is a small Python class on a shared physics core.

02 · formats

Shorts, long-form and 3D

9:16 Shorts, 16:9 level compilations that ramp up the difficulty and write their own chapter list, and 3D marble races rendered in Blender.

03 · channel data

Insights and uploads

Reads the channel's own numbers from the YouTube Analytics API (views, how many viewers stayed, retention), records which scene made which video, and uploads finished videos through the YouTube Data API.

How ballsim handles YouTube account data: privacy policy · terms of service.

NextGet in touch