Re-storyboarding: Cutting up a short film and remixing it

What is it —

A program that automatically identifies scene changes in videos and reorganizes the clips in a new order.

It’s an experiment that sparked from a conversation one night as a, “I wonder how hard it would be to make that.”

Turns out it only took a couple hours to write one from scratch; no special libraries, no tutorials, etc. The main tools used are ffmpeg and youtube-dl (for demo purposes).

Additional work was later added to identify audio spectrums, scene colorization, and scene contexts. By adding more parameters, one could use such a tool to change the tone of a movie trailer, create an art piece, or re-storyboard a film.

See it in action —

Realtime example of re-storyboarding

What’s happening in the video —

First, it uses youtube-dl to download a selected short film. The film “The Elevator” was chosen due to its duration.

It parses frames out of the film at a calculated resolution and interval for processing. Comparisons are run on the frames in various batch sizes to determine differences. When a threshold is exceeded, it gets flagged.

It then runs through the map of flagged cuts and starts parsing clips out using ffmpeg.

This is around the point where we’d run additional processing or identification, but in the demo above, we start to restitch the film in random scene order.

At the end, you see the two films play simultaneously. Top is the original, bottom is the freshly generated film.

Like it?

See more work at Polymer Mallard (https://www.polymermallard.com)

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Matt Kenefick

Matt Kenefick

13 Followers

Chief Product Engineer. Frontend, Backend, Systems, Electronics, 3D Printing, Photography.