I’m currently doing a trial of a site that lets you access a bunch of different AI tools in one place. Having all those services easily accessible through one subscription instead of multiple different ones has its ups and downs in terms of feature parity, and some other issues. But it does let you work relatively rapidly in a single shared UI to generate multiple different assets pretty quickly in parallel.

From that experience, I just wanted to capture a really simple workflow to use for creating AI videos. It goes something like this:

  • Create a clear premise for the video, and a desired output length, and any other structural, stylistic, or content constraints
  • Use that to generate a script in [AI text generation tool of your choice], where each part of the script is broken up into blocks which correspond to video segments that will be generated later, and which include prompts to use in the image & video generators you use
  • For each block, generate one still image using [AI image generator of your choice] if the subject of that block is singular, or two or more to use as keyframes in a block that has more complex or multiple subjects, actions, transitions, etc. that occur in it. (And where the video generation tool used in the subsequent run is compatible for that use)
  • Then use the still frames as image to video prompts for each segment from the script, where you also include the text prompt.
  • [I haven’t mastered how to handle audio within individual clips yet, so generally I am just having a single voice-over for my tests for right now. For this, I simply input the script made at the beginning, defined the voice type and voila!]
  • Once you’ve got your script, your video segments, and your audio components, drop it all into Codex and say “make this into a video that matches the script.”

I did basically that with Codex, dropped all the assets I’d generated elsewhere, gave it the script as a guide, some brief context and directions, and a few minutes later, it did output for me a completed short video, around 1m22s. It was coherent and watchable, but had some editing choices that would have been a lot more effective if they’d come in a beat or two later or earlier. But overall I was pretty blown away by the process, which is why I wanted to capture it in rough outlines here.