Stable Diffusion audio-reactive ANIMATION driven by Spitfire pianos [Music Video]

Music tools used in this clip: Bitwig Studio, Spitfire Cinematic Soft Piano, Spitfire Felt Piano, Spitfire Olafur Arnalds Composer Toolkit, Spitfire Olafur Arnalds Stratus, Spitfire Albion Neo, Soundiron Rumba Boxes Hopkins Instrumentarium, NI Una Corda Felt piano, Pianoteq 7, Pro-Q 3, Pro-L 2, Valhalla Room, Gullfoss, PSP Vintage Warmer.
Visuals:
Adobe Premiere Pro
Deforum Stable Diffusion colab v0.7:
colab.research...
For this video, I used approximately 80 compute units on NVIDIA A100-SXM4-40GB with v2-1_768-ema-pruned.ckpt checkpoint. I set my composition to 112.50 BPM so the sixteenth notes would match every 4th frame of the 30fps video (diffusion_cadence = 4). Then I wrote my custom strength/noise schedule to match the music.
…and btw, my new album ‘Heroides’, with the renowned opera singer Małgorzata Walewska, is now out:
Tidal: tidal.com/brow...
Spotify: open.spotify.c...
Apple Music: / heroides