My Video:
https://youtu.be/12tiLp4LQAY?si=5OI7GUGSa7oZdPSH
Compose a brief reflection to accompany your video (in the blog post), discussing your creative process, the challenges of maintaining consistency, and how you utilized AI tools to generate, refine, and audibly enhance your world and its narrative.
Imagery: Custom Chatgpt
Animation: RunwayML
Audio FX / Ambience / Vocal FX: Audiogen.co
Reese Bass / Some ambience: Me
This was a very challenging project that I ended up having to scale back on in order to reach completion.
Maintaining consistent imagery was a monumental task when trying to work with unique generations that adhered to a user chosen art style over “going with the flow” of whatever the AI generated.
Morphing and other inherent flaws of RunwayML prompted me to delve quite deep into the realm of generating unique content through AI.
Using a data-set of images generated using a custom GPT, I was soon able to slowly generate frames using RunwayML that provided smooth animation without morphing, however I was unable to get farther then basic arm movements and facial movement in the time I spent.
While I was unable to fully flesh out my idea, I feel like I am well on my way to understanding the basics of some of the advanced concepts used in some very impressive generations and am very keen to continue working on projects in my own time.
Many of the processes I found for completing high level projects involved running videos and images through several different AI-based programs in order to reach a profound level of control and manipulation of the content, instead of remaining at the whim of vague generations arbitrarily based on prompts.
Some interesting generations I found during the process:
Solid art style with basic animation, editing, and fx. Audio is great but would have been cool to see lipsync through Runway or Pikalabs.
https://youtu.be/EsbpCV0vWlM?si=mg504HDTEcsldGf9
Open Source AI animation is moving forward as well:
https://civitai.com/models/331718/bubbling-rings-or-animatediff-motion-lora
https://civitai.com/models/335070/simple-lcm-img2vid-workflow-or-comfyui
An amazing rendition of Zelda as a 1950s trailer:
https://www.youtube.com/watch?v=TNdvRHq8Hs8
Open Source AI datasets:
https://www.openml.org/
Example of how to build a dataset in order to generate unique content using Stable Diffusion.
https://www.youtube.com/watch?v=-qmnGSjvWII