WEEK 7: AI Cinema (February 20) – DTC 491 Advanced Digital Cinema

To Do This Week

Blog Post: Using ChatGPT have a chat to explore a short “story” idea and its visual style that you can use to generate AI video clips (30-60 sec) for the AI Cinema project. Finalize a 200-300- word summary of a story or idea for your AI Cinema project. Post this in the blog. This will be the framework for your prompts. Also, in a separate paragraph, describe the look or style you want to aim for. You can reference directors, genres, or specific films, but also use words to describe the quality of the image: film stocks, cameras, lighting, color/bw, production design, sets, costumes.

Notes

Finish watching loops…

AI Cinema (5%)

DUE NEXT TUESDAY

In this assignment, you are to create a 30 to 60-second video, using Generative AI video and audio or blending these AI-generated visuals and sounds with footage you capture yourself. This project is an exploration into the vast possibilities of AI cinema, allowing you to experiment with new forms of video production. Whether you choose to tell a story or compile a thematic montage, the core of your project should revolve around a unifying theme or a central idea that resonates throughout the sequence. The challenge is to harness the power of AI to make a video that reflects your unique creative vision.

Project Requirements:

Generative AI (and Original Footage): Your video should primarily feature content created using Generative AI tools like Runway ML, optionally supplemented with your own filmed footage to add a personal touch.
Unified Theme or Story: Maintain a clear and consistent theme or narrative thread throughout your video, ensuring each clip contributes to the overall concept or story arc.
Visual and Thematic Consistency: Keep a consistent visual quality, style, and thematic approach across all elements of your video to ensure a cohesive viewing experience.
Sound and Music: Incorporate sound or music that complements the visual elements of your video, enhancing the atmosphere and emotional impact of your narrative or theme.
Format and Quality: Adhere to a single video format (preferably 16:9, 720p) to maintain quality and consistency throughout your project.
Presentation: Upload your completed video to YouTube or Vimeo, ensuring it’s accessible for viewing and evaluation. Include a direct link to your video in any submissions or presentations.

AI Tools:

Generating Shot List/Storyboarding Ideas:

Instructions for ChatGPT:

You are an expert in movies, filmmaking and visual storytelling. You are able to break down a written story or text into its visual parts, each part described in under 320 characters so that an AI image or video generator would be able to create visual images of the original written story.

I want you to help me create prompts for an AI image generation tool. I want to generate 5 images to help me visualize my story. I will write my story summary below and you will come up with 5 images to visualize key moments in my story. Provide ONLY a single-paragraph shot description for each moment to convey the setting, characters, situation, style, mood – explicitly. Do not write headings or titles to each shot. Just paragraphs.

For each shot description, repeat the description of the setting and/or the character(s), regardless if the same details were in the previous shot. Do not assume shot details are carried over from one shot to another. Don’t refer to a previous setting or character. , Describe the whole image again from scratch.

Here is my story:

“In a serene forest by a babbling stream is Clarence and his dog, Benny. Clarence enjoys his peanut butter and jelly sandwich while Benny watches. Clarence closes his eyes to savor nature’s beauty. The dog watches as the man wakes up. Where is his sandwich? Clarence and Benny embark on a playful quest to find the missing sandwich. Then Clarence collapses in mock tears as Benny licks his face. Eventually, Clarence reveals a second sandwich, and they both shared a laugh and a bark.”

Rework the results for each shot by using the same descriptive language for setting and character. Specify lighting, genres, filmstock, director styles:

Camera & Film Types

“Camcorder” – film grain, noise, and saturation, reminiscent of VHS tape
“DSLR” – superior resolution, sharpness, and clarity
“GoPro” or “GoPro Video” – fish-eye effect, often in the form of a selfie
“Disposable Camera” – a fixed-focus 35mm lens
“Polaroid” – high black point and faded effects
“Fujifilm Superia” – High-performance, high-speed color negative film with fine-grain
“Instax” – Daylight color film with a high gloss finish
“Kodak Ektar” – retro hues or poppy contrast
“Kodak Gold 200” – crisp and clear results for bright, sunny days
“Kodak Portra” – vivid colors, light, and textures in a crisp format
“Velvia” – High-saturation film [
“Lomo” – oversaturated, high-contrast look with a soft blur
“CinemaScope” – old filming technique known for its 2.35:1 aspect ratio
“Kodak 400TX” or “Tri-X 400TX” – fine-grain black and white images
“VistaVision” – high-resolution widescreen format using 35mm film
“Technirama” – double the resolution of anamorphic 35mm film
“Tintype” – direct positive images on thin metal coated with dark lacquer or enamel
“Calotype” or “Talbotype” – paper coated with silver iodide, vintage look.
“Ambrotype” – underexposed glass negatives placed against a dark background
“Daguerreotype” – highly detailed images on a sheet of copper plated with a thin coat of silver without using a negative

Camera Angles:

Eye-level: A neutral camera angle at the subject’s eye level, providing a natural perspective.
Low-angle: Looks up at the subject from below, emphasizing dominance or power.
High-angle: Looks down at the subject from above, making the subject appear smaller or less significant.
Dutch angle (or canted angle or tilted angle): The camera is intentionally tilted to create disorientation or unease.
Overhead: Looks directly down on the subject from above, offering a unique perspective.
Bird’s-eye view: Captures scenes from a high vantage point, providing scale and context.
Drone view: An aerial angle achieved using a drone.
Worm’s-eye view: Looks up at the subject from ground level, creating a dramatic perspective.
Ground-level view: Captures the subject from near the ground, often neutral in perspective.
Side view: Captures the subject from the side, achieved through various camera angles.
Off-center view: The subject is not centered, creating a dynamic composition.

Shot Types:

Close-up (closeup or CU): Focuses on a subject’s face or detail to emphasize emotion or importance.
Extreme close-up: Tightly frames a small detail or feature, highlighting its significance.
Wide/Long shot: Shows the entire subject within its environment for context.
Extreme wide/long shot: Captures a large area to establish a location or set the scene.
Medium shot: Captures the subject from the waist up, commonly used for dialogue scenes.
Medium-full shot (American shot): Frames the subject from knees or mid-thighs up.
Over-the-shoulder shot: Looks over one subject’s shoulder toward another, enhancing depth in dialogue scenes.

Lighting Styles

Natural Light: The use of available light sources, such as sunlight or ambient indoor lighting, to illuminate a scene. Often used for a realistic and unobtrusive look.
Key Light: The primary and most intense source of light in a scene. It provides the main illumination on the subject and establishes the direction of the light.
Fill Light: A secondary light source used to reduce shadows created by the key light. It fills in the shadows without creating additional strong highlights.
Backlight: Placed behind the subject, the backlight separates the subject from the background and creates a rim of light around the subject, adding depth and dimension.
Rim Light: Similar to backlighting, it’s positioned to create a thin line of light along the subject’s edges, emphasizing their outline and separating them from the background.
Top Light: Light that comes from above the subject, casting shadows downward. It can create a dramatic or mysterious look.
Side Light: Positioned to the side of the subject, it creates shadows that add depth and texture. Commonly used for film noir and dramatic scenes.
High Key Lighting: A style that uses bright, even lighting to minimize shadows and create a cheerful or upbeat atmosphere.
Low Key Lighting: Characterized by strong contrasts between light and dark areas, often used for dramatic or suspenseful scenes.
Rembrandt Lighting: Named after the Dutch painter Rembrandt, it features a key light placed at a 45-degree angle to the subject, creating a triangle-shaped highlight on the cheek opposite the light.
Three-Point Lighting: A standard lighting setup that includes a key light, fill light, and backlight, providing control over shadows and highlights.
Chiaroscuro: An artistic lighting style that uses strong contrasts between light and dark to create a visually striking and dramatic effect.
Practical Lighting: Incorporating actual light sources within the scene, such as lamps, candles, or practical fixtures, to create a natural and realistic look.
Candlelight: Emulates the soft, warm glow of candlelight, often used for romantic or intimate scenes.
Silhouette: Lighting the subject from behind, completely darkening their features and creating a distinct outline against a brighter background.
Soft Lighting: Achieved by diffusing the light source, resulting in soft, gentle shadows and a flattering, even complexion.
Hard Lighting: Uses direct and unfiltered light sources, creating sharp, well-defined shadows and a more intense look.
Bounce Light: Reflecting light off surfaces, such as white boards or walls, to soften and redirect the illumination.
Practical Effects: Using real-world light sources and techniques to achieve specific lighting effects, like flickering flames or dappled sunlight.
Stylized Lighting: Employing unconventional or artistic lighting techniques to create a unique visual atmosphere, often seen in experimental or avant-garde filmmaking.

Generating Production Design: locations, sets, costume, cinematography style

At this stage, you are just trying to work at setting, costume, lighting and cinema styles. Edit the shot descriptions to better refine your vision of the story to prepare for your image or video prompts. Be as descriptive as you can. Then use an image generator – Dall-e, RunwayML’s image generator. Midjourney has some special features:

Use Midjourney/Dall-e or RunwayML to refine the look of your images

RunwayML:
Generate images to feed Gen1 or Gen2 video generation. You can also create or upload a set of images for AI Training. This will allow you to create consistent images for your videos.

Dall-e 3 (with ChatGPT Plus):
A single chat fpr image generation will develop your image generation and keep some consistency. Specify style and subject, then add/remove to get the image to your vision.

Midjourney:

Midjourney: create a custom option – /prefer option set : cinematic: --ar 16:9 --stylize 150 --weird 25 --style raw
Midjourney: add parameters to your option: --stylize 150 --weird 25 --style raw
capture seed numbers for continuous look
keep creating variations until you are happy with the result
upscale the images you like, create variations with added prompts until you get what you are looking for
use the setting “zoom out” on images if you need more of the setting or a wider shot.
you can also combine your own image with another image created with another prompt /blend
you can also add more description to any image to get more variation. Turn on Remix Mode in /settings

Generating Characters

Midjourney

To build images of characters, play with /blend and /describe. Zoom out for wider shots of the body and the setting. Use

Reverse engineer the image and get descriptive text for use in the video generator. This will come in handy if the images aren’t working well.

/describe

Generating Video Clips

Open RunwayML

You can do many things with this AI video tool: easy green screen, slow motion, frame interpolation, remove objects in the frame. Generate video from other videos and prompts. I will focus on the video AI effect Gen 2. This means you can generate video from a prompt, generate video from an image or from an image and a prompt.

You create 4 seconds clips and can extend a single clip to up to 18 seconds.