Transformative Tools in Filmmaking
Jordan Graff
Tools Through Time
The tools to make films today don’t look the same as the tools that first made films over a century ago. Filmmaking is the at the intersect of art and technology and the tools available to filmmakers have been changing since the very beginning.
Source: National Film Board of Canada
Nothing illustrates that change better than the fact that filmmaking doesn’t require film.
Technological Waves
With every new wave of technology there are filmmakers who embrace it and filmmakers who resist it.
In 1927 the first film to have sound was released and during the 1930s silent films began to disappear. While sound brought with it new creative opportunities some filmmakers were initially skeptical that there would be a loss of the universal or visual nature of silent films.
The introduction of color film in the 1930s had a more gradual transition. Despite its expensive and complicated process some filmmakers choose to shoot in color.
Source: Warner Brothers
The Wizard of Oz (1939) utilized technicolor to create a memorable moment of Dorthy leaving Kansas and entering Oz
However the majority of films weren’t shot in color until the mid 1950s partly because it became easier and partly to compete with television.
While sound and color had the most lasting impacts over time other tools were added to the filmmakers toolkit, notably widescreen formats, CGI, and 3D. Some advancements have been used more heavily than others, but every advancement left the filmmaker with an additional tool and new ways to use them.
The New Wave: The Rise of AI
Now a new tool presents itself with algorithms, artificial intelligence and machine learning. AI is an all encompassing term for computer processes that perform a variety of tasks from object recognition to generating imagery.
Its role in the creative process has been met with both excitement and concern.
AI Speech-to-Text
In documentary work one of the most tedious task is creating interview transcripts. It’s so time consuming that a lot of budgets don’t have room for it and when they do the edit process can’t begin until the transcripts are finished, further extending the post-production schedule.
Transcription built into Adobe Premiere
By using AI speech-to-text a transcript can be generated faster than real time. First it popped up in third party applications and now it’s built directly into multiple industry-standard editing programs.
After a rough cut it typically becomes apparent that some additional soundbites or words could help smooth out sections or be used to transition topics. By performing a keyword search on the transcript we can quickly pull up relevant content without needing to rewatch all the interviews to find these specific moments. These auto-generated transcripts cut down on the time spent searching, making more time to focus on the creative decisions.
Working film sets are an exercise in ideals. It’s creating to the best of our abilities in the given time under the specific circumstance; which are never ideal.
Enhancing Audio with AI Tools
When watching a film good clear audio is essential. Audiences can watch something with great audio and bad visuals but it’s almost unbearable to watch something bad audio even if the visuals are great.
New AI tools don’t mean we should neglect capturing the best sound on set but it does mean we have a little more leeway if things do pop up. AI algorithms do a decent job in minimizing disruptive background noise in audio, meaning your audience only hears what we intended them to hear. Removing such sounds in the past have often been time consuming or cost prohibitive.
Sometimes we get to work with composers who are creating original music for our projects but other times clients don’t have the budget for it and in that case music is sourced from stock music libraries. We can cut the music in original ways but AI lets us take it a couple steps farther. We can now quickly change the key of music tracks to match other music cues, or even generate our own stems allowing us to remove certain instruments or more easily loop specific movements of a track. This helps us put a unique spin on it, making it feel a little more like it was made just for this piece.
Enhancing Imagery with AI Tools
Not only are film sets an exercise of aiming for ideals but sometimes the cutting room is too. The film is cut out of whatever raw material is provided and occasionally that material isnt’ captured by professionals or with professional equipment but comes directly from the client. It usually involves visuals vital to the narrative but the moment has passed to capture it. Maybe it’s a photo of someone who has passed away, or video from an event that has already taken place. This material is always better than nothing, but sometimes it comes at a cost of image quality.
Now tools exist that can take this material and enhance it. It can sharpen the image, remove grain, and even increase the resolution providing a broader range of creative options in how it is used.
The Future
Future advancements will allow us to index our footage making even the content of the footage searchable with keywords to find specific objects, people, or colors. The advantage of every asset being indexed and searchable is the ability to quickly find the material and explore different options even when dealing with massive amounts of footage.
Some MAKE/ projects have over 50 hours of footage
Creative Empowerment
Filmmaking is a way for a storyteller to connect with an audience; to foster understanding or empathy. Every time a new tool is added it opens up new possibilities to accomplish this task.
The opportunity with AI is accessibility to tools that remove hurdles from the creative process, allowing us to make creative choices instead of being constrained by choices made out of necessity or limitations.
For now AI is only as powerful as the hands that wield it. Much like a pencil or a paint brush the art may come out of it but it isn’t created by it alone. With most AI models the quality of the output is directly impacted by the quality of the input. These tools can empower filmmakers, assisting them to create with less friction. Utilizing these tools, like the tools that came before it can empower us to create more freely.