Chapter 7
Content Creation
Technological Innovations
Still Image
GPU
OVERVIEW
A Graphics Processing Unit, or GPU, is a specialized electronic circuit designed to rapidly perform the mathematical calculations needed to render images, video, and animations on a display. Unlike a Central Processing Unit (CPU), which handles a wide variety of general-purpose tasks using a relatively small number of powerful cores, a GPU contains thousands of smaller, simpler cores optimized for parallel processing—meaning it can perform many calculations simultaneously rather than sequentially. NVIDIA, under the leadership of Taiwanese American entrepreneur Jensen Huang, coined the term "graphics processing unit" for the launch of the GeForce 256 graphics card in 1999, which is widely considered the first true GPU. NVIDIA defined the graphics processor as a "single chip processor with integrated transform, lighting, triangle setup/clipping, and rendering engines that is capable of processing a minimum of 10 million polygons per second." Modern GPUs come in two main forms: dedicated cards with their own dedicated video memory (VRAM), used in gaming PCs, workstations, and data centers, and integrated graphics processors built directly into a CPU or system-on-chip, common in laptops and smartphones. Today, GPUs are essential not only for visual rendering but also for any compute-heavy workload that benefits from massive parallelism.
INFLUENCES
The GPU did not emerge from a single invention but evolved from decades of work in computer graphics, parallel computing, and arcade hardware. GPU concepts go back to the 1970s, when computer graphics was developed for computer-aided design of automobiles and airplanes, and early computer graphics systems were adopted by the film industry and simulators for airplanes and high-energy physics. A major early influence was Jay Miner at Atari, whose custom display chips in the Atari 2600 and the Atari 400/800 home computers pioneered dedicated graphics hardware in consumer products. Arcade systems and the gaming consoles that followed—especially the PlayStation and Nintendo 64 in the mid-1990s—drove demand for hardware-accelerated 3D graphics and pushed manufacturers like S3, 3dfx, ATI, and NVIDIA into fierce competition. Academic research played an equally critical role. Government-funded academic research on parallel computing, stream processing, real-time shading languages, and programmable graphics processing units directly led to the development of GPU computing, drawing on earlier parallel architectures such as the Cosmic Cube and J-Machine. The introduction of programmable shaders around 2001 with NVIDIA's GeForce 3 and ATI's Radeon 9700 marked a turning point, transforming GPUs from fixed-function rendering chips into flexible, programmable processors.
IMPACT
Originally built to accelerate video games and 3D rendering, the GPU has expanded into one of the most consequential technologies of the 21st century. GPUs were originally designed for rendering graphics, but since the introduction of the GPU programming platform CUDA by Nvidia in 2006, developers have found countless applications for these powerful processors. CUDA allowed programmers to use GPUs for general-purpose computing tasks, unlocking their parallel power for fields far beyond graphics. CUDA accelerates physics simulations, fluid dynamics, and molecular modeling, and NASA and CERN use CUDA-powered clusters for simulations that would otherwise require supercomputers. In finance, GPUs analyze millions of market scenarios in real time, and hospitals leverage CUDA for medical imaging reconstruction, diagnostics, and drug discovery simulations. The most transformative application, however, has been in artificial intelligence. AlexNet ignited modern deep learning in 2012, made possible by custom CUDA kernels for AI operations like convolution, activations, pooling and normalization, and the horsepower a GPU can provide. This sparked the deep learning revolution that now underpins large language models, image generation, autonomous vehicles, and recommendation systems. Subsequent innovations include specialized Tensor Cores for AI matrix math, ray-tracing cores for cinematic lighting in real time, and competing accelerators like Google's TPU and AMD's Instinct line—all descendants of the original GPU concept.
Motion
3 Step Process
OVERVIEW
The three-step production process—pre-production, production, and post-production—is the standard framework used to organize the making of films, television shows, commercials, music videos, video games, and most other media projects. Pre-production is the planning and preparation stage that occurs before any filming or recording takes place. Production is the stage where the actual filming or recording of the project occurs. Post-production is the final stage that occurs after all filming or recording is complete, involving editing, visual effects, sound mixing, color correction, and preparing the project for distribution. Pre-production includes activities like script breakdown, storyboarding, casting, location scouting, hiring crew, budgeting, and building schedules. Production covers principal photography, with each day driven by the day's call sheet and shot list, and supported by departments handling camera, lighting, sound, costumes, makeup, and art direction. Post-production assembles the captured material into the finished work, layering picture editing, color grading, visual effects, music, dialogue cleanup, sound design, and final mixing. This stage is a critical planning and preparation period; on major productions, it can take up to one to two years. The same three-phase logic now structures nearly all collaborative media work, regardless of scale.
INFLUENCES
The three-stage process emerged from the industrial logic of early 20th-century Hollywood, where filmmakers needed a way to make complex, expensive movies on a reliable schedule. The origins of the studio system can be traced to the early 20th century. In the 1910s, the American film industry was in its infancy, dominated by small independent studios and theaters. However, as the popularity of motion pictures grew, so did the need for more organized production and distribution methods. The establishment of "movie factories" in Hollywood allowed for greater control over the filmmaking process. A major early influence was producer Thomas Ince, who pioneered the use of detailed shooting scripts and centralized production planning. Ince anticipated and effectively defined the roles of film producer and production executive during the nascent studio era. He refined a number of key aspects of the emerging system, from the shooting script as a blueprint for production to the centralized studio system and the assembly-line construction of multiple films. The studio system of the Golden Age then formalized the workflow across whole companies. Between 1930 and 1945, the studio system produced more than 7,500 features. Borrowing principles from industrial manufacturing—division of labor, standardized blueprints, sequential workflow—studios broke filmmaking into specialized departments that could work in parallel, with each phase handing off cleanly to the next.
IMPACT
Beyond feature films, the three-stage process has become the default organizing structure for virtually every form of professional media production: television series, documentaries, commercials, music videos, corporate video, video games, animation, podcasts, and even live-event broadcasting. The three key stages of video production—pre-production, production, and post-production—encompass everything from storyboarding, scriptwriting, scheduling, and location planning, through the shoot itself, to editing and the rough cut and final cut stages, and the same framework now structures corporate communications and YouTube channels alike. The impact extends well beyond entertainment: project-management practices in software, advertising, and even academic research borrow heavily from this sequential planning-execution-refinement model. Subsequent innovations have transformed each phase. Pre-production now relies on digital tools for script breakdown, virtual location scouting, previsualization, and animatics. Production has been revolutionized by digital cinema cameras, LED virtual production stages like those used on The Mandalorian, drones, and remote directing. Post-production has been the most radically reshaped, with non-linear editing systems (Avid, Premiere Pro, DaVinci Resolve, Final Cut Pro) replacing physical film splicing, and visual effects, color grading, and sound mixing now done almost entirely in software. More recently, AI-assisted editing, generative VFX, and cloud-based collaboration platforms have begun to blur the boundaries between the three phases, allowing certain post-production decisions to be made live on set.
Audio
Sound Synthesis
OVERVIEW
Tonal synthesis—more often simply called sound synthesis—is the process of generating musical tones electronically rather than recording them from acoustic instruments. Instead of capturing existing sound, a synthesizer creates a waveform from scratch (or modifies one) using oscillators, filters, envelopes, and modulators, producing pitches and timbres that can range from realistic instrument imitations to sounds that no acoustic instrument could make. Several distinct synthesis methods have become standard. Subtractive synthesis is the method of starting with a harmonically rich waveform created by an oscillator and attenuating it with a filter to create your desired timbre. This type of synthesis is the most commonly used, associated with the classic synths that started it all. The waveforms most commonly used in subtractive synthesis are square, saw, sine and triangle. Additive synthesis builds complex tones by layering individual sine waves, producing crystalline sounds ideal for orchestral emulation. FM synthesis uses frequency modulation between waveforms to generate bright, metallic textures. Other major approaches include wavetable synthesis (cycling through stored waveforms), granular synthesis (built from tiny audio grains), physical-modeling synthesis (simulating real instruments mathematically), and sample-based synthesis (manipulating recorded sounds).
INFLUENCES
The conceptual roots of sound synthesis stretch back to 19th-century mathematics and early electrical experimentation. Additive synthesis dates back to the early 1900s. The theory of additive synthesis, called Fourier's Theorem by Joseph Fourier, goes back all the way to 1822. The first commercially available additive synthesizer was the Telharmonium released in 1906. It was originally developed by Thaddeus Cahill in 1896 and patented in 1897, and is considered the first electromechanical instrument. Fourier's insight—that any complex waveform can be built from sine waves—gave engineers a mathematical recipe for sound. A church's pipe organ is a kind of additive synth. The different ranks of pipes correspond to different harmonics: by pulling out the stops, an organist can combine these harmonics to create sounds of different timbres. This method was carried into the electronic age thanks to electromechanical organs like the Hammond. The real revolution came with Bob Moog, whose voltage-controlled modular synthesizers in the 1960s made subtractive synthesis accessible to musicians. The next decisive leap was John Chowning's FM synthesis research at Stanford in the 1970s. The GS1 used frequency modulation developed by John Chowning at Stanford University, which Yamaha had licensed, and offered a foretaste of the crystalline timbres and metallic textures that were to become the signature of the DX7. Personal computers and digital signal processing then opened the door to software-based synthesis available to anyone.
IMPACT
Sound synthesis has transformed nearly every corner of music and audio production. It is the foundation of electronic genres like house, techno, EDM, ambient, synthwave, and hip-hop, while also providing iconic sounds in pop, rock, film scores, video games, advertising, and television. FM synthesis features heavily in 90s video game music as many sound cards used it to create complex sounds with little processing power. The Sega Genesis is the most famous example of this. The Yamaha DX7, released in 1983, was arguably the most influential single instrument in this lineage. It was the first successful digital synthesizer and remains one of the best-selling synthesizers in history, with over 200,000 units sold. The DX7's electric piano sound became heavily associated with 80s ballads, and its unique sounds were used by artists such as Whitney Houston, Tina Turner, Kenny Loggins, A-Ha, Hall & Oates, and many more. Its affordability also democratized music technology. Bob Moog, the inventor of the Moog synthesizer, spoke of the revolution in synthesizer pricing, which the DX7 helped to spur, as a "democratization" of the synthesizer. Subsequent innovations have steadily expanded the field: digital audio workstations like Ableton Live, Logic Pro, and FL Studio now host software synthesizers such as Serum, Massive, Omnisphere, and Pigments; physical modeling synths simulate acoustic instruments with stunning realism; and AI-driven tools are beginning to generate, design, and even compose with synthesized sound.
Creative Coding
Processing
OVERVIEW
Processing is an open-source programming language and integrated development environment (IDE) built specifically for the visual arts, designers, and creative coders. Processing is a free graphics library and integrated development environment (IDE) built for the electronic arts, new media art, and visual design communities with the purpose of teaching non-programmers the fundamentals of computer programming in a visual context. Processing uses the Java programming language, with additional simplifications such as additional classes and aliased mathematical functions and operations. It also provides a graphical user interface for simplifying the compilation and execution stage. Programs in Processing are called "sketches," and each one typically consists of a setup() function that runs once and a draw() function that runs continuously to update the screen. With just a handful of lines of code, a beginner can draw shapes, animate them, respond to mouse and keyboard input, generate sound, or load and manipulate images. Its syntax, inspired by languages like BASIC and Logo, allows beginners to quickly prototype ideas with code that generates dynamic visuals, such as shapes, colors, and motion, while supporting more advanced applications through community-contributed libraries. Processing is released under the GPL and LGPL, has the file extension .pde, and is freely downloadable from processing.org, making it accessible to artists, students, and educators everywhere.
INFLUENCES
Processing's roots lie in the MIT Media Lab, in a small group exploring how code could become a medium for artists. Cooper was concerned with how computers could be leveraged to expand the notion of traditional publication design and its accompanying elements, like typography. When Maeda joined the Media Lab in 1996, he pushed at the edges of Cooper's work, exploring what code could enable formally, in the contexts of both art and design. Cooper and Maeda established a long lineage of designers and artists who were interested in pushing the boundaries of what code could create. Among them were Ben Fry and Casey Reas, two research assistants in Maeda's group. The direct ancestor was Maeda's own teaching language. Ben and I both came to MIT to study with John specifically because he was bridging ideas of computer science and design together. We were both John's research assistants in the Aesthetics + Computation Group, where we were working with John on Design By Numbers (DBN), which was a programming language and environment and system to teach designers the basics of coding. DBN limited users to drawing in a 100-by-100-pixel space, and only in grayscale—faithful to my Bauhaus-style approach to computational expression. But Fry and Reas figured that people needed color. They needed canvases larger than 100 by 100. The project was initiated in 2001 by Casey Reas and Ben Fry, both formerly of the Aesthetics and Computation Group at the MIT Media Lab. In 2012, they started the Processing Foundation along with Daniel Shiffman, who joined as a third project lead.
IMPACT
Processing has shaped a generation of artists, designers, and educators working at the intersection of code and visual culture. It is hard to overemphasize the impact Processing has had on digital artists, creative coders, and even CS education. Processing is now used by professionals and hobbyists alike and has produced work featured in music videos (e.g., Radiohead's House of Cards), exhibited at art galleries, and featured in movies, TV, and other media. It is widely used to teach introductory programming in art schools, design programs, and increasingly in computer science classes, and it powers data visualizations, generative art, interactive installations, performance visuals, and architectural projections. Its influence reaches far beyond its own IDE. The Processing language and IDE have been the precursor to other projects including Arduino and Wiring. The Arduino IDE itself is built on Processing, which means the world's most popular hobbyist electronics platform inherits its look and feel. Subsequent innovations have extended Processing to new platforms and audiences. p5.js is a library and set of tools that make it easy to use the JavaScript programming language for creative coding. It's based on Processing, a creative coding environment originally developed by Ben Fry and Casey Reas. The advantage of using the JavaScript programming language is its broad availability and ubiquitous support: every web browser has a JavaScript interpreter built-in, which means that p5.js programs can (generally) run in any web browser. Daniel Shiffman's hugely popular "Coding Train" YouTube channel has further amplified its reach.