Home Creators Posts Import Register Favorites Logout

Content

CHAPTER 99: The Sound of the World and the Color of Memory

Lena had waited eleven months for this moment.

Since George announced Los Santos in January, Lena had been working in parallel with the NGN team: taking notes on the systems, understanding how the streaming worked, studying the limitations of each platform. She couldn't build the audio system until the rendering engine was stable because the audio depended on knowing where the player was, which chunk was loaded, what time of day it was, and what objects existed in the space. But she could design it. And she had designed it in a forty-page document that had been growing for months, feeding on every technical decision the team made.

The NGN audio system had three layers.

The first layer was the positional audio: sounds that existed at a specific point in 3D space. A car passing down the street emitted engine sound from its position. A buzzing streetlamp emitted sound from its position. A talking NPC emitted sound from its head. The volume of each source diminished with the distance to the player following an attenuation curve Lena calibrated by type of sound: the low sounds (engines, explosions, music bass) attenuated more slowly than the high ones (voices, glass, footsteps) because low frequencies travel farther in real air.

The direction of the sound was stereo: if a car passed to the player's left, the sound came out louder from the left speaker. If it passed to the right, from the right. If it was directly ahead or behind, both speakers at equal volume. Lena implemented a panning function based on the angle between the sound source and the direction the player was looking: 0 degrees (front) was central balance, 90 degrees (left) was full left balance, 180 degrees (behind) was central balance with a high-frequency filter that simulated the occlusion of the player's head, the acoustic effect that sounds coming from behind sound more muffled because the head blocks the high frequencies.

The second layer was the ambient reverberation. The same sound was heard differently in an alley than in an open street than in an interior. Lena defined four types of reverberation: open exterior (minimal reverb, the sound dissipates fast), street between buildings (medium reverb, the sound bounces between the facades), narrow alley (high reverb, echoes that return fast and give the sound an enclosed quality), and interior (reverb adjusted to the size of the room: a small bathroom had more reverb than a large warehouse).

The engine detected the player's environment with raycasts that measured the distance to the nearest walls: if the walls were less than 5 meters away, the environment was an alley. If they were more than 20, it was an open street. If there was no visible sky, it was an interior.

The third layer was the ambience: sounds that didn't come from a specific source but that existed as the texture of the space. The murmur of distant traffic. The hum of the city. The birds of the morning. The crickets of the night. These sounds had no position: they came out of both speakers at the same volume and changed according to the district and the time of day. They were the sounds Lena was going to record in Los Angeles months later, but that for now were placeholders: generic city loops that served to test the system without having the final content.

The three layers blended in real time. At any given moment, the player could be hearing: the engine of the car he was driving (positional, ahead), a police siren (positional, to the right, attenuated by distance), music from a store (positional, to the left, with street reverb), the ambience of South Central (non-positional, stereo), and CJ's footsteps on the sidewalk (positional, below). Five sources playing simultaneously, each with its own position, its own attenuation, its own reverberation.

"How many simultaneous sources does the system support?" George asked.

"Thirty-two on PC. Sixteen on Xbox. Eight on PS2. The sources are prioritized by distance to the player: the nearest ones always play. The far ones are eliminated when the limit is reached, starting with the most distant and least audible."

"Eight sources on PS2. Is it enough?"

"It's enough if I prioritize well. The engine of the player's car always has a reserved slot. The radio always has a reserved slot. The gunshots always have a reserved slot. That leaves five for everything else. Five well-chosen sources do more than thirty-two badly chosen ones."

*

Lena integrated the positional audio into the Grove Street block one Thursday at seven in the evening.

George was in the office. Lena asked him to put on the headphones and drive down the street with his eyes closed. George looked at her like someone who thinks the request is absurd but trusts the one making it.

He put on the headphones. He closed his eyes. He drove.

The sedan's engine purred in front of him: low frequency, constant, with a vibration that rose when he accelerated and dropped when he let off the accelerator. The tires on the asphalt produced a continuous hiss that changed tone with the speed: low and slow at 20 km/h, high and fast at 60.

A car passed on his left. The other car's engine appeared in the left speaker, moved across the stereo field as the car passed, and faded out through the right speaker. The Doppler effect Lena had implemented as a frequency shift proportional to the relative velocity made the sound of the car drop in tone as it moved away, a detail so subtle that George wouldn't have noticed it consciously but that his brain registered as right.

A dog barked to the right. Two barks, spaced a second apart, with the reverberation of a residential street that bounced the sound between the facades of the houses. Then an NPC to the left said something in a murmur George couldn't make out because it was 15 meters away and the attenuation had reduced the volume to 30%.

George opened his eyes.

"I can hear the street," George said. "With my eyes closed I can tell where the other car is, where the dog is, which side the NPC's voice is coming from. I can orient myself by sound."

"That's the point," Lena said. "The positional audio isn't decoration but information. The player who listens carefully knows where the dangers are before seeing them. A siren to the right means the police are coming from the right. A shot behind means the shootout is behind. The sound is a second pair of eyes."

George took off the headphones and looked at Lena. It was the same thing he had felt seeing the Greenpath tiles and the City of Tears rain: what he had just experienced was better than he expected, because Lena always produced things better than expected.

*

The post-processing was the last step of the rendering pipeline: the effects that were applied to the complete image after all the triangles were drawn.

George implemented three post-processing effects Marcus had requested since the first day of the project.

The first was bloom: the glow that spread around the intense light sources. The streetlamps, the headlights of the cars, the sun, the neons of the stores. Without bloom, the lights were bright points with sharp edges that looked artificial. With bloom, the light overflowed beyond its source the way it does in reality when you look at a bulb with your eyes half-closed: a soft halo that blurs the edges and gives the light an organic quality.

George implemented the bloom with a four-step process: extract the brightest pixels of the image to a separate buffer, blur the buffer with a two-pass gaussian blur, add the blurred buffer to the original image, and adjust the bloom intensity according to the time of day. At noon with direct sun, the bloom was subtle. At sunset, it was pronounced: the streetlamps and the neons glowed with wide halos that turned the street into a river of unfocused light. At night, each light source was a star with rays that extended in the four directions.

Marcus saw the bloom working and nodded with the silent approval of someone who sees a tool he was expecting. "The bloom is what separates a computer image from a camera image," Marcus said. "Cameras have bloom because lenses have imperfections. Computers don't have bloom because renders are perfect. Perfect renders look fake. The bloom adds the imperfection the eyes expect."

The second effect was fog: the distance fog that faded the distant objects toward the color of the sky. It was the permanent fog of the atmosphere, different from the weather system's (that one was a climatic effect that activated and deactivated): the effect where distant objects lose contrast and saturation because the light scatters in the air between the object and the observer. George implemented it as a linear blend between the color of the object and the color of the sky, proportional to the distance. At 500 meters, the object kept 70% of its original color. At 1,000, 40%. At 2,000, 10%. It was the transition that made the horizon impostors blend with the sky naturally.

The third effect was the color grading: the global adjustment of the color of the whole image that gave Los Santos its distinctive visual look. Marcus had defined the color grading as "the film of the 90s": warm tones with a yellow-orange tint more pronounced than reality, blacks that weren't completely black but had a bluish tint, and a saturation 15% higher than real life. The color grading was applied as a lookup table (LUT): a 256-color texture that mapped each input color to an output color.

Marcus created the LUT by painting the colors by hand, adjusting each tone until the image looked the way he remembered the films he had seen as a teenager about Los Angeles looking: Boyz n the Hood, Menace II Society, Friday. The look of Los Santos was the look of the films about LA, a layer over the look of LA itself. And the difference was the color grading.

*

Marcus created two LUTs: one for the day and one for the night.

The day LUT was warm: reinforced yellows, intensified oranges, greens that leaned toward olive. It was the look of Southern California at two in the afternoon: everything bathed in a light that made the whites cream and the grays beige.

The night LUT was cool: amplified blues, reduced oranges except in the artificial light sources, deeper blacks. It was the look of a city at night filmed with high-sensitivity film: the shadows absorbed the color and only the streetlamps and the neons kept their warmth, like islands of light in an ocean of dark blue.

The engine interpolated between the two LUTs according to the time of day. At noon, the day LUT at 100%. At sunset, a blend of 70% day and 30% night that produced the most cinematic moment of the cycle: the warm tones of the day mixing with the cool ones of the night, the orange of the sun competing with the blue of the shadow, every surface of the world tinted with the visual tension of two palettes fighting for dominance.

"The sunset is when the two LUTs fight," Marcus said. "And the fight is beautiful."

*

On November 3, 2005, George compiled NGN with the thirteen systems integrated.

Renderer. Lighting and shadows. Asset streaming. LOD with impostors. Occlusion culling. Day/night cycle. Weather system. Vehicle physics. Skeletal animation with ragdoll and IK. Combat system. Third-person camera. NPC AI. Positional audio with ambience. Post-processing with bloom, fog, and color grading.

Thirteen systems that had taken eleven months and that worked together like an organism. Each system depended on the others: the renderer drew what the occlusion culling allowed it to, lit by the day/night cycle, loaded by the streaming, simplified by the LOD, sounded by the audio, inhabited by the AI, driven by the physics, animated by the skeletal, and filtered by the post-processing. Removing any system was like removing an organ: the rest kept working but the result was incomplete.

*

George drove down Grove Street at sunset.

The real Grove Street, now without cubes: stucco houses with dry yards, bars on the windows, a mural on the corner wall that was a placeholder but that occupied the space where a real mural painted by the street artists of the Mission would later go. The streetlamps turning on with their random delay. The shadows of the lighting system lengthening across the asphalt with the sun on the horizon line. The smog of the weather system tinting the sky an orange the color grading amplified until it became fire.

The sedan advanced down the street. The suspension absorbed a bump Elena had put in the asphalt as a level detail. An NPC walked down the sidewalk with his five-step routine and turned his head to watch the car pass. Another NPC stood on the corner with two companions, the group animation Priya had created where three characters stood in a triangle looking at each other as if conversing.

The radio didn't exist yet. But Lena had put in a placeholder: a loop of seventies funk that played from the car speakers with the attenuation of a closed interior, the low frequencies bouncing against the simulated windows with the reverberation of a two-cubic-meter space. The funk mixed with the sound of the engine and with the ambience of South Central: a distant dog, a helicopter that didn't exist visually but whose rotating audio existed as a positional source 500 meters over the map.

George braked at the end of Grove Street. The sedan stopped in front of CJ's house with the frontal compression of the suspension and the screech of the brake Lena had recorded from a real car in the Townsend Street parking lot. The engine turned off. The funk turned off. And the ambience of the neighborhood filled the headphones: the dog, the distant helicopter, the crickets that started to sound because the day/night cycle was going from six to seven and Lena had programmed the crickets to start at sunset with a 30-second fade.

George stayed sitting in the car parked in front of CJ's house looking at the sunset through the windshield. The bloom of the streetlamps created halos of light on the glass. The color grading transitioned from the day LUT to the sunset mix. The shadows disappeared as the directional light weakened and the point lights of the streetlamps took control.

Eleven months. Thirteen systems. An engine built from scratch that could render a street, light it, sound it, populate it, and make it feel like a place where someone lived.

NGN was complete.

George turned off the monitor. He got up. He walked to the wall where Marcus was going to paint the new Townsend Street mural and where there was still nothing except white concrete. He stood looking at the empty space. And he thought about what was coming.

NGN was the foundations. Los Santos was the house. And building the house was going to take twice the time the foundations had taken. But the foundations held. Eleven months of code, each line written with the knowledge of twenty years of future applied to the limitations of the present, each system designed to be replaceable and extendible and portable to hardware that didn't exist yet.

Thirteen systems. Zero loading screens. A street that felt like a neighborhood. And everything that Marcus and Lena and Elena and the team were going to build on top was going to work because the ground beneath their feet was solid.

George turned off the office light. The monitors went dark. The whiteboard with NGN written in permanent marker by Daniel shone with the light from the street.

End of Chapter 99

Comments

No comments found for this post.