Home Creators Posts Import Register Favorites Logout

Content

CHAPTER 91: What the Player Doesn't See

Every frame of NGN had a budget.

It was a budget of polygons: the maximum number of triangles the renderer could draw in 33 milliseconds (30 fps) without the framerate dropping below the target. On PC, the budget was 200,000 triangles per frame. On Xbox, 150,000. On PS2, 80,000.

A Los Santos building had between 500 and 3,000 triangles depending on its complexity: a low Grove Street house was at the lower end, a Downtown skyscraper at the upper. A car had between 800 and 2,000. A palm tree, 400. An NPC, 1,200. A streetlight, 50. A traffic sign, 30.

George did the math one afternoon in April with a spreadsheet Claire would have approved of for its precision. A typical Los Santos street visible from the player's position contained: 15 buildings (average 1,500 triangles = 22,500), 8 cars (average 1,200 = 9,600), 20 NPCs (1,200 = 24,000), 30 palm trees and trees (400 = 12,000), 40 assorted props (signs, lampposts, benches, trash = 3,000), and the ground and the sidewalks (2,000). Total: 73,100 triangles. For a single visible street.

The player didn't see a single street. From an elevated position in Hollywood Hills, the player saw dozens of streets extending to the horizon. If each street had 73,000 triangles and there were 30 visible streets, the total was 2.2 million triangles. Eleven times more than the PC budget. Twenty-seven times more than the PS2's.

The math was impossible. Unless most of those triangles didn't exist.

*

George implemented LOD: Level of Detail. Each 3D model in the game existed in three or four versions with a different number of polygons.

The high version was the complete model: all the details, all the edges, all the curved surfaces represented with enough triangles for the model to look smooth up close. A car at high LOD had 2,000 triangles with doors, side mirrors, detailed bumpers, individually modeled hubcaps, and windows with thickness.

The medium version eliminated the details that were invisible at 50 meters: the mirrors became flat rectangles, the hubcaps simple discs, the windows lost their thickness and became planes. The car at medium LOD had 600 triangles. It looked fine at medium distance. Up close it looked like a toy.

The low version eliminated everything except the recognizable silhouette: the car was a box with four cylinders that were the wheels. 120 triangles. At 200 meters, on a 640x480 screen, it took up less than 20 pixels wide. At 20 pixels, the difference between 2,000 triangles and 120 was imperceptible. The brain saw "car" in both cases.

Each model in the game needed these three versions. Marcus and the art team couldn't model three versions of each of the 500 types of vehicles, 200 NPC models, and 3,000 buildings. They would have needed an army. So George built a tool.

The LOD generator was a script that took the high-resolution model and automatically produced the medium and low versions by reducing the number of polygons with a mesh simplification algorithm. The algorithm evaluated each edge of the model and calculated how much the silhouette would change if that edge were eliminated by merging the two vertices it connected. The edges that least affected the silhouette were eliminated first. The process continued until reaching the target number of triangles.

The result had flaws: the algorithm sometimes eliminated edges that were visually important and left others that weren't. Marcus reviewed the generated LODs of the most important models (main vehicles, iconic buildings) and adjusted them by hand. The less important models (trash cans, traffic signs, benches) used the automatic LODs without review.

"I'm modeling every car three times," Marcus complained one afternoon, adjusting the medium LOD of a lowrider the generator had turned into something Marcus described as "a bathtub with wheels."

"You're modeling the good car once and adjusting the cheap versions," George corrected. "Without the generator you'd be modeling from scratch. With the generator you adjust. It's the difference between cooking and reheating."

"If I'm the chef, your generator is a microwave with opinions."

*

The most visible problem of the LOD was the transition between versions.

When the player approached a building that was at low LOD, the engine needed to replace it with the medium LOD. And when he got closer, with the high LOD. If the transition was instantaneous, the building changed shape from one frame to the next: from a simple box to a model with windows and cornices and facade details. The change was visible and distracting, a pop that broke the illusion that the building had always been there with all its details.

George implemented cross-fading: during 8 frames (a quarter of a second), both versions of the model were rendered simultaneously with inverse opacities. The low LOD started at 100% opacity and dropped to 0%. The high LOD started at 0% and rose to 100%. During those 8 frames, both models overlapped and the eye saw a smooth transition where the details appeared gradually instead of instantly.

The cross-fading had a cost: during 8 frames, the renderer drew two versions of the same model, which doubled the number of triangles of that object. If ten buildings cross-faded at the same time, the renderer had to draw all 10 at low LOD AND at high LOD simultaneously, which could consume the polygon budget.

George limited the cross-fading to a maximum of 5 simultaneous objects per frame. If more than 5 objects needed to transition at the same time (which happened when the player drove fast and the buildings changed LOD in bursts), the additional objects made an instantaneous transition with no cross-fade. It was a compromise: at high speed, the player could see some pops, but the framerate stayed stable.

Daniel suggested an elegant optimization: instead of limiting by number of objects, limit by number of additional triangles. If the cross-fade budget was 5,000 triangles per frame, a 3,000-triangle building and a 2,000-triangle car could transition simultaneously, but a second building would have to wait or pop. It was more granular than the limit by objects and produced better results in practice.

"Five thousand cross-fade triangles," George said. "It's two and a half percent of the PC budget. Enough for the transitions to be smooth without affecting the framerate."

"On PS2 the cross-fade budget is a thousand," Daniel said. "Which means that on PS2, almost everything pops."

"On PS2 the player has fog at 800 meters. Most of the LOD transitions happen inside the fog. The player doesn't see them."

"Except when he drives fast and the buildings come out of the fog popping."

"Yes. Except then."

Daniel looked at George. He knew that the weakest platform always pays the highest price, and that accepting that price was part of the job.

*

Beyond the LODs, at distances where even the simplest model was a waste, were the impostors.

An impostor was the most blatant lie of the 3D engine: a 3D object replaced by a 2D image. A sprite. A flat billboard that always faced the camera and that showed a photo of the object rendered from the approximate angle of the player.

George used impostors for everything beyond 400 meters on PC, 200 on Xbox, and 100 on PS2. At those distances, a 1,500-triangle building took up less than 10 pixels on screen. Rendering 1,500 triangles for 10 pixels was an absurd waste. An impostor of 2 triangles (a rectangle) with a 32x32-pixel texture produced the same visual result at a fraction of the cost.

The challenge of the impostors was rotation. A 3D model looked different from each angle. An impostor was a fixed photo from one angle. If the player moved around an impostor building, the photo didn't change perspective, which gave away that it was flat.

George implemented impostors with 8 angles: each object had 8 pre-rendered photos, one every 45 degrees. The engine selected the photo whose angle was closest to the player's current angle. When the player moved enough that another angle was closer, the impostor changed photo with a 4-frame cross-fade.

For the trees and the palm trees, 8 angles weren't necessary because a tree seen from the front looked almost the same as seen from the side. George used 2-angle impostors for the trees: front and profile. The difference was invisible because the trees were organic and irregular: they didn't have the symmetrical geometry of a building that gave away the lack of perspective.

The result was that a panoramic view of Los Santos from Hollywood Hills showed hundreds of buildings, thousands of trees, and tens of thousands of details. And of all those objects, 90% were impostors: flat rectangles with photos that faced the player like sunflowers that follow the sun. The player saw a city. The engine saw a facade: a real street with real buildings surrounded by a cardboard set that was replaced by real buildings when the player got close.

*

Marcus hated the impostors for exactly one week.

The first time George showed him how they worked, Marcus looked at a test street where the buildings in the background were impostors and the ones in the foreground were 3D models. He turned the camera slowly. The impostors changed angle with their 4-frame cross-fade. And Marcus made a face George knew: that of an artist who sees his work reduced to something he finds unworthy.

"My buildings are sprites," Marcus said. "Sprites. In 2005. I'm making sprites again."

"They aren't sprites. They're 2D representations of your 3D models generated dynamically according to the camera angle."

"They're sprites with perfume."

"They're the reason the player can see the whole city from a hill without the framerate dropping below 30. Without impostors, the skyline of Los Santos would have 15 million triangles. With impostors, it has 200,000. Your 3D models are there. They just become photos of themselves at a distance."

Marcus didn't respond. He went to his desk. And for a week, he didn't mention the impostors.

Then he came back with a proposal.

"The impostors need art direction," Marcus said, putting a three-page document on George's desk. "If the impostors are photos of my buildings, the photos have to be good photos. I need to control the lighting of the impostors, the angle of each capture, and the resolution of the textures. A building that looks mediocre as an impostor at 400 meters ruins the skyline more than a building that isn't there."

George read the document. Marcus proposed an impostor quality pipeline: each iconic building of Los Santos (Downtown skyscrapers, the Hollywood mansions, the East LA churches) would have manually rendered impostors with sunset lighting (the most iconic moment of the city) at a resolution of 64x64 instead of 32x32. The generic buildings would use automatic impostors. The trees would use impostors hand-painted by Priya with the watercolor technique they had invented for Hollow Knight.

George approved the pipeline. The cost was more production time for the manual impostors. The benefit was that the skyline of Los Santos, the first image the player saw when going up to any elevated point, was going to be beautiful even being composed 90% of flat rectangles with photos.

"The Castlevania sprites were art," Marcus said when George approved the document. "The Isaac sprites were art. The Hollow Knight tiles were art. The Los Santos impostors are going to be art. I don't care that they're 2D. I care that they're good."

It was the statement of an artist who had made peace with a technical limitation by mastering it. The impostors were going to be another form of art, not merely a necessary compromise.

*

George built the first panoramic view of Los Santos at the end of May.

Not the real view with the buildings and the textures. Those didn't exist yet. The test view: colored cubes in the nearby chunks as 3D models, colored cubes farther away as medium LODs, colored cubes in the distance as low LODs, and beyond 400 meters, gray impostors that were flat rectangles with the silhouette of a cube painted on their texture.

George put the camera at an elevated point that corresponded to the future position of Hollywood Hills. He looked south and west. The view extended to the edge of the map: cubes in the foreground with shadows from the lighting system, medium cubes with no shadows at middle distance, flat silhouettes of cubes cutting against the sky in the background.

It was horrible and brilliant at the same time. Horrible because they were cubes. Brilliant because it worked. The renderer processed the whole scene at 45 fps on PC. The LODs transitioned with smooth cross-fade. The impostors rotated with the camera. The distance fog (which George had implemented as a gradual fade of the objects' color toward the color of the sky) blended the farthest impostors with the horizon so that the transition between object and impostor and between impostor and sky was a continuous gradient.

There were no seams. There were no edges where the player could point and say "that's where the 3D becomes 2D" or "that's where the real models become a set." The transition was gradual, continuous, invisible if you didn't know exactly what to look for.

George called Daniel. Daniel stood behind George and looked at the view of cubes.

"How many triangles?" Daniel asked.

"186,000 in frame. Within the PC budget with margin."

"How many objects are impostors?"

"Of the 340 visible objects, 280 are impostors. Eighty-two percent."

"Eighty-two percent of the view is cardboard cutouts and it looks like a city."

"That's the job. Making the cardboard look like a city."

Daniel looked at the view a moment longer. The cubes that were buildings. The rectangles that were impostors of cubes. The sky that was a gradient. The fog that blended it all. And beneath the ugliness of the placeholders, the structure of a trick that was going to fool millions of players into believing they were looking at a whole city when in reality they were looking at a street surrounded by cardboard sets that were replaced by real city when no one was looking.

"When Marcus puts his buildings in this," Daniel said, "it's going to be spectacular."

"I know."

"And when Lena puts the sound of the city underneath."

"I know."

"And when the sun of your lighting system paints everything orange at sunset."

"I know, Daniel."

Daniel nodded. He didn't need to say more. The two of them knew what they were looking at: the bare structure of something that was going to be beautiful. The skeleton of a city that didn't have skin or muscles or clothes yet but that already stood on its feet.

George closed the panoramic view. The next system on the list was occlusion culling: teaching the engine not to render what was behind a building. Every triangle saved was a triangle that could be spent on visible detail. And in a game where 82% of the view was cardboard, every real triangle counted.

End of Chapter 91

Comments

No comments found for this post.