Home Creators Posts Import Register Favorites Logout

Content

CHAPTER 90: Streaming

Los Santos had thirty-six square kilometers of city.

Thirty-six square kilometers of buildings, streets, sidewalks, lampposts, traffic signs, parked cars, trees, power poles, fire hydrants, benches, trash cans, murals painted on walls, cracks in the asphalt, puddles of dirty water, beer cans in the gutters, and every other detail that made a city look like a city and not like a collection of gray boxes floating in the void.

Thirty-six square kilometers of geometry, textures, collision data, NPC spawn points, traffic routes, gang zones, and the audio information that told NGN which sounds to play at each point of the map.

None of that fit in memory.

A high-end PC in 2005 had 1 gigabyte of RAM. The PS2 had 32 megabytes. The Xbox had 64. And Los Santos, with all its uncompressed assets, was going to take up more than 2 gigabytes. Twice the RAM of a PC. Sixty-four times the RAM of a PS2.

The solution was streaming: load only what the player could see and unload what he could no longer see, in real time, without the player noticing the process. The player had to believe that the whole city existed at the same time, that Los Santos was a continuous space with no seams, no loading screens, no interruptions. The reality was that at any given moment, only a fraction of the city existed in memory, and that fraction moved with the player like a spotlight that lit what it had nearby and left everything else in the darkness of the unloaded data.

Daniel led the streaming module. But the architecture was George's, because George knew exactly how it should work. He had seen it work. In engines that didn't exist yet.

*

George divided the map of Los Santos into chunks.

A chunk was a square of 384 by 384 meters of the map. It contained all the information of that section: the geometry of the buildings and the streets, the textures of the surfaces, the collision data that determined where the player could walk and what he collided with, the spawn points of NPCs and vehicles, the ambient audio information, and the triggers of missions and events. Each chunk was an autonomous unit that could be loaded and unloaded independently of the others.

The complete Los Santos was divided into 256 chunks arranged in a 16-by-16 grid. Each chunk had a size on disk of between 15 and 40 megabytes depending on the density: the Downtown chunks with their skyscrapers had more geometry than those of the Hollywood hills with their scattered houses. In memory, a chunk took up between 8 and 20 megabytes after decompressing the textures and preparing the geometry for the renderer.

The player needed to see the nearby chunks. George defined a load radius of 3 chunks in each direction: the chunk where the player was plus all the adjacent chunks in a 7-by-7 square, which added up to 49 potential chunks in a zone of 2.7 square kilometers but which in practice were fewer because the chunks at the edge of the map didn't exist. The actual load radius depended on the platform: on PC with 1GB of RAM, 49 chunks was viable because 49 times 10 megabytes average gave 490 megabytes. On PS2 with 32 megabytes, only 3 to 5 chunks could be in memory simultaneously.

"On PS2 the player is only going to see the immediate," Daniel said, doing the math. "Three chunks are 1152 meters of visibility. Everything else has to be behind fog."

"Correct. PS2 has fog at 1200 meters. Xbox at 1,800. PC at 3,000 or no fog if the hardware allows it."

"The difference between versions is going to be enormous."

"I know. And the PS2 version is still an open world with no loading screens. The fact that it has fog doesn't change that the player can drive from Grove Street to Hollywood Hills without a single loading screen. That's what matters."

*

The streaming ran on an execution thread separate from the game's main thread.

George explained the concept to the team with an analogy Elena understood better than anyone: "Imagine you're reading a book and someone turns the pages for you. You read the page in front of you. Meanwhile, another person prepares the next page so that when you finish reading the current one, the next is already ready. You never wait. You never see a blank page. Because someone is working behind the scenes so that the next page is always ready before you need it."

"The main thread is the reader," Elena said. "The loading thread is the one turning the pages."

"Exactly. The main thread renders, runs the physics, updates the AI, processes the player's input. It's busy at a hundred percent of its capacity producing 30 frames per second. It doesn't have time to go to the disk, read a 40-megabyte chunk, decompress it, and prepare it for the renderer. That takes hundreds of milliseconds, and each frame has a budget of 33 milliseconds. If the main thread stops to load a chunk, the game freezes."

"So the loading thread reads from the disk while the main thread renders," Daniel said.

"Yes. The loading thread reads the data from the disk into a buffer in memory. When the buffer is complete, it decompresses the textures and prepares the geometry. When everything is ready, it passes the data to the main thread in a package the main thread can integrate in a single frame without freezing. The transfer from the loading thread to the main thread has to be instantaneous: a swap of pointers, not a copy of data."

Daniel drew the flow in his notebook: main thread running the game, loading thread reading disk in parallel, data transferred when ready, old chunk unloaded when the player moved away. It was a pipeline that required precise synchronization: if the loading thread didn't finish before the player reached the edge of the chunk, there would be a visible void. If it finished too soon, the data took up memory needlessly.

The timing had to be perfect. And for it to be perfect, George needed prediction.

*

The streaming system couldn't wait for the player to reach the edge of a chunk to start loading the next. It had to anticipate.

George implemented a movement-prediction algorithm that analyzed the position, direction, and velocity of the player every frame and calculated which chunk he was going to be in within a number of seconds. The algorithm looked at the player's trajectory as a line that extended into the future and marked the chunks that line crossed as load candidates.

On foot, the player moved at 5 meters per second at most. A 384-meter chunk took 77 seconds to cross on foot. George could start loading the next chunk 45 seconds in advance and have margin to spare.

On a bicycle, 10 meters per second. 38 seconds per chunk. 20 seconds of anticipation were enough.

In a car, things changed. A muscle car in a straight line reached 60 meters per second. 6.4 seconds per chunk. If George waited for the player to be a chunk away to start the load, he had 6.4 seconds to read 40 megabytes from the disk, decompress them, and prepare them. On PC with a hard drive, 6.4 seconds was manageable: a 2005 hard drive read at 30-60 megabytes per second. On PS2 with DVD, 6.4 seconds was still tight: the DVD read at 3-5 megabytes per second.

The solution was to anticipate more. Much more. In a vehicle at high speed, NGN's streaming system loaded chunks 3 in advance in the direction of movement. Three chunks ahead meant 1152 meters of margin, which at 60 meters per second was 19.2 seconds. Nineteen seconds were enough even for the PS2's DVD.

But there was a cost: loading 3 chunks ahead meant the memory had to hold not only the visible chunks but also the anticipated chunks that weren't visible yet. On PC, this was manageable. On PS2, with 32 megabytes of RAM, loading 3 anticipated chunks in addition to the 3-5 visible ones was impossible.

Daniel solved the problem with a technique he called partial loading: on PS2, the anticipated chunks weren't loaded complete. Only the geometry and the collision data were loaded, which were the most critical (without geometry, the player saw a void; without collision, the player fell through the ground). The textures were loaded afterward, when the chunk went from anticipated to visible, with a priority that loaded first the textures of the ground and the facades of the nearby buildings and last the details like traffic signs and trash cans.

The result was that on PS2, arriving at a new chunk at high speed, the buildings appeared first with low-resolution textures that were replaced by the final textures in 1-2 seconds. It was a visible compromise the PS2 players were going to notice but that was infinitely preferable to a loading screen.

"Texture pop-in," Daniel said. "It isn't pretty."

"It's the price of an open world in 32 megabytes of RAM," George said. "And it's better than loading screens."

"Anything is better than loading screens."

*

George tested the streaming for the first time at the end of April with a scene that was deliberately cruel: a car accelerating in a straight line at maximum speed down a long road that crossed eight consecutive chunks without stopping.

The test scene had no real buildings or Marcus's textures or NPCs. Just colored cubes that represented buildings, a gray plane that was the road, and green markers at the edges of each chunk that told George where one ended and another began. The car was a box with wheels that accelerated to 60 meters per second.

George got in the car. He accelerated. The box with wheels shot down the gray road.

First chunk: colored cubes on both sides of the road. The world existed.

Second chunk: the cubes of the first chunk started to disappear behind the camera while the cubes of the second appeared ahead. The transition was invisible. The cubes of the third chunk, anticipated by the prediction system, were already loaded and waiting.

Third chunk. Fourth. Fifth. The box with wheels crossed chunks at the rate of one every four seconds and the world was built ahead and destroyed behind with the fluidity of a river that flows in one direction and that carries the player with the current.

Sixth chunk. Seventh. George watched the debug counters that floated in the upper left corner of the screen: memory usage, loading time of the secondary thread, number of chunks in memory, framerate. All green. All within the parameters.

Eighth chunk. The box with wheels braked at the end of the test road. George had crossed three kilometers of virtual space in thirty seconds with not a single frame of loading, with not a single moment where the world stopped existing, with not a single visible seam between chunks.

Daniel was watching from his desk.

"It works," George said.

"On PC," Daniel confirmed. "Tomorrow I test it on PS2 and tell you how many things I have to break."

*

On PS2, Daniel broke a lot of things.

The PS2's DVD read at 3.2 megabytes per second in the best case and at 1.5 in the worst, depending on whether the data was on the outer or inner edge of the disc. A compressed chunk of 15 megabytes took between 5 and 10 seconds to read from the DVD. At 60 meters per second, the player crossed a chunk in 4 seconds. The math didn't close: the disc was slower than the player.

Daniel implemented three solutions that worked together.

The first was the data layout on the disc. Daniel organized the chunks on the DVD so that adjacent chunks were physically adjacent on the disc. On a normal DVD, the files were written wherever there was space. Daniel wrote a custom tool that organized the disc layout so the read head didn't have to jump from one end to the other: if the player went from chunk A to chunk B, the data of chunk B was right after that of chunk A on the disc. The seek time was reduced from milliseconds to microseconds.

The second was the aggressive compression. The PS2 chunks used textures compressed to 50% of the original size with a codec George had implemented based on lossy compression techniques that sacrificed detail in the high frequencies of the textures. The high frequencies were the fine details that were only seen up close: microscopic cracks in the asphalt, the grain of a door's wood, the small letters of a sign.

On PS2, with a resolution of 640x448 and a draw distance of 800 meters, those details were invisible anyway. The compression reduced the size of the chunks from 15 megabytes to 8, which reduced the read time proportionally.

The third was the read-ahead buffer. Daniel reserved 4 megabytes of the PS2's 32 exclusively for the streaming buffer. It was 12% of the total RAM dedicated to data in transit between the disk and the memory. It was an enormous sacrifice: those 4 megabytes couldn't be used for textures or for geometry or for anything else. But the buffer allowed the loading thread to read data from the disk continuously, with no waits, filling the buffer with the chunks the prediction system anticipated. When the main thread needed a chunk, the data was already in the buffer waiting.

The three solutions combined reduced the effective load time of a PS2 chunk from 10 seconds to 3. Three seconds at maximum car speed meant the prediction system needed to anticipate a chunk and a half. With the partial loading (geometry first, textures after), the system worked. Barely. With the texture pop-in Daniel had accepted as an inevitable cost.

Daniel ran the car test on PS2 one Friday at eight at night. The box with wheels accelerated down the road of cubes. The chunks loaded ahead and unloaded behind. On the third chunk, the ground textures took a second to appear: during that second, the ground was a gray plane with no detail that filled with texture like a Polaroid photo developing.

On the fifth chunk, a cube representing a building appeared with 2 frames of delay after its collision geometry was already loaded: the car didn't pass through it (the collision worked) but the cube was invisible for 66 milliseconds.

Daniel noted both problems. He adjusted the load priorities. He tested again. The ground pop-in was reduced to half a second. The invisible cube disappeared because Daniel added a frame of margin to the transfer of visible geometry.

"On PC it's transparent," Daniel said. "On Xbox it's almost transparent. On PS2 it's visible if you run at maximum speed, which is exactly what the players are going to do all the time."

"Is it playable?"

"It's playable. It isn't perfect. But it's an open world in 32 megabytes with DVD, and that's an engineering miracle even if it has pop-in."

"Then it's enough."

"For PS2, yes. Enough is the most PS2 can give."

*

George drove the box with wheels down the test road one last time. This time he didn't look at the debug counters. He didn't look at the edges of the chunks. He didn't look at the memory or the framerate or the loading time of the secondary thread.

He looked at the road. The cubes that passed on the sides. The horizon that extended ahead with more cubes loading in silence, appearing where before there was nothing, building a world that only existed because the player was there to see it.

There was no loading screen. There was no pause. There was not a single moment where the illusion broke and the player remembered he was inside a computer program instead of inside a city. The road continued. The cubes appeared. The world existed around the player as if it had always existed, as if the buildings that had just been loaded from the disk had been there from the beginning and as if the ones that had just been unloaded would still be there when the player wasn't looking.

It was a lie. The fundamental lie of every open world: that the world exists complete when in reality only the fragment the player can see exists. But it was a lie so well told that it felt like truth. And in games, the lies that feel like truth are the only ones that matter.

George parked the box with wheels at the end of the road. Three kilometers of continuous space, built and destroyed in real time, with not a single visible seam.

Hollow Knight had separate rooms with fade transitions. Castlevania had zones that loaded when the player crossed a door. Isaac had floors that were generated complete before starting.

Los Santos was going to have none of that. Los Santos was going to be continuous. A city with no doors, no fades, no loading screens. A space that existed because the player moved through it and that stopped existing when he moved away, but that never, at any moment, left the player alone on an empty screen waiting for the world to be built.

The streaming of NGN worked. And everything that came after depended on it continuing to work: every building of Marcus, every NPC of the AI, every car of the physics, every note of Lena, everything was going to be loaded and unloaded by this system, in real time, without the player knowing it.

It was the most important invisible system in the engine.

End of Chapter 90

Comments

No comments found for this post.