WorstUsernameEver
But best title ever!
Should you have played a game from Bethesda this generation you'll probably be familiar with slowdowns related to the increasing size of save-game files, a problem which wasn't fixed even in the recently published Skyrim. Since the problem was present in New Vegas too, project director J.E. Sawyer from Obsidian took some time to answer to some questions about it on his Formspring account, and explained why the bug couldn't be fixed in a patch:<blockquote>Question from one of lesser knowledge... Is there a save game size that you would call "normal" for F:NV? Mine is 14mb, and i'm having intense lag in multiple sections of the Mojave/ all DLC lands. Just curious if that is/could be an issue!
That can easily be a big problem, especially if you're on the PS3. The longer you play a character, the more bit differences on objects (characters, pencils on tables, containers, etc.) get saved off and carried around in memory. I think we've seen save games that are pushing 19 megs, which can be really crippling in some areas.
...
If it's an issue why hasn't it been resolved? The pace at which game sellers basically dish an alpha or beta game to the marketplace is sickening. New V stopped being "beta" in mid-2011. I know as a dev my workplace wasn't AAA dishing but we had integrity.
Since you're a developer, you should understand the implication of what I wrote. It's an engine-level issue with how the save game data is stored off as bit flag differences compared to the placed instances in the main .esm + DLC .esms. As the game modifies any placed instance of an object, those changes are stored off into what is essentially another .esm. When you load the save game, you're loading all of those differences into resident memory.
It's not like someone wrote a function and put a decimal point in the wrong place or declared something as a float when it should have been an int. We're talking about how the engine fundamentally saves off and references data at run time. Restructuring how that works would require a large time commitment. Obsidian also only had that engine for a total of 18 months prior to F:NV being released, which is a relatively short time to understand all of the details of how the technology works.
...
Hey Josh, different user, but same kind of 'lesser knowledge' bloke. What's a bit difference and how does it determine a save file?
http://en.wikipedia.org/wiki/Bit_field
It's a compact way to store data. The bit differences in this case are just flags set up to mark what data has changed (i.e., are different) from what's in the core .esm.
Let's say that I, as a designer, set up a creature in an area. I set all of the character's statistics and gear and save it in the master FalloutNV.esm file that gets loaded into the game. You, the player, run through the area and shoot that dude. You loot him of his gear and put a shovel in his inventory because you are wacky.
The game needs a way to mark that his a) position B) health c) inventory d) some other stuff has changed on him. It does that by marking what fields have changed (by setting individual bits) and then indexing the individual (changed) values for reference later.
When you load the save game, it loads up all of the bit fields marking changes in your save game. When the individual objects load, it applies the indexed changes to those objects. That way, when you come back to the area you left two nights ago, the character is still sprawled out where you left him, naked, with a shovel in his inventory.
Individual bits of data are tiny, but there are thousands upon thousands of objects in F:NV, each one containing numerous data fields that could potentially be changed in your save game. Over time, it adds up.
...
Is the inflating save file just an issue for the PS3 (I've seen lots of lag/crash complaints from PS3 users) or does it happen on all platforms? I'm just wondering if other platforms handle it better than the PS3.
As with Fallout 3 and Skyrim, the problems are most pronounced on the PS3 because the PS3 has a divided memory pool.
...
(Same fellow you just answered) So....basically, every time I manipulate an object, it fluctuates the save file up or down? I've noticed files can get rather huge, is there no way this could have been greatly diminished in a game as big as New Vegas?
It almost always goes up. Some areas will reset contents after three (game) days, but a lot of stuff lingers. Additionally, we also have to deal with "persistent references". These are objects that are immediately loaded with the game because we need to be able to reference them anywhere/everywhere in the world -- even if the player is nowhere near the object. Characters are the most common example. All of the companions need to be able to move around the world even when they are not in your current area, so they are all persistent references.
All object data (excluding art assets like .nifs and audio assets [VO]) for persistent references is loaded at all times, so that's more-or-less a permanent chunk of resident memory. The number of persistent references invariably goes up with each DLC, so as the number of DLCs increases, the system has less and less memory available. Of course, the player's save game file only gets bigger and bigger, since he or she is going through more or more areas manipulating an increasingly large number of objects.
This is why some of our later patches actually removed content from the core game (e.g. Primm). Even though we had balanced the memory footprint for the core game, DLC content was pushing down the available resources.
...
"divided memory pool"?
The Xbox 360 has a unified memory pool: 512 megs of RAM usable as system memory or graphics memory. The PS3 has a divided memory pool: 256 megs for system, 256 for graphics. It's the same total amount of memory, but not as flexible for a developer to make use of.
...
But why did the patch remove content from the PC? Most PCs nowadays have 2-4GB of RAM, plus 500MB-1GB of GPU memory. So I doubt the DLCs negatively impacted most PCs to justify content removal...
If we had generated .esms per-platform, that would have been a crazy nightmare for a lot of reasons. A slightly less risky approach could have been to script the removal of assets using the IsPC/IsXbox/IsPS3 functions, but that also introduces its own host of potential problems, especially if objects are attempting to reference something as the script removes it.
We ran into a small but non-trivial number of crashes in F:NV involving persistent references attempting to interact with an object as the player transitions out of his or her current area. E.g. Chief Hanlon attempts to sit in a chair. The player leaves the area, the chair Hanlon wants to sit in is unloaded, and the game crashes.
...
Hypothetically, if I were to play through FNV without opening any non-necessary containers (e.g. never opening an ammo case) or touching any non-necessary objects (e.g. never moving a pencil) would the game be more stable?
By some margin, yes, but randomized loot I believe is generated on the area's first load, which also applies to the equipment in many characters' (e.g. Fiends') inventories.</blockquote>Thanks GameBanshee.
That can easily be a big problem, especially if you're on the PS3. The longer you play a character, the more bit differences on objects (characters, pencils on tables, containers, etc.) get saved off and carried around in memory. I think we've seen save games that are pushing 19 megs, which can be really crippling in some areas.
...
If it's an issue why hasn't it been resolved? The pace at which game sellers basically dish an alpha or beta game to the marketplace is sickening. New V stopped being "beta" in mid-2011. I know as a dev my workplace wasn't AAA dishing but we had integrity.
Since you're a developer, you should understand the implication of what I wrote. It's an engine-level issue with how the save game data is stored off as bit flag differences compared to the placed instances in the main .esm + DLC .esms. As the game modifies any placed instance of an object, those changes are stored off into what is essentially another .esm. When you load the save game, you're loading all of those differences into resident memory.
It's not like someone wrote a function and put a decimal point in the wrong place or declared something as a float when it should have been an int. We're talking about how the engine fundamentally saves off and references data at run time. Restructuring how that works would require a large time commitment. Obsidian also only had that engine for a total of 18 months prior to F:NV being released, which is a relatively short time to understand all of the details of how the technology works.
...
Hey Josh, different user, but same kind of 'lesser knowledge' bloke. What's a bit difference and how does it determine a save file?
http://en.wikipedia.org/wiki/Bit_field
It's a compact way to store data. The bit differences in this case are just flags set up to mark what data has changed (i.e., are different) from what's in the core .esm.
Let's say that I, as a designer, set up a creature in an area. I set all of the character's statistics and gear and save it in the master FalloutNV.esm file that gets loaded into the game. You, the player, run through the area and shoot that dude. You loot him of his gear and put a shovel in his inventory because you are wacky.
The game needs a way to mark that his a) position B) health c) inventory d) some other stuff has changed on him. It does that by marking what fields have changed (by setting individual bits) and then indexing the individual (changed) values for reference later.
When you load the save game, it loads up all of the bit fields marking changes in your save game. When the individual objects load, it applies the indexed changes to those objects. That way, when you come back to the area you left two nights ago, the character is still sprawled out where you left him, naked, with a shovel in his inventory.
Individual bits of data are tiny, but there are thousands upon thousands of objects in F:NV, each one containing numerous data fields that could potentially be changed in your save game. Over time, it adds up.
...
Is the inflating save file just an issue for the PS3 (I've seen lots of lag/crash complaints from PS3 users) or does it happen on all platforms? I'm just wondering if other platforms handle it better than the PS3.
As with Fallout 3 and Skyrim, the problems are most pronounced on the PS3 because the PS3 has a divided memory pool.
...
(Same fellow you just answered) So....basically, every time I manipulate an object, it fluctuates the save file up or down? I've noticed files can get rather huge, is there no way this could have been greatly diminished in a game as big as New Vegas?
It almost always goes up. Some areas will reset contents after three (game) days, but a lot of stuff lingers. Additionally, we also have to deal with "persistent references". These are objects that are immediately loaded with the game because we need to be able to reference them anywhere/everywhere in the world -- even if the player is nowhere near the object. Characters are the most common example. All of the companions need to be able to move around the world even when they are not in your current area, so they are all persistent references.
All object data (excluding art assets like .nifs and audio assets [VO]) for persistent references is loaded at all times, so that's more-or-less a permanent chunk of resident memory. The number of persistent references invariably goes up with each DLC, so as the number of DLCs increases, the system has less and less memory available. Of course, the player's save game file only gets bigger and bigger, since he or she is going through more or more areas manipulating an increasingly large number of objects.
This is why some of our later patches actually removed content from the core game (e.g. Primm). Even though we had balanced the memory footprint for the core game, DLC content was pushing down the available resources.
...
"divided memory pool"?
The Xbox 360 has a unified memory pool: 512 megs of RAM usable as system memory or graphics memory. The PS3 has a divided memory pool: 256 megs for system, 256 for graphics. It's the same total amount of memory, but not as flexible for a developer to make use of.
...
But why did the patch remove content from the PC? Most PCs nowadays have 2-4GB of RAM, plus 500MB-1GB of GPU memory. So I doubt the DLCs negatively impacted most PCs to justify content removal...
If we had generated .esms per-platform, that would have been a crazy nightmare for a lot of reasons. A slightly less risky approach could have been to script the removal of assets using the IsPC/IsXbox/IsPS3 functions, but that also introduces its own host of potential problems, especially if objects are attempting to reference something as the script removes it.
We ran into a small but non-trivial number of crashes in F:NV involving persistent references attempting to interact with an object as the player transitions out of his or her current area. E.g. Chief Hanlon attempts to sit in a chair. The player leaves the area, the chair Hanlon wants to sit in is unloaded, and the game crashes.
...
Hypothetically, if I were to play through FNV without opening any non-necessary containers (e.g. never opening an ammo case) or touching any non-necessary objects (e.g. never moving a pencil) would the game be more stable?
By some margin, yes, but randomized loot I believe is generated on the area's first load, which also applies to the equipment in many characters' (e.g. Fiends') inventories.</blockquote>Thanks GameBanshee.
