Hex patch for color character portraits

[Melindil]

In this thread on fixing the Team Player/Loner perks, a request was made to look into allowing character portraits to be displayed in full color, rather than the green-scanline effect of the Pipboy. I think I have it working:

The hex changes to apply are below (be sure to make a backup of the BOS.EXE file before editing - if something goes wrong, it shouldn't corrupt any saves, but it could crash more frequently). Changes can be applied individually if desired.

Change 1: Allows selected character portrait to be in color:

000bc8c4: D8 0D 98 D5 7F 00 D9 05 9C DE 8B 00 D8 0D 98 D5 7F 00 8B 86 3F 01 00 00
-> D9 05 A0 DE 8B 00 D9 05 9C DE 8B 00 8B 86 3F 01 00 00 8D 4D C4 51 EB 45

Change 2: For unselected character portraits (in the character select and recruits list), all color channels will be dimmed equally (existing algorithm is biased to a greenish-gray):

000bc93b: D9 05 3C DE 8B 00 D8 0D 84 D7 7F 00 8B 86 3F 01 00 00 8D 55 C4 52
-> D8 05 98 D5 7F 00 D9 C0 D9 C0 8B 86 3F 01 00 00 8D 55 C4 52 EB 41

Change 3: Eliminate the scanline effect in the portrait

00345eb4: 74 0C
-> 90 90

Caveats:

• For best results, once an image is cropped to the right size, use an image editor to convert it to 8-bit indexed color, then back to RGB. This is necessary because the game engine will perform this conversion itself (details below), and you will probably get better quality from doing this in a proper image editor.
• Portrait size is still 75x100. It might be possible to increase this, but given the requirement for indexed color, higher-resolution images likely won't provide much benefit in quality.
• Existing portraits in-game are all grayscale, and will be displayed as such (see the last two portraits in the screenshot). Only newly added character portraits will be shown in color.
  
• Turning off the scanline effect for portraits also disables it for Pipboy displays which use it (e.g. the inventory screen has an image of the character sprite(s)).
• Image display for the (big) prefab character screen still appears to be green. I'm surprised - this means there's a second set of code just for those images. If there's enough request, I can try to track this one down too. The portrait display inside the customization window is correct, though.

Patch details:

At a high level, the code does the following when a character is selected from the Pipboy display:

• Load the corresponding ZAR if it's not already in memory
• Decompress the ZAR into a 32-bit ARGB image
• Copy the image a couple of times - one of these resizes it for the location being displayed. It's likely one of these also crops the image if it's too large
• Generate a 256-color palette from colors in the image, and convert the image data into 8-bit indexed using that palette (passing through a 17-bit (!!!) color lookup table in the process)
• Copy the image and palette a few more times (never did track down what these did - just that they kept interfering with memory breakpoints in the debugger)
  
• Tweak the colors of the palette based on the desired image effect (in this case, applying the green or green-gray color bias)
• As needed, draw the image to the screen, converting back from indexed to RGB, and applying scanline effect if enabled

The first two changes above modify the color values for the palette adjustment - luckily, the code that calculates the RGB multiplier for portraits is separate from anything else, so I could modify that routine to just set all values to either 1.0 or 0.625 (for selected or unselected) without affecting anything else. (My first attempt accidentally changed the palette adjustment code itself - this resulted in all the text going white!)

The final change just disables the flag used to generate the scanline effect when copying an image bitmap to screen. This means that any bitmap which used the scanline effect will no longer do so. As far as I know, the only things which use this are the portraits, and the green character sprite in the inventory screen. If anyone sees that this has any other effect, please let me know.

 

[RobertHouse]

Wow! Absolutely awesome Hex work Melindil!

I’m not near as experienced as you are with editing in hex. I’ve had so many attempts at trying to create this exact hex edit (and so has many others) but without any luck, mostly game crashes or closing is the end results.

I haven’t been super active recently, really it’s so hard to find motivation to mod anymore. My life has never been more stressful or busy or as hard as it has been recently. I like to spend my free time with my family because it’s a rarity. But seeing these kinds of post really give me a lot of motivation and encouragement to continue developing on my own projects and mods.

Thank you for your contribution to the Fallout Tactics modding community.

 

[Melindil]

Thanks for the encouragement!

Every so often, I go on one of these wild threads, where I dig into some program (usually a game I like) to see how it works internally. This is really how I got started with programming in the first place, back when a high percentage of programs were in source-visible languages like BASIC (so you didn't have to do anything too crazy to see or change what a program was doing). Plus, it gives me a chance to practice in case I need these techniques for my day job (also coding).

I have one more project I'll be making a post on in a day or two, which has had very promising results. A teaser screenshot:

And I do want to make sure that I document, and post somewhere, the learnings and results of what I come up with - like everyone else, I will probably start to lose interest or have higher priorities for my time, so I want to leave a good starting point for whoever comes next.

 

[nadeauhugo]

OH MY MY WHAT DO WE HAVE HERE?... I am without words and it makes my day! A dream come true. I'll have so much work to do soon to implement all these portraits everywhere in my project, but it is really worth the time already! Any idea if that works for BIG portrait too?

I promised something and will be happy to deliver Melindil wow! I am sometimes searching hard for special locations ideas anyway and your statue kind of thing will be a fun implement to add somewhere in my mod and credits! Once again, congratulation. I'll post a thing or two in this tread before the end of june when I'll have time to go back on the working table and test all this. Yess yess yess.

 

[nadeauhugo]

Oh... send me the all white text HEX change too if you have time, I'll play with that a little to see if something good can come out of it.

 

[Melindil]

The scanline fix works for the big portrait, but the color one does not yet. I'll poke around for it, since I think I know what I'm looking for now - shouldn't be hard to find.

The white text bug (I'll call it that) is probably not good to keep on - I didn't play enough to see what other effects it had. Turning off the palette engine entirely might cause other anomalies. But again, since I now know how to find specific class objects (part of my other work), I should be able to find the specific palette that ties to text, and make it modifiable. Ideally, it would be an adjustable setting either for the hex patch, or for the Lua interpreter.

 

[nadeauhugo]

The scanline fix works for the big portrait, but the color one does not yet. I'll poke around for it, since I think I know what I'm looking for now - shouldn't be hard to find.

The white text bug (I'll call it that) is probably not good to keep on - I didn't play enough to see what other effects it had. Turning off the palette engine entirely might cause other anomalies. But again, since I now know how to find specific class objects (part of my other work), I should be able to find the specific palette that ties to text, and make it modifiable. Ideally, it would be an adjustable setting either for the hex patch, or for the Lua interpreter.

Allright! Just tell us when you found something and I'll test that out and post some screenshots of my new characters and/or interface texts...

 

[Melindil]

Color is working on the big prefab portraits now. Example:

Patch (this one is a bit lengthy, but just one location):
318ba8: d80d98d57f00d9059cde8b00d80d98d57f00c605b7e08b00018b86d700000083c0648b8e
-> d9c0d9c08d44e41cd95ce420d95ce424c605b7e08b0001d95ce41c508b8efa000000eb5f

Take note that the same caveats as for the previous fix apply. Also, the background on this screen is not perfectly black, so it may help to set alpha channel (transparency) to 0 for background pixels (this was done in the example).

I've also added this patch to the FOTExtender project (included with the existing color portrait patch section in the configuration JSON file). So once that is ready, this can be turned on or off without needing to poke around in the EXE.

 

[nadeauhugo]

Color is working on the big prefab portraits now. Example:

View attachment 10354
Patch (this one is a bit lengthy, but just one location):
318ba8: d80d98d57f00d9059cde8b00d80d98d57f00c605b7e08b00018b86d700000083c0648b8e
-> d9c0d9c08d44e41cd95ce420d95ce424c605b7e08b0001d95ce41c508b8efa000000eb5f

Take note that the same caveats as for the previous fix apply. Also, the background on this screen is not perfectly black, so it may help to set alpha channel (transparency) to 0 for background pixels (this was done in the example).

I've also added this patch to the FOTExtender project (included with the existing color portrait patch section in the configuration JSON file). So once that is ready, this can be turned on or off without needing to poke around in the EXE.

Incredible! Can't wait to play with all this. Say, will your extender work in conjonction with FTImprover in your idea? Thanks again!

 

[Melindil]

Incredible! Can't wait to play with all this. Say, will your extender work in conjonction with FTImprover in your idea? Thanks again!

Yes and no (and, well, yes).

I don't think there's anything in what FTImprover changes which will impact the extender I made, and vice versa. Plus, they both load slightly differently, so that shouldn't cause any issues. But this may change as I add more hooks - if FTImprover and my extender try to modify the same code location, then one could break the other.

That said, I've been toying with the idea of adding some (most?) of FTImprover's functionality to the extender - I don't like the idea of taking jarekfall's ideas, but without source we can't continue making improvements to it. I'll hold off on that until I get more input, or explicit permission to do so.

 

[nadeauhugo]

Oh that is great news! And I really don't think Jarekfall would mind having his work included and continued as long as we credit him for the amazing work that he did. And besides his ideas, if you are interested by some other challenges (Ah!) , I could talk about another that is dormant since quite a long time...

The other hack that would be a real changer would be to let armours overwrite some of the character's color layers, probably the first and fourth one. Maybe only the fourth one as the first one is also the color of the character's blood. That would make black leather jacket actually look black for example. But I don't know if that is even possible with HEX?

This is also also good news because FT Improver is a problem for me when attempting to create MAC versions of my mod. Wine GOG version that I use only launch BOS.exe, so I never figured out how to change that to FTImprover.exe. Will your version only transorm BOS.exe and be comptable with MAC GOG version? I'm asking way too much questions, It's just to tell you how much I am interested by what you are creating here ha!..

 

[Melindil]

Oh that is great news! And I really don't think Jarekfall would mind having his work included and continued as long as we credit him for the amazing work that he did.

I kinda thought this too, which is why I didn't rule it out entirely. And of course credit will be included for him on anything that FTImprover can help with (there's a lot of internal structures I haven't touched yet).

And besides his ideas, if you are interested by some other challenges (Ah!) , I could talk about another that is dormant since quite a long time...

The other hack that would be a real changer would be to let armours overwrite some of the character's color layers, probably the first and fourth one. Maybe only the fourth one as the first one is also the color of the character's blood. That would make black leather jacket actually look black for example. But I don't know if that is even possible with HEX?

This probably isn't easy to do with a normal hex edit, but it's something that could be controllable by the custom hooks. Ideally, a hook placed at the point where a character's armor changed, given the character entity/actor object and the armor type, could modify the color parameters stored in the actor object. The Lua code for that hook would need to know the right colors to apply for each possible armor type (so it wouldn't be an editable field in the armor itself, but a programmed behavior for armor-change events). I still have a lot of digging to do in the actor object/class, but once I find the color entries I could try to whip up a proof-of-concept.

This is also also good news because FT Improver is a problem for me when attempting to create MAC versions of my mod. Wine GOG version that I use only launch BOS.exe, so I never figured out how to change that to FTImprover.exe. Will your version only transorm BOS.exe and be comptable with MAC GOG version? I'm asking way too much questions, It's just to tell you how much I am interested by what you are creating here ha!..

I can't answer for certain (don't have a machine to test with). But, the DLL doesn't use much other than the C++ runtime, and the VirtualProtect function in KERNEL32.DLL. If Wine supports VirtualProtect, then it's almost certainly workable (could get around the C++ runtime issues by statically linking, or using an older version of Visual Studio, if necessary).

 

[nadeauhugo]

This probably isn't easy to do with a normal hex edit, but it's something that could be controllable by the custom hooks. Ideally, a hook placed at the point where a character's armor changed, given the character entity/actor object and the armor type, could modify the color parameters stored in the actor object. The Lua code for that hook would need to know the right colors to apply for each possible armor type (so it wouldn't be an editable field in the armor itself, but a programmed behavior for armor-change events). I still have a lot of digging to do in the actor object/class, but once I find the color entries I could try to whip up a proof-of-concept.

Oh yeah that will certainly do it! As we can add as many types of armour as we want, we could even have some specific colored types or armour, and then chose between them when designing objects.

I can't answer for certain (don't have a machine to test with). But, the DLL doesn't use much other than the C++ runtime, and the VirtualProtect function in KERNEL32.DLL. If Wine supports VirtualProtect, then it's almost certainly workable (could get around the C++ runtime issues by statically linking, or using an older version of Visual Studio, if necessary).

I have the plan to hire a friend of mine to create a wine MAC version from scratch anyways if that proves to be a challenge, I guess he would be able to run directly FTImprover or your extender instead of BOS from his version.

Keep on working, we are already getting so far, thanks to all your discoveries!

 

[Melindil]

So .. about those text/font color changes .. :/

(Warning: Long rant incoming. Skip ahead to TL;DR if you aren't interested in FoT implementation details.)

When working with colors, the FoT engine seems to not be able to make up its mind. The game uses typical 32-bit ARGB values for ZAR and PNG files, downsamples to 17-bit format, then to indexed color, then right back to 32-bit ARGB. Through this conversion, it also makes a pass through a floating-point value. In many places, color information is divided by 255, multiplied by floating-point RGB channel values between 0.0 and 1.0, then multiplied by 255. Floating-point math is actually how many graphics cards today handle color data, but in 2001 this needed to be done in the CPU, which was extremely expensive when done per-pixel.

(Historical aside: This is also why it took so long for the color portraits fix to be found. Searching for the raw ARGB values (e.g. 00FF00) was actually a good approach, but because the game held the color multipliers in floating point, the actual value to search for was more like (0x00000000, 0x3f800000, 0x00000000), though even this would have missed it because the green used for the portraits wasn't a "pure" green, but had about a 24% red channel multiplier.)

To keep track of these multipliers, the game has an internal class named DefaultStyle. This holds color multipliers for many of the text elements in the game. Setting these values can actually be done outside of hex editing, modifying the gui/def_style.cfg file. But that file is missing the most-used colors:

color_bosText = {Color(r,g,b)} used for the yellow text on most panels
color_consoleGreen = {Color(r,g,b)} used for the default green text
(and coincidentally, also the portraits!)
color_consoleSelect = {Color(r,g,b)} used for highlighted text

If these three colors are added to the gui/def_style.cfg file, the text displayed in the game is modified to those colors.

Well, most of it.

For some windows, there are certain fields which use the same colors as the defaults, but hardcoded into the window code, rather than pulling from the DefaultStyle object. It's like one development team didn't know the DefaultStyle existed, but knew which colors things were supposed to be. So in order to change all of the text to match, we need to find any hardcoded text displays, and modify them to match the equivalent DefaultStyle value. So far, I've counted color information in 4 different places, and I'm still missing a few.

---
TL;DR: I've added a hook to FOTExtender (please, someone help me come up with a better name!) to allow adjustment of the DefaultStyle parameters I've figured out so far (some I can't see any effect from), and also the matching hardcoded values for the character stats screen (the worst offender of hardcoded values). The Lua code to adjust looks like:

function DefaultStyleChanges(style)
  style:SetColorDefaultText(0.9,0.6,0.0)
  style:SetColorHighlightText(1.0,1.0,0.0)
  style:SetColorOptionsPages(1.0,1.0,1.0)
  style:SetColorPanelTitles(1.0,1.0,1.0)
  style:SetColorBuffs(0.0,0.2,1.0)
  style:SetColorDebuffs(0.0,1.0,0.0)
  style:SetColorTags(0.45,0.3,0)
end

This horrible color pattern looks like the following in-game:

With enough digging, it should be possible to pick up any stragglers.

 

[nadeauhugo]

Haha, can't really blame FOT engine for programming as messy as that. I realized over time that it is really hard to keep the game simple, I even created a hell of a mess in certain areas (like tiles) when attempting to put some order... It is amazing how you were able to find these errors so quickly. The change of colors is very promising, means that I will no longer need to play with the fonts directly to be able to alter game's colors. Much easier, and bearing a lot more potential...

A new name for the extender? That would be great, since it can get confused with Improver. I like Extender anyway, specially if it replaces Improver at the end. FOTMasterkey or Lockpicker could be good I think, and maybe more suitable for what kind of work you are actually doing to it, but I understand the main objective of the Extender will be to add new stuff to the game, like perks and else

 

[Imagination Engineer]

I remember this discussion when this issue was brought up and you invested the work to do this, congrats! Dunno if I'm gonna use it, but options are always great to have(especially since it gives the ability of using FoT to create mods on other stuff).

FoTEvolutionaryProject ? (FoTEP?)
FoTAdvancedAbilitiesCell? (FoTAAC)?

MasterFoT? (Though in my opinion, this would be an appropriate name for a suite that unifies all the relevant FoT Programs like yours and Redviewer in one application{wouldn't need too much coding either, something that opens each individual tool through a menu or something))

Now all we need is some app that produces dialogue trees without the past "trick" methods and we're set for serious modding.

 

[nadeauhugo]

Dialog tree? Hell yeah!

 

[Imagination Engineer]

Yeah! Though who would do that? I'm not even sure how much work it would require. Maybe programmers like Melindil here, could tell us.

 

[Melindil]

Dialog scripting is on my to-do list, but it's pretty far down the list. Not because it isn't useful or cool, but because I know I need a lot more experience with FoT internals before trying to tackle it.

A rough guideline for what I think would be needed is:

1) The Lua scripting language used in the extender could certainly run the dialog processing, so all of that could be done outside of hex editing (far easier). It might be good to have a sample model/implementation of a game with a decent moddable dialog tree system already, mainly for ideas on what pieces are necessary.

2) The dialog engine needs the ability to set and check mission and campaign variables, for cross functionality between the built-in FoT scripting behaviors and that driven from the extender's Lua scripts. This is much higher on my to-do list, and given my investigation in other areas, I don't expect too much trouble here.

3) The UI is likely to be the hardest part. Best-case (effort-wise at least) is if there's an existing UI window that's close enough to a working dialog interface that only minimal change is necessary (maybe adding a text box to the existing conversation UI window for showing the player's dialog options could work). If not, it may involve coding a window interface in the extender DLL, either using the individual FoT UI controls, or by calling DirectDraw functions directly. This is much harder, but could be a bit more customizable.

I'm not getting a lot of time to work on things lately (baby care ), but I'm fairly close to having a first release ready. Capabilities will be mostly limited to those I've shown in other posts already, but a couple of things have been added to give a little more flexibility (mainly visibility to the game timers, and functions to adjust both permanent and temporary attribute modifiers).

 

[Gizmojunk]

Seems out of character for the setting; especially since all three games (that matter) used 2-bit monochrome color portraits; as per the implied computer screens in the game... and they certainly weren't limited by the VGA palette for that. That aside, it's interesting work to see done.

*BTW... WTH happened to the Wilma character in FO:Tactics?
(Does anyone know offhand?)

When I load old save games it shows the correct photo in the game thumbnail, but in game, it substitutes a new character called Wilna, with a different portrait.

Spoiler

???? The Wilma PC is no longer an option. Is this the unadvertised side effect of a mod?