MHServerEmu Progress Report: November 2024
Progress Report is back yet again to bring you all the latest updates on the development of MHServerEmu.
0.4.0 and Beyond
As I am writing this report, we are putting the finishing touches on the fourth stable release of MHServerEmu: version 0.4.0. The highlights of this release include all the loot updates we have done over the last three months, as well as a feature-complete implementation of the mission system that has been in the works since July. 0.4.0 will be released during the first week of December, before a certain event on the 6th that will hold the attention of many ARPG fans, including myself. Sorry if there will be limited updates next month!
While most of the updates in 0.4.0 focus on restoring features that give you reasons to play the game in the long term, for the next stable version I plan to focus on the long-awaited improvements to the moment-to-moment feel of the game, most of which require implementing additional power subsystems. I cannot give you an in-depth list of what is going to make it into 0.5.0 right now, but the first thing I am going to work on will be conditions, which is the term the game uses to refer to various buff and debuff effects. In addition to the obvious benefit of being able to apply various bonuses and penalties, this system is also important because enemy AI relies on some of the status effects applied by conditions to make decisions, such as moving around or using powers. Not having the right conditions can cause various issues, including client-server desynchronization when fighting teleporting bosses like Living Laser and MODOK, or Bullseye being able to run around while he is charging his instant kill attack.
In addition to this, Alex and Kawaikikinou have started working on implementing the achievement system. Unfortunately, this may take a little longer than expected: while most of the game data is mirrored to the client, the achievement system is implemented in a way that allowed the developers to modify achievement data without patching the client. The client receives a dump of the server’s achievement database when you log in, and we have a copy of such dump we extracted from packet captures, but the data the client receives is incomplete. There is enough information to reconstruct the missing data by cross-referencing localized text with prototype file names, but this is a pretty laborious and mostly manual process that is going to take some time.
We will talk more about 0.5.0 features in future reports as we get them working.
Vendors
One extra feature I was able to sneak into 0.4.0 is vendors. Previously it was already possible to sell items to vendors, but now you can also buy items using all sorts of currencies (including buying back the items you sold for credits), as well as level up vendors by donating items or completing influence missions.
As with many other systems in this game, there is a lot of smoke and mirrors to sell you (no pun intended) the illusion of buying items from NPCs in the game world, but the reality is much more self-centered. Items sold by various vendors are actually contained in inventories belonging to the Player entity, which represents your account data. When you interact with a vendor NPC, all the items available for sale technically already belong to you, and the cost you pay to purchase them is to move them from one inventory to another. The same is true for buyback: when you sell an item, you get credits for moving it from your general inventory to your buyback inventory. The buyback inventory is not persistent, so it resets when you transition to another region. Some items are flagged as ClonedWhenPurchasedFromVendor, which means purchasing them creates a duplicate that gets put into your inventory.
But where do these items actually come from? When you press the refresh button or level up a vendor, the game rolls four vendor-related properties that exist and get saved as part of your Player entity: VendorRollAvatar, VendorRollLevel, VendorRollSeed, and VendorRollTableLevel. The values of these properties are used as input settings for rolling the loot tables associated with the vendor type you refreshed or leveled up. Each vendor type contains one or more inventories, which are initially empty. When you interact with an NPC that has a VendorType property referring to a vendor type prototype, the game uses the previously determined input settings to roll the loot table and create the items this vendor is supposed to be selling. Because the loot table rolling algorithm is deterministic and all the input settings remain the same, you get the same items every time. The game tracks inventory slots the contain the items you have already purchased and filters them out the next time the loot table is rolled.
The prices displayed in the vendor panel are calculated by the client in parallel to the server, and this has been a major source of frustration. I have checked everything multiple times and calculated the formulas by hand for various input parameters, and no matter how you look at it, there are what appears to be pretty massive rounding errors happening client-side when calculating prices in credits. Unfortunately, I have not been able to find a way to replicate this behavior server-side yet, so for the time being the prices items are being bought and sold for will be slightly off. This affects only prices in credits as far as I can tell, and buy prices are affected more than sell prices, most likely due to the difference in coefficients used for calculating them.
With vendors implemented there is one side effect you can see: crafter NPCs now have their recipe lists populated and can also be leveled up. This is because crafters share their backend with vendors: every recipe is actually an item that is stored in the player’s inventory associated with a specific crafter type. “Items” representing recipes that unlock by leveling up your crafters are rolled using loot tables, same as items sold by regular vendors. As for recipes you can learn by using consumable items, they are stored separately in a special inventory called PlayerCraftingRecipesLearned. After a crafter’s recipe list is rolled, additional recipes contained in the PlayerCraftingRecipesLearned are duplicated and appended to the list. Please note that while you can level up your crafters in the current work-in-progress server builds, you cannot craft any recipes or learn new ones from consumable items.
Special vendor types, including Eternity Splinter and Odin Mark vendors, also work now, meaning you now have access to additional content, like the Classified Bovine Sector and Bovineheim portals you can purchase from Eternity Splinter vendors, as well as legendary items you can buy with Odin Marks. I have also put in some additional work to get legendary item leveling working. With all of this you should have plenty of medium-term goals to work towards while we work on restoring other features of the game.
Map Rotation
In this section I would like to talk about not a flashy new feature, but rather a frustrating bug that I have spent way too much time on than I probably should have.
First, let’s establish the background. Entities that can have a location in the game world are called world entities. A world entity can enter and exit the world: for example, when an item drops on the ground, it enters the world, but as soon as you pick it up, it exits the world and gets put into one of your inventories. Whether an entity is in the world or not can also be different between the client and the server. This is because the client simulates only the part of the world that is within the player’s proximity, while the server manages entire regions. However, the client can still be aware of entities that exist outside of its proximity, like other players, valuable items, and transitions to other regions.
In most cases entities outside of proximity are represented by map icons, and the client needs some way of knowing where to put them. The client cannot simulate locomotion without loading the environment, so instead there are two properties that world entities can have: MapPosition and MapOrientation. As the server simulates entities and locomotes them, it also periodically updates these two properties to notify when distant entities move or rotate. However, there was a problem: for some mysterious reason, map icons were not able to face bottom left directions. While this is an incredibly minor issue, it was bugging me (pun intended), so I had to get to the bottom of it.
Rotation in Marvel Heroes is expressed in radians rather than degrees. This is how most game engines do this, and the main reason is that while radians are less human-readable, they are easier and therefore faster to do calculations with. The game normalizes the results of most rotation-related calculations to a range of [-π ; π], which is also true for the MapOrientation property that holds a float type with a value range of approximately [-3.14 ; 3.14]. Here is a diagram to visualize what it looks like within the context of the coordinate system used by Marvel Heroes:
If you look carefully, you can probably notice what was causing the issue with bottom left directions specifically: they are expressed by negative values. Why is this an issue? Each entity property has a minimum and maximum allowed value, and everything outside of this range is clamped. For MapOrientation this range is [0 ; 65535], meaning any negative values get clamped to zero. What greatly added to the confusion was that in the packet captures we have the client in fact did receive negative values for the MapOrientation property from the server. So the instinct was to look for bugs in how properties work. However, this turned out to be a massive waste of time. You see, the clamping happens within the PropertyCollection::SetPropertyValue() method. However, the replication of the value is handled by the ReplicatedPropertyCollection class that inherits from PropertyCollection. In its SetPropertyValue() override it passes all the arguments to the implementation from the base class and then notifies the client if the property actually changed. Here is what the code looks like in our reimplementation:
protected override bool SetPropertyValue(PropertyId id, PropertyValue value, SetPropertyFlags flags)
{
bool changed = base.SetPropertyValue(id, value, flags);
if (changed)
MarkPropertyChanged(id, value, flags);
return changed;
}
The issue is that the new value is passed to the base implementation by value rather than by reference, which means the code clamps a copy of the value that was passed, and the value that is sent to the client is unaffected by it! In other words, both the client and the server clamp the original negative value to zero. Therefore, it is not possible to make the client accept negative values for MapOrientation without modifying its prototype data.
Figuring all of this out was the hard part, and the solution was actually incredibly easy. Rather than normalizing the value to a range of [-π ; π], we now get rid of negative values by using a range of [0 ; 2π]:
And with that icons for distant players can now face bottom left, as they should:
As insignificant as this bug may seem to be, I wanted to talk about it because it is a great example of one of the main difficulties we have with this project. Because we no longer have access to the original game, at times it can be hard to distinguish issues with our implementation from bugs that existed back in the day.
Thank you for following the development of MHServerEmu. See you next time!