MHServerEmu Progress Report: February 2024
February may be the shortest month, but we are definitely not short on progress this time.
The Merging is Complete
The biggest news of the month is without doubt that the changes that AlexBond and Kawaikikinou have been implementing and testing on the experimental branch over the past few months have been merged with the master branch. The highlights of these changes include:
-
Fully-featured implementation of DRAG (Dynamic Random Area Generator), which makes all regions not only explorable, but also different between visits. The current system refreshes regions every five minutes as long as there are no players in them, but this is something we will continue to iterate on.
-
Early implementation of the population system, which handles the spawning of entities, including enemies, NPCs, and interactable objects, across the entire game. There is still a lot of work to be done here, including dynamic enemy spawning, but the regions now feel a lot less lonely.
-
Early experiment that allows to deal damage to enemies. This is more of a quick hack than a real implementation, but it does make exploration way more fun.
This is an important milestone that brings us closer towards our first official “release”, 0.1.0, that we currently expect to happen in March. In addition to what you can see in the current nightly build, it is also going to have some more additional polish, including a streamlined setup process that would not require you to manually configure a web server.
Once that is out, the current plan is to shift our focus back towards fundamentals, including overhauling our entity management and replication systems.
With the important news out of the way, let us take a closer look at some of the more technical happenings of February.
A Song of Bits and Bytes
What I have personally spent the most of my time working on this month is finally implementing one of the core systems of Marvel Heroes - properties.
All dynamic objects in the game are called entities. This includes player characters (internally called avatars), NPCs, enemies, projectiles, interactable objects, items, and so on. Entities are essentially collections of properties that define their state: what level they are, how much health they have, whether they are visible or not, and so on. Because of their integral role in pretty much all gameplay interactions, everything related to properties is very highly optimized, which makes it very difficult to untangle. If you include all the research and the foundational systems that had to be done beforehand, you could say that implementing properties actually took six months.
A property is a pair of two 64-bit values: a PropertyId and a PropertyValue. As you can probably guess from their names, one identifies a property, and the other contains its actual value.
The simplest form of a PropertyId looks like this:
The game initializes the so-called property info table and enumerates all property types during startup. Most of the properties are defined in code and are sorted alphabetically by their name. However, there are some additional data-only properties that are sorted by their blueprint id and appended at the end of the enumeration. Overall in version 1.52.0.1700 there are 1030 properties, of which 29 are data-only.
11 of the 64 bits in a PropertyId are allocated to the enum value, so the maximum possible number of property types is 1 << 11 = 2048. The remaining 53 bits are distributed amongst 0-4 parameters. For example, this PropertyId for a Waypoint property contains a single parameter that specifies which waypoint this property unlocks:
There are three supported types of parameters: integers, prototype ids, and asset ids. However, there is a problem: prototype and asset ids are actually 64-bit hashes, so how are we supposed to fit multiple 64-bit values in the 53 bits we have for parameters? This is where trickery comes in.
During game database initialization the game sorts all 64-bit prototype ids it contains, and then divides them into two types of buckets: by C++ class they bind to, and by Calligraphy blueprint they use. So you end up with arrays of sorted ids for each prototype class and blueprint. And by knowing which array to look in and where, you can retrieve the full id. A similar process happens for assets, so by knowing the asset type and its index you can get the id.
The game databases server-side and client-side are in sync, and the game makes heavy use of this in network communication to reduce the amount of data that needs to be sent back and forth. Instead of sending a number like 421791326977791218, which is the prototype id of the playable Iron Man avatar, you can send just 3, which is the index in the array of prototypes that use the avatar blueprint.
Property types have corresponding prototypes that contain additional metadata, including the types and subtypes (prototype blueprint or asset type) of all parameters. During property info table initialization the game processes all property prototypes and allocates the 53-bit param budget to defined parameters. First, it allocates the amount of bits needed to store the maximum index value for all prototype and asset parameters, and then it splits the remaining bits amongst any integer parameters, up to 31 bits (so only positive integers with a value up of up to 2147483647). For example, here is a PropertyId for an AvatarLibraryLevel property that defines the displayed level in the hero roster for Iron Man:
7 bits are allocated to hold the maximum value of the avatar prototype index, and then 31 of the remaining 46 bits are taken by an integer value. 15 bits in this case are left unused.
Finally, here is an example of a Proc property that has four parameters:
Param0 is the asset value that defines the proc trigger type, Param1 is the power prototype of the power triggered by this proc, and the remaining params are additional integer values. First 7 + 15 = 22 bits are allocated to asset and prototype params, and then the remaining bits are split evenly between integer params, 31 / 2 = 15 bits per parameter.
This is not where trickery ends however. Marvel Heroes makes heavy use of the encoding format developed by Google for their Protocol Buffers technology, and the way it works is that values can take anywhere from one to ten bytes depending on how high they are, with higher values taking more bytes. The number you end up with for a PropertyId often has a lot of zeroes at the end, so you waste a lot of network traffic if you send it as is. To circumvent this, the order of bytes is reversed before serialization: so 0x18E0000000000000 from the first example becomes 0x00000000000018E0, or just 0x18E0. However, there is an annoying inconsistency: when serializing a single property instead of a whole property collection, the game reverses the order of individual bits rather than bytes. So 0001 1000 1110 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 becomes 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0111 0001 1000, or just 111 0001 1000, which is equivalent to a hex value of 0x718.
And then we have PropertyValue. Thankfully, this one is not nearly as complicated. There are eleven data types that can be stored in a property value, most of them are simple 64-bit integer or 32-bit floating point values. But there are two funny ones: Curve and Int21Vector3.
With a curve property rather than assigning it a value, you assign it an index property and a curve. And then the value of the curve property gets automatically updated from the curve, using the value of the index property as the curve index. For example, all heroes have a HealthBase property that uses CombatLevel as its index. So the value of HealthBase automatically changes whenever CombatLevel is updated (i.e. you level up). The default curve used by playable characters, CharacterHealthMAX.curve, actually goes all the way to level 101 and looks like this:
With an Int21Vector3 we have a Vector3 to store, which consists of three 32-bit floating point values (X, Y, and Z coordinates), but we only have 64 bits. In this case the compression is lossy: individual values are rounded and cast to integers. The 64 bits are split into 64 / 3 = 21 bits per value. The values remain signed, so we end up with 20 magnitude bits + 1 sign bit, which gives us a range from -1048576 to 1048575 for each coordinate.
One challenge with implementing PropertyValue has been adapting the original C++ API to C#. Property collections in the client use templates for property getters and setters that look something along these lines:
// Getter
int characterLevel = properties->GetProperty<int>(PropertyId(CharacterLevel));
// Setter
properties->SetProperty<int>(60, PropertyId(CharacterLevel));
For the C# API we ended up with a combination of indexers and implicit casting that looks like this:
// Getter
int characterLevel = properties[PropertyEnum.CharacterLevel];
// Setter
properties[PropertyEnum.CharacterLevel] = 60;
The final piece of the property puzzle is aggregation. A property collection can be attached as a child to another property collection, and the values of the parent are going to be aggregated with the child. Each property collection actually contains two lists of properties: one for base values held in this particular collection, and one for values aggregated with all of its children. This system is what enables, for example, equipping and unequipping items that affect your character’s stats, or applying buffs and debuffs (internally called conditions). There is not too much interesting to say about aggregation, it is just a lot of mundane work to make sure that all values are updated properly.
And with all of that we now have a working property system. We have already started making use of it to replace some of the hardcoded data from captured packets we had to rely on, as well as doing little experiments, like the recently implemented damage dealing hack. One remaining aspect that we are going to have to tackle on in the future is the eval system, which allows properties to be used as variables in scripted formulas (for example, for calculating maximum health taking into account all bonuses and penalties). But that is going to be a whole massive endeavor of its own.
An Interesting Development
As Alex was implemeting region generation and entity spawning, an issue immerged. Turns out loading an entire region worth of environments and entities puts a significant amount of strain on the client, especially if you are running on lower-end hardware. So we needed to come up with a solution, at least a temporary one.
The way this is supposed to be handled is through a process called replication. The server is the “dungeon master”, it is aware of everything happening to everyone, but each client should only be aware of what is relevant to them. Not only does it reduce the amount of data that needs to be exchanged and improve performance, it also helps prevent some forms of cheating, such as map hacking. If the client literally has no information on what lies ahead, it is much harder to peek beyond what should be observable (there is actually a potential way around this, but I am not going to disclose it here).
Internally the game calls the data that needs to be sent to clients their interest, with each client having their own areas of interest (AOI). There are at least five area of interest channels that we are currently aware of:
-
ReplicateToProximity: the client needs to know what is happening around its physical location in the game world. -
ReplicateToParty: the client needs to be informed of its party members even if they are not in proximity. -
ReplicateToOwner: some entities should be replicated only to their owners. For example, this is not Diablo II, so only you should be able to see the loot you get from defeating enemies. -
ReplicateToDiscovered: when the client finds an NPC or a transition to another region, the client should still be somewhat aware of the entity, even if they are no longer in proximity with one another (a bit like fog of war in RTS games). -
ReplicateToTrader: when trading items the clients should be aware of what is being traded.
These channels can be mixed together, like ReplicateToProximity and ReplicateToOwner, so the client is aware only of the loot that belongs to it and is in proximity. We are still investigating how this system works, so some of these examples may not be completely accurate.
In the current version of MHServerEmu we now have an early implementation of proximity-based area of interest. This allows us to send cell and entity data as you move in the game world, reducing load times significantly. We also have some additional functionality not present in the original game for players running on ultrawide monitors and/or with a custom camera maximum distance: by typing !player AOIVolume value in chat you can customize the “draw distance” of entities that are considered to be in proximity.
Replication is a core part of any online game, because it is what keeps all clients synchronized with the game state happening on the server. We will most likely go more in-depth on this in future reports, once it is closer to being finished.
Live Tuning
One fun thing we were able to do this month is get the live tuning system up and running. This is a system that allowed the developers to do quick hotfixes without patching the game. While somewhat limited compared to changing game data directly, there are some interesting things you can do with it.
For instance, regions have the following live tuning “knobs” available to them:
enum RegionTuningVar {
eRTV_PlayerLimit = 0;
eRTV_Enabled = 1;
eRT_BonusXPPct = 2;
eRT_XPBuffDisplay = 3;
eRT_BonusItemFindMultiplier = 4;
eRTV_NumRegionTuningVars = 5;
}
eRTV_Enabled allows the server to disable individual regions. This is how, for example, the Mystic Mayhem in Limbo event is implemented: Limbo is a regular region that is disabled with live tuning unless the event is running. So by turning off this setting we can make Limbo accessible in-game.
Or we can go in the opposite direction and disable everything but Limbo. Feels like 2013 again!
Another group of tuning parameters affects avatar entities:
enum AvatarEntityTuningVar {
eAETV_BonusXPPct = 0;
eAETV_XPBuffDisplay = 1;
eAETV_EternitySplinterPrice = 2;
eAETV_Enabled = 3;
eAETV_NumAvatarEntityTuningVars = 4;
}
eAETV_Enabled here works similarly to regions, allowing us to disable individual heroes. One possible use for this feature could be implementing an Infinity War themed event where half of the playable heroes would be picked randomly and disabled for the duration of the event.
All the tuning parameters are defined in the protocol and can be found here. You can experiment with them yourself by editing LiveTuningData.json located in MHServerEmu\Data\Game\.
This is all we have to share today. Thank you very much for following the development of MHServerEmu. See you next time!