MHServerEmu Progress Report: March 2024
Spring has come to the Northern Hemisphere, and the development of MHServerEmu is blooming.
0.1.0 Release
This month we have reached an important milestone: our first “official” binary release. Version 0.1.0 represents the progress we were able to achieve in the first 8 months of this project, including all the work on the game database, properties, and region generation we have been talking about in these reports. In addition to that, we have implemented enough fundamental systems to remove the last remaining hardcoded data packets, which allows us to fully control the data that is serialized to the client and no longer be bound to a specific version of the game.
If you have been following the development and tried out the nightly snapshot builds, most of what is contained in this release should be familiar to you. However, there are a few extras we included.
0.1.0 comes with the new Setup Sorcerer Supreme tool, which is our flavored spin on the setup wizard idea that is common in software. Here is what it currently does:
-
Verifies the game client version by finding the main executable, hashing it, and comparing the hash to the expected one.
-
Copies the
.sipfiles required to run the server to the MHServerEmu data directory. -
Creates a
StartClient.batfile that launches the client configured to connect to a local server. -
Creates a
StartClientAutoLogin.batfile that launches the client configured to connect to a local server with automatic login using pre-defined credentials.
This release also comes bundled with the Apache web server and web assets for the in-game store and news panels. These assets are also available separately in the new MHServerEmuWebAssets repository.
As soon as 0.1.0 was released, we started working on version 0.2.0. Currently we do not have any estimates for when it is going to be out, but like always you can follow the progress on GitHub and via nightly builds.
Game Database Browser
Kawaikikinou has finished the work on a new game database browser tool that allows you to easily browse all the static game data we have been decoding over the past few months. Now you can relatively easily find out, for instance, what was the actual drop rate of the infamous Gem of the Kursed.
The browser works by plugging in existing server code into a WPF-based GUI. This is more of a development tool rather than something that is intended to be used by end users, but it has already proven to be extremely helpful in getting a better understanding of how everything fits together. The source code is available on GitHub.
Server Architecture Improvements
One of the first things I wanted to do in 0.2.0 was to make improvements to the overall server achitecture and how data flows around. Here is an overview of what has been done so far.
Major parts of the server (game, player management, billing, and so on) have been decoupled from each other and split into separate library projects. This allows us to potentially remove or substitute certain service implementations. For example, the database access layer can now be more easily modified to use various persistent storage solutions, such as a fully featured database management system. Also this allows us to reuse game code for tools, like the previously mentioned game database browser.
Duplicate login handling has been improved. Previously, the player manager would refuse connections for any players that are trying to log in using an account that is already logged in. This could cause potential issues if for some reason a player remained logged in due to an unexpected crash or some other problem. Now it works like most online games: when a duplicate login is detected, the already logged in player is disconnected, their data is saved, and then reassigned to the new session.
Our continuous integration pipeline that is used for nightly builds now has an additional step - running automated tests. As the server codebase grows, it becomes harder and harder to keep track of all the potential things that could break, especially because there are certain parts that have to be 100% in sync with the client. With daily automated testing hopefully we will be able to discover and fix any potential issues with existing code as soon as they come up.
The network message flow has been overhauled. Messages are now deserialized asynchronously before they reach game simulations, which frees up valuable frame time. The way messages are batched together and sent to the clients has also been redone, and is now closer to the client implementation. Connected players are now represented in the game they are in by PlayerConnection instances, which allowed us to clean up message handling and improve individual player state tracking.
The work on this front continues, and some more progress has been made with properly reimplementing the custom Gazillion serialization archive system, which we are going to talk more about in the next report.
Steam Deck Support
Thanks to the efforts of FF_Lowthor from our Discord server we have been able to get the game running on Linux (and Steam Deck).
The game client has a rather peculiar compatibility issue when running under Proton. During the authentication process the client receives a session token and a 256-bit AES encryption key from the auth server. It then generates an initialization vector (IV) that it mixes with the encryption key to encrypt the token it received, which is then used to authenticate with the frontend server. Something goes wrong somewhere in this token encryption process under Proton, which causes the authentication to fail.
FF_Lowthor has made a patch that allows the client to ignore this error and proceed with the login without a properly encrypted token. You can apply this patch by changing 75 to EB at 0x019B317E in the Win64 version of MarvelHeroesOmega.exe either manually with a hex editor, or using our MHPatcher tool.
Server-side you also have to enable BypassAuth in Config.ini to allow clients to log in without valid session tokens. As quality of life feature, the server now also saves data for the default account used when BypassAuth is enabled in a JSON file.
Entity Spawning and Navi
This month AlexBond is back to talk about world entity spawning and the NaviSystem. If you have not already, be sure to check out his explanation of the procedural region generation from the January report.
Hey everyone, it’s AlexBond. In this report I would like to talk about the enemy spawning process and a little bit about NaviMesh generation.
ClusterObject
ClusterObject is the foundation of all spawners in the game, from idle NPCs standing around and chatting with each other, to armies of enemies in a formation with their leaders and henchmen.
At the heart of these objects lies data from PopulationObjectPrototype instances. They describe the number of regulars enemies (called riders), their formation, and a bunch of other settings. This prototype class has various derived classes that define whether a group has a leader, how many henchmen it contains, and many other specifics. ClusterObjects also belong to a ClusterGroup, and there is a derived ClusterEntity class that is responsible for spawning specific enemies.
PopulationMarker Spawning
Cell prototypes contain SpawnMarkerPrototype instances. When we load region missions, we look for the PopulationSpawns field that describes the amount of enemies on the map and the name of the spawn marker. We assemble all of this information into the PopulationMarkers array, and in the Cell.PostGenerate() method we add enemies to their places.
For random placements we have the SpawnMarkerRegistry class that uses SpawnReservation to randomly pick free markers that are appropriate for specific enemies.
After determining the required marker coordinates, we place the enemy group using the assigned formation method. There are four kinds of formations:
-
BoxFormation - grid-type formation that uses rows and columns.
-
LineFormation - formation along a line.
-
ArcFormation - formation along an arc.
-
FixedFormation - formation that uses fixed coordinates (FormationSlot).
Other than formations, there is also placement using coordinates defined in EncounterResourcePrototype. For example, Jessica Jones and Ben Urich in the Avengers Tower are placed this way.
Spawner
A spawner is a special entity type that contains information about the number of enemies, their spawn timers, the uniqueness of their spawn distance, and many other parameters. Since spawners are derived from entities, they can be a part of a ClusterObject.
Currently we do not take spawn timers into account, so all enemies spawn at the same time (which occasionally looks overly populated).
One important distinction of spawners is that they place enemies randomly using the PickPositionInSector() method. To better illustrate how this works, let’s imagine a game of darts:
We divide the space in a radius around an object into equal sections, and then we randomly determine the slot we are going to use for spawning. Then we check for collisions with other objects, and if this check fails we randomly pick another spawning point. Currently we do not have any Navi checks, so many enemies spawn in the air or inside walls.
In addition to spawners described in missions and cells, there are also spawners defined in MetaState prototypes.
MetaState Spawning
All region prototypes have a MetaGame field. We can get MetaState by going through the following sequence:
For wave-based regions, such as the S.H.I.E.L.D. Holo-Sim, the path is slightly different:
There are different varieties of MetaStates:
-
MetaStateMissionProgressionPrototype
-
MetaStateMissionActivatePrototype
-
MetaStateMissionSequencerPrototype
-
MetaStateWaveInstancePrototype
-
MetaStatePopulationMaintainPrototype
Right now we don’t have properly working events in the game, so we just get population objects from the PopulationRequiredObjectPrototype, and then we retrieve PopulationMarkers, which we then proceed to use as described above.
There is one more type of enemy placements defined in PopulationThemePrototype instances. Instead of markers, it uses BlackOutZonePrototype instances and weights. However, we cannot implement this method of spawning without the NaviSystem.
NaviSystem
Over the last month I have been working on reimplementing the NaviSystem, and I have already achieved some results. Essentially, the NaviSystem is a NaviMesh, which is a set of NaviTriangles consisting of three NaviEdges built from two NaviPoints. The generation process happens in the NaviCDT class (CDT stands for Constrained Delaunay Triangulation). Here is an article about this technique in case you are interested to learn more.
When we initialize a game region, we create a region-sized rectangle consisting of two triangles. Then we add cell information to ModifyMeshPatch using AddNavigationDataToRegion(), which we then use to add points to our NaviMesh by breaking our triangles into smaller SplitTriangles. Afterwards these points are connected by an edge that splits other edges through SplitEdge(). After adding all edges, we mark triangles with MarkupMesh(). We then apply the following PathFlags as necessary:
-
Walk
-
Fly
-
Power
-
Sight
-
TallWalk
By knowing the coordinates of a point we can determine what triangle it is in, and this way the game understands whether you can walk, fly, use abilities, or aim in this point.
Here is an animation of how triangles are broken down for the NPEAvengersTowerHUBRegion:
(Note: P=100 means that 100 NaviPoints were added to the NaviMesh.)
If we filter the triangles and show only those that contain the Walk PathFlag, we get an outline of the Avengers Tower:
There is still a work to do with the NaviSystem. For example, it took me three days to find the cause of one of the issues. The system is complex, but it is the foundation of all physics in the game.
This is all we have for you today. See you in a month!