MHServerEmu Progress Report: May 2025

May this Progress Report be with you.

Leaderboards

After many trials and tribulations, the leaderboard system is finally in a production-ready state, and it is now available in nightly builds and on public community servers:

While it may seem like a relatively simple system on its surface, it is actually somewhat more involved than it looks:

• This is our first properly implemented cross-game social system that requires extensive bidirectional communication between game instances and a separate service, so our service system needed some major upgrades to handle it.

• There is a relatively large amount of data that needs to be handled, because we need to store data not just for active leaderboards, but also archived leaderboard instances that ran in the past. This meant the Leaderboard Service needed to be backed by a database.

• Leaderboards operate on relatively long time cycles (from 4 hours to a week in most circumstances), which made real world testing time consuming.

The overall structure looks like this:

• The Leaderboard Service manages leaderboard instances and notifies game instances of any leaderboard state changes.

• Game instances use a tracking system similar to the one utilized by the achievement system to track player scores for active leaderboards.

• Player scores are batched and periodically sent from game instances to the Leaderboard Service, where they are accumulated, sorted, and cached.

• When a client requests leaderboard score data from a game instance, this request is relayed to the Leaderboard Service, which retrieves score data from its cache and generates a LeaderboardReport. This report is then relayed back to the client through the game instance.

• When leaderboard instances expire, the Leaderboard Service distributes participation rewards to players. These rewards are recorded in the database, so even if a participant is not online when a leaderboard expires, they will receive their reward when they log in at any point in the future.

In contrast to achievements, which can be added and removed with just server-side adjustments, leaderboards are fully defined by prototypes. Prototypes are mirrored to the client at build time, so this unfortunately means it is not possible to add new leaderboards without modifying the client. In total there are 27 usable leaderboard prototypes in version 1.52. Some leaderboard functionality, like guild leaderboards, is not usable, because there are no valid leaderboard prototypes utilizing it.

While leaderboards took longer to implement than expected, some of the backend improvements that had to be done will be very usable in the near feature when we work on bringing back other social features.

0.6.0 Status

Getting leaderboards out the door in 0.6.0 was a major goal for me. Although I was able to make it, some sacrifices had to be made, and the previously planned Player Manager overhaul has been pushed back to 0.7.0.

On the bright side, 0.6.0 is now almost ready for release, which we aim to do early next week, just in time for the game’s 12th anniversary. I have also prepared a bit of a surprise, which will hopefully be somewhat amusing for people who followed Marvel Heroes since its release. Stay tuned for news!

As soon as 0.6.0 is out, I plan to work on an updated roadmap that will go all the way to the long awaited version 1.0.0 and the final wipe. It is somewhat hard to believe, but we are actually not that far from the finishing line!

As for 0.7.0, the first order of business is to bring the gameplay side of things to a feature-complete state by implementing crafting, which is an umbrella term that also includes features like runewords and Pet Tech. After crafting is fully up and running, which I currently expect to happen in June, I will return to working on the various backend and social features.

OpenCalligraphy

I was looking into some power-related bug reports earlier in the month, and it became apparent that our tools were long overdue for an upgrade. Historically we worked on tools in a spur of the moment manner, doing just the bare minimum to solve issues as they arised. Because of this, tech debt and inefficiencies started to accumulate, resulting in our tools becoming significantly less versatile and performant than they could have been. This project is also almost in its polishing stage now, meaning that we rely on our tools more and more as we try to distinguish problems with our code from Gazillion Issues™.

For Game Database Browser in particular, which has been our primary tool for identifying data-related bugs, there were two major problems: it worked only with game version 1.52.0.1700 and it was slow. The only alternative was to use a tool called MHDataParser to convert binary data files to the slightly more readable JSON format. MHDataParser is version-agnostic, but its output is nowhere near as convenient to work with as Game Database Browser, especially when it came to examining relationships between prototypes, which is a major aspect of how the game is “scripted”.

I started by making a prototype that was effectively a Windows Forms based GUI for MHDataParser. Here is what it looked like:

This prototype quickly evolved into its own tool called OpenCalligraphy, named after the Calligraphy data framework used by the game. At the time of writing it looks like this:

I was able to port most of the functionality from Game Database Browser without sacrificing any flexibility or performance. We now have a tool that we can use to efficiently examine data from any version of the game all the way back to 1.9 (May 2013), which is the earliest version of the game that has been preserved.

There is also some new functionality, like being able to inspect .curve files:

One use case for OpenCalligraphy is figuring out how events were implemented. In many cases events were enabled and disabled by patching the data files, and it is not always obvious how everything is supposed to fit together when you are looking at pieces of a disabled event. To aid with this, OpenCalligraphy includes a tool called Pak Diff Utility, which can be used to compare data file archives from different versions of the game:

I have also used this as an opportunity to upgrade some of our older tools, like MHExecutableAnalyzer, which can be used to extract source file structure from client executables. An updated version of it is now included in OpenCalligraphy:

The plan going forward is to use OpenCalligraphy as the foundation of all our game data related tooling, including any potential editing functionality.

Both the source code and prebuilt binaries are available on GitHub under the MIT license, so feel free to do practically whatever you want with it.

---

I am going back to my programming cave. See you in a month.