Alikhan's GSoC Blog

My second week of GSoC is over and has been full of different activities: I wrote two news items, took screenshots for qdEngine and SLUDGE games, started working on a new engine (WAGE) and had a video meeting with my mentor.

As I wrote in my previous posts, the main work on qdEngine is done, but few things were needed before the announcement: preparing screenshots, creating wiki pages for the games and writing the announcement post itself. And I started the week this those tasks. Taking the screenshots turned out to be quite useful as I discovered the regression caused by one of commits. In particular, when playing “Wait for it! Issue 3. Song for a Hare” I found that the game went black screen and stalled after playing a video. After probing the packets with ffprobe, I found that it happened because of a packet with invalid timestamp, which I did not consider in my PR. I quickly fixed the issue, and the videos are playing normally now. After the screenshots, I created a wiki page for the newly added games and I wrote a news item to announce the full support for qdEngine.

Following the qdEngine, I did the same kind of tasks for the SLUDGE engine. Again, taking the screenshots were useful as I discovered that I did not cover one case with save/load system – autosaves. By that time, I was very comfortable with the saves, so it was an easy fix.

After the SLUDGE, I started working on WAGE engine. This engine is also very close to completion with only a few small things left to be done. However, to properly implement the leftover features and fix the bugs, I needed to compare with the original, which wasn’t that easy to set up due to intricacies of the file system of Macintosh. But that wasn’t that big of an issue, thanks to ScummVM project leader and my mentor – Eugene (sev), who kindly guided and explained every step for me in video call.

Right now I am continuing my work on WAGE. I’ve already fixed a couple of bugs, and there is only problem with the text (About dialog) left for now. After that, I plan to redump the games which is also part of the tasks for this engine.

In the first week, I started with finishing the work on qdEngine, then began working on the SLUDGE engine.

With qdEngine, what needed to be done was to improve the performance in the game Dog-n-cat: In the Footsteps of Unprecedented Beasts. After a little investigation, it turned out that whenever the tiled animations were drawn, the uncompress() method was called. This was very costly, and the solution was to cache the already uncompressed animations.

After finishing with this task, I was assigned to finish the SLUDGE engine. The engine is almost finished, with only a couple of small things to be done. I began with implementing the support for extended saves, which turned out to be a little harder than I expected. In particular, there was a problem with the saving the files in the ScummVM format. The saves with simple names are in this format: %s.%03d, where d is slot number. At first, I didn’t know how to get the slot number from the name of a save. This was needed for loading the save files in the games. The problem was that the SLUDGE game interpreter is based on stack machine and the only thing which is pushed to stack when loadGame function is called is the name (description) of the save.

This problem took quite a while to solve, but after many unsuccessful attempts and tries, I finally came up with the solution. I realized that almost every game when loading/saving the game shows the save file list:This meant that I could create a lookup table that maps the save name to slot number of the save file and use it for saving/loading procedures. And indeed, it worked out.

There was also a little interesting task was to testing transition modes of the engine. For that, I needed to create a demo with SLUDGE development kit, run it on original interpreter and on ScummVM, compare, and see if they are different. The transition modes appeared to be the same, so there was no additional work needed from my side.

After that, I quickly implemented return-to-launcher feature, fixed a bug in thumbnail loading and wrote a code to simulate the right click of a mouse. Lastly, I made it so message box with a specific message set by a game pops up when exiting the game:

In this post I would like to go over some of the highlights of the changes I made to qdEngine.

Adding support for RGBA8888 format

At first, there wasn’t any intent to add a support a 32bpp color format. The engine was using RGB565, and it was pretty much fine with most of the games. However, when I began playtesting Pilot Brothers 3D, and got to a specific place in the game, I quickly noticed that something was off:

When looking more closely, you can see that the shadows under actor’s bodies are dark green:

This problem took quite some time to solve. I was checking every line of the drawing functions, comparing with the original source code, but was unsuccessful. Then, I tried to change things, in particular, tried to remove alpha_blend_565() inside the methods, and realized that it was responsible for the shadow color. I looked again at the the original, it saw that it was RGB888. But we were using RGB565, which of course has less bits and a thus created noise for the shadows. Since there is an extra bit for green channel, the shadow was a little greenish. Thus, I proceeded with adding the support for this color format, and indeed got the shadows fixed:

Although it took about 2 days at the time of fixing that, I now much understand to engine’s drawing functions and learning few things about color formats. Here is the PR for the fix: https://github.com/scummvm/scummvm/pull/6552.

Checking different game (engine) versions and learning Ghidra

One of the aspects of working on QdEngine was comparing the code across different engine versions. The engine developing over time, which meant that because of the changes made to the engine, things such as pathfinding, inventory selection and collision system could work different across different the games. We had sources for multiple versions of the engine, so once the sources are compared and the change is found, a typical fix could be like this:

here the date (number against which g_engine->_gameVersion) indicates the engine version where the a certain change was introduced.

When the sources were unavailable, we had to rely on reverse engineering. For example, we didn’t have the sources for the engines of two games. To find the exact cutoff date, I would write the exact the method and how it was changed to Sev, and he would reverse and tell me the date. Thanks for his help, we were able to fix a decent amount of bugs and make the games completable to the end.

In beginning of May, I started with playtesting Brother Pilots 3D-2 game and could not even run it. This was because one of the tags in the script specific to this game (engine) version was missing. At the time, Sev couldn’t help me, as his laptop was in repairment. So I thought to myself, maybe I could try fixing it myself. I downloaded Ghidra and uploaded three games (including Brother Pilots 3D-2) with consecutive engine versions. Then I found the place where the the tags registered:

After that I took decompiled code of this function, pasted to my editor and diffed between the games:

In the image above, I compare the code for Brother Pilots 3D-2 and the game that came before it. As can be seen from the image, rotation_angle_per_quant tag is added, so the I adjusted in the code the date and was able to run the game. This was a small change, but quite useful for me, as I learned a little about reverse engineering and Ghidra.

Fixing file loading and adding support for advanced minigames games

This was supposed to be my of the main tasks for this GSoC. I needed to change how the interface for “advanced” minigames worked. Not that they are called advanced, but they are complex in comparison to the ones in other games, and added more things to previous interface.

To understand the problem better, I first tried to run the games, and see where I crash, or where things are supposed to not work. I took the first game, Dog-n-cat: In the Footsteps of Unprecedented Beasts and tried to run it and noticed that I can’t get past the logo image. What’s more important, the memory usage panel in Visual Studio was showing 3.8 Gigabytes:

That was quickly fixed by adding back early end-of-stream check from the original sources:

But something was still missing. I was still getting this warning: MinigameManager::init(): Game could not be initialized. This meant that state (file) loading was unsuccessful and returned false. The code for reading from file looked something like this:

At this stage, the method could only return false only from code branch I just introduced. So it was reaching the end of stream every time. I did not realize ScummVM’s ReadStream::eos() would return true only when reading beyond file size. In the original, however, their eof() would return true once file size bytes are read, and of course the code in this section relied on that. But once I figured out and adding small fixes, the minigames started showing up. We first though that more complex approach and rewiring would be needed, however, it was much simpler. After that, unstubbing the constructors the each minigame was quite easy and I was able to finish the work early.

In this post, I would like to share a story of solving one of the recent bugs I had in qdEngine.

First, let me tell you about the bug. The problem was with the minigame in the game called “Dog-n-cat: In the Footsteps of Unprecedented Beasts”. In the minigame, players swap and move around triangle pieces to assemble the picture of an animal. The problem was that when swapping triangles, the triangles would move to swapped positions, instead of folding/unfolding in place. And the animation itself had artefacts. This is how it looked before the fix:

and how it looks now, after the fix:

 

I first thought the problem was with the code logic of this minigame. Even though we already had the sources of the minigames, as it turned out, some of the parts of code were not present there. This led me to think that maybe some crucial piece of code responsible for the rotation was absent. I told this to my mentor – Eugene Sandulenko (aka sev), and he provided me with the decompiled code of the dll for the minigame. But, it turned out to be the same. Then, he went with reversing the several games which were developed in later versions of the engine to see if important differences were introduced. Again, differences that could help were not found.

We started to lose hope on this, so I started to think about the problem from beginning and try a different approach. Sev adviced to look again at what exactly was responsible for rotation and tranformation of the triangle. And I looked very closely at qdScreenTransform::change(), the method that changes object’s angle and scale when transformation happens. In our case, it was called when triangle is transformed during swapping. I noticed that the angle was always 0.0 (wasn’t changing), but the scale in y-axis was decreasing. What’s important is that I was testing it by swapping the triangles vertically. And then, it really came to my mind, that no rotation should not even be happening in this case. Turned out, to simulate the effect of folding, the game was only scaling down the y-component of the triangle object. This meant that I only needed to check the drawing methods with scale parameter. And indeed, after comparing with the original source code, the issue was in the wrong conditional operator:

This bug a took quite some time to solve, because of wrong a wrong assumption I had. However, with the help of my mentor and an insight that came to me eventually, the problem was successfully solved.

Thanks for reading.

Hi everyone!

My name is Alikhan and I am one of the GSoC students of this year. The task in my proposal was finishing the implementation of QdEngine. However, I’ve been working on the engine since March, and the task to implement interface for minigames appeared to be easier than was thought initially. Thus, there is only one task left for me to finish, so I plan to work on other engines after that.

In these blog posts I’ll be sharing my progress of each week, post about problems I faced, how I dealt with them and things a learnt along the way.

That’s all for now. Thanks for reading.