Nipun Garg

Since I won’t be available during Week 3 and I have gotten somewhere over the past two days, this seems like an appropriate moment for an update.

I dedicated the past few days with drawing the first level. Naturally, this led to the map-loader class. What I didn’t expect at first was that my draw-manager was woefully incomplete. Hence, it took some back and forth between the two to finally get the first level going.

First Level

The good things first, here’s the partial image of the first level.

It looks blocky and incomplete since more than half of the drawing functionalities aren’t finished yet. In fact, it’s only drawing a portion of foreground tiles at this moment. At this point, I can easily blit Tiles and Pictures to the screen but haven’t got started with Masked Blitting, and Alpha-blended Blitting.

How I Got Here

To put it simply, HDB draws in a number of different layers, ranging from 2 to 4 depending on the level. Each level consists of a Background and a Foreground. Certain levels have an additional layer below the Background, known as the Sky Layer. The Sky tiles are clubbed together with the Background tiles in memory but are drawn separately. Moreover, in certain places, the foreground tiles have gratings placed over them, so those have to be separately rendered as Masked Tiles.

When the draw-manager is initiated, it creates a reference table for all the tiles in the game. Note, no MSM files or tile data is loaded at this point. Alongside the init function, draw-manager provides a set of helper functions for drawing tiles, skies, stars, etc.

Broadly speaking, the map-loader utilizes three different functions to draw the map: loadTiles, load, and draw.

1. load: It takes the MSM file wrapped in a SeekableReadStream as input, and loads the different portions of the file into the Map member variables. It also calls loadTiles.
2. loadTiles: For all the tiles present in the current map, its load their tile data memory, and updates the reference table created in draw-manager. It also tells us which Sky tile, if any.
3. draw: It iterates over all the loaded tiles, and draws all the background tiles. It also marks all the foreground, and grating tiles, indicating which blitting method is to be used with them.

The Skies, Foregrounds, and Gratings are drawn through separate draw functions, which are still stubbed out. They will be discussed in the next update.

The XY-Crutch

This is one of the main problems that I encountered over the past few days. It occurred because a) I didn’t trace the call stack that is used to start a map completely and b) due to the structure of MSM file.

In a nutshell, in each level, the hero has a starting position. The Map::draw function works on the assumption that the map has been centered to the hero’s starting position. The code that fetches the foreground indices does so relative to the position that the map has been centered on, referred to as _mapX and _mapY. If I had fully traced the Game::StartMap function in the original, I would have figured this out sooner. Hence, simply calling the draw function from the main file fed garbage values to the foreground indices, and I kept getting an out-of-bounds error from them.

While tracing the entire Game::StartMap function would have been a good idea, we ended up reading the MSM file in a hex editor. At the foreground offset(0x343C), we found a series of FF values which continued as we reached 0x3A2C. From there on, a clear pattern emerged that you can see below.

So right now, what we have done is set _mapX and _mapY to predefined values in accordance with the above pattern. This has been of some use, and now we can draw the partial map that you saw above. However, even this does not allow all the foreground tiles to be drawn. My next order of business would be to properly implement the Game::StartMap to the extent that I can right now so _mapX and _mapY don’t have to depend on predefined values.

Avoidable Problems and Missing Files

There was another stupid bug that I should’ve caught earlier. Since I managed to spend more than an hour debugging this, I think it deserves a mention. If anything, it demonstrates how easy it is for a game-breaking bug to slip in if you’re not careful.

The draw-manager has a set of three Sky tiles that are always loaded into memory whenever the draw-manager is initiated, regardless of the map-loader. Once I started debugging the Map::draw function, it became apparent to me that these tiles were not getting loaded.

Since I had been dealing with missing for a while now, I decided to check that over. Thankfully, the files were present in the archive. However, it seemed that the draw-manager refused to acknowledge that they were present. From there, I thought that the tile reference table might be messing up the Sky tiles. I went through the init function, checking and rechecking the tile reference table code. Along the way, I improved the code with more standard methods but it still gave the same error.

At this point, I was fairly certain that the Sky tiles existed, and the reference table was aware of them. Then, the logical conclusion becomes that the Sky tile was messing up the indices somehow. The Sky tile code is distributed throughout three functions: drawSky, isSky, setSky. They interact with each other, and that made it slightly hard to think of them intuitively. I soon learnt that the wrong Sky index was being passed into the function calls, and at the root of it was isSky. Simply speaking, isSky checks if a tile is a Sky tile and returns its skyIndex. Since there only 7 Sky tiles, they are indexed in a Sky tile array, indexed from 0 to 6. This array maps the skyIndex to the corresponding index of the tile reference table.

My mistake was that I had set isSky to return index + 1, where index is the tile index of the input tile. So instead of returning an index for the Sky tile array, it was returning an index for the tile reference table — and a completely wrong index at that! Changing index + 1 to i + 1 solved the problem.

Another issue: I’m still getting used to git in a project of this size. One thing that happened this week: I added a few lines that I thought were perfectly correct, and I pushed them to my remote. As one might expect, the changes broke the project to the point it stopped compiling. After I had corrected the problems, sev has given me a simple workflow to standardize commits:

Compile -> Diff -> Commit

A similar thing happened during the debug process. I had to rebase a few commits sev had pushed. There were a few merge conflicts, and I accidentally overwrote the changes that I had rebased for. In a hurry to correct the changes, I reset my local branch by three commits. Thankfully, I had merely added a few declarations and small initializations at that point which was easy to replicate.

Apart from my own foolishness, I found that certain files were missing from the MPC demo. According to the original source code, it should have been packaged with both the demo and the full version. In the beginning, I had started building the HDB engine based on the demo files. After this, I had to switch files and now I’m basing it on the full game.

Objectives

1. Trace and implement the StartMap function.
2. Add the Masked Blitting and Alpha Blitting functions to DrawMan.
3. Complete the stubbed-out drawSky functions.
4. Add code to draw Foregrounds and Gratings.

After a few days of code crunching, I can finally say that my LuaScript subsystem is operational enough that a portion of the game can be run with it.

LuaScript

The LuaScript subsystem allows us to load a Lua script and execute its default code in a new Lua environment. It also register the custom Lua extensions that were created for HDB.

The process consisted of implementing a simple interpreter, hooking it up to the file-manager and executing the bundled Lua scripts through Common::SeekableReadStreams. Alongside with the interpreter, Lua error reporting facilities were added to the engines. In case Lua faces a problem, a stack traceback is printed to the screen.

Along with a standard interpreter, the Lua subsystem in the original had a series of Lua extensions, Global Strings, Global Values, Sound and Entity Spawn names attached with them. As of right now, I have created stubbed-out versions of the extensions and registered them with each Lua environment. I’ll be adding the Globals as and when I need them. The Sound and Entity Spawn names will be added once those subsystems have been implemented.

The coming weeks would involve filling out these stubbed functions by building the subsystems they rely upon.

Changes from Lua 4.0 to 5.1.3

During the proposal period, I had performed a high-level overview of the differences between Lua 4.0 and Lua 5.1.3 in order to gauge how difficult it would be to run Lua 4.0 code through a Lua 5.1.3 interpreter. While the HDB codebase was largely compatible with the changes, it seems that there were a number of smaller changes that were not. Here is the list of the ones I have found so far.

1. Upvalue Syntax: Upvalues are the closet thing to static function variables in Lua. The syntax for accessing them inside a function was changed from %x to x for an upvalue-referenced variable x.
2. setglobal: Lua 4.0 had a predefined function for assignment. In certain cases, such as dynamically determining the name of a variable, it led to cleaner code and has been used in certain places.
3. for Loop Syntax: From Lua 5.0 to 5.1, there was a subtle change in the scope of the implicit variables of the for and for the repeat statement. Now for i, v in table has to be replaced by for i, v in pairs(table).

Another thing I found, which is not a difference in the Lua syntax, is the usage of C++ style comments in the Lua codebase. Apparently, these were put there for a reason since there is code to preprocess it in the original sources. For my implementation, I have replicated their method entirely.

Patches

In the beginning, the plan was to write a preprocessor function to transform the code into valid Lua 5.1.3 code. However, the syntax changes proved to be both unique and countably few that makes more sense to just patch the differences wherever they occur. Now, we have a ScriptPatch system in place that keeps a track of known differences, and updates them before executing a file.

Objectives

1. The current objective to build the map-loader, and draw an entire map to the screen.
2. Add the map animations once a complete single frame of the map has been drawn.
3. Add the main hero to the map, at which time we can work on input and AI.

The first week of GSoC is nearly over, so this seems to be the perfect opportunity for an update.

Accomplishments

Starting on a positive note, let’s go over the things that well. When I originally planned my project, I expected that I will have to build the low-level subsystems before going on to anything higher-level. Now that I have had some time to go through the original engine’s codebase in depth, I have realized that I don’t need that much low-level work to basically get started. So, I am revising my expectations: I would likely spending a lot of time going back and forth between the low-level subsystems and the actual game code.

Moving towards the actual tangible accomplishments, I have gotten pretty comfortable working with Surfaces, and drawing things to the screen. I started by drawing lines, and then moved on to drawing from the data file. The filesystem I had setup before had a few errors and I spent most of yesterday rooting them out. But it definitely paid off since now I can extract the various tiles, pictures and maps from the MPC file.

Today, I finally nailed down the drawing from file portion. Now that I have a good understanding how to extract data through the ScummVM API, I have managed to draw images and tiles to the game screen. Below is a screenshot of the Logo Screen, with a Ground Tile put on top.

As far as Lua is concerned, I’m just getting started with it and its my next target. From the Lua code I have read in other engines, it seems I have write an extended Lua interpreter with code extensions. Seems doable, but I’m still trying to get a feel for the Lua-C API. As far as the compatibility problem is concerned, I’m going to go ahead with 5.1.3 right on, and add the compatibility file if errors start propping up.

Objectives

1. Setup the LuaScript code so I can load and execute Lua chunks.
2. Understand the existing Lua code further, namely how to extend the default interpreter with new functions.
3. Once the Lua system properly works, I can try to load the Lua and execute portions of the game.