joefish: ScummVM Google Summer of Code

(Source: https://www.youtube.com/watch?v=qfEXShKCVnU)

The first week is already over and brought up some quite interesting problems. Here is a short list of what has been done.

• Decoding EALIB files (1)
• Converting audio files (1,2)
• Rendering .BM (fullscreen) images (1)

Implementation

The game’s assets are LZSS encoded in two libraries, imm1.lib and imm2.lib. The header starts with a 5 byte long signature “EALIB” and an uint16LE of the number of files encoded. The following table consists of 18 byte entries per file and a list terminator (flag x00) as its last entry.

Fullscreen image files like the title screen and the screen frame are stored linearly so that each nibble that represents one of the 16 possible colors just need to be written in its own byte.

Music files are stored as .BIN and consist of a custom header as colorfully shown above, followed by MIDI data. The header appears to be similar to Ski or Die but there are still questions open. This shows my best guess at what it could be + lots of googling

struct AudioHeader {
        uint16 midiOffset
        byte mt32ChannelActive[16]
        byte mt32ChannelVolume[16]
        byte adlibChannelMapping[9]
        byte cmsChannelMapping[12]
        // here starts the guessing for real
        byte cmdChannelTuning[12]
        byte TandyChannelSetup[8]
        byte speakerSetup[2]
        
        uint16 midiOffset
        byte anotherMt32ChannelActive[16]
        byte another_ChannelMapping[16]
        char signature[6] = "rsxx}u"
        byte data[10] // probably for editor but all zeroes from what I have seen
}

It is worth mentioning that the first MIDI event is missing its delta time, that I append as I convert the header to a type-0 MIDI header (code).

Audio effects have not been implemented yet. Unfortunately they are generated directly by calling into the sound driver that stills need to be reverse engineered.

Next Week

Although the audio is not yet done, I will leave it for now and implement handling of .ANM files and sprite rendering next.

Feels good to be back 🙂

I was one of the lucky four students to participate in GSoC under ScummVM.
For a more thourough introduction of myself see my post from last year. Over the course of this summer I will be porting The Immortal, a dungeon crawling adventure with difficult puzzles and cruel battles waiting within.

So far I have written the extractor and decoder for the library files of the DOS version, what will be the base for this port.

You can follow the progress by either keeping an eye on this blog or my working branch.

Overview

During the last three months I have worked on porting the game Mission Supernova to ScummVM, a project aiming to reimplement engines of point and click adventures for cross-platform support.

For the latest changes, see the commits to my fork.
Besides porting I also worked on the ScummVM launcher and fixed theme related bugs

• font fallback
• unexpected theme change

Goals

The following lists features that are implemented:

• Image and Text rendering
• Audio playback of sounds and music
• Animation system
• GUI
• Saving and restoring game state
• Game logic converted to engine reimplementation

TODO

Most crucial engine components work but there is still work left to be done:

• Translate the remaining strings to English
• Implement dialog system
• Fixing pesky bugs
• Make it all work together in harmony

While this may be the end of GSoC 2017, my journey has just begun.
Thanks to everyone involved sharing their knowledge and providing help whenever needed. Also thanks to Google for giving the incentive and exposure to all those incredible projects. The experience of staring at hexdumps, deciphering disassembly and the communal effort has been invaluable to me.

I will keep contributing to the ScummVM project in the future beyond finishing my project (just you wait, Chewy!) and hope that some of you out there will find entertainment in my work.

SPOILERS IN VIDEO! (0:22 – 0:31)

As announced last week, the timer bugs are no more!

The original game used ticks of 18.2Hz instead of milliseconds for example, so while this little helper converted ticks to milliseconds it was not always easy to recognize what number actually represented a time value.

For instance, I assumed that a variable named time would most likely be the perfect candidate. Oh, how wrong I was. It actually stored the remaining days until the ship arrives at its destination. Do not ask how long it took me to figure this out although comparing time with energy should have made me wary.. (1,2,3)

Adjusting the brightness of the palette was more an issue of rendering order. While the function worked fine it either did a buffer swap too early or the brightness was restored to its previous state by the following draw call.

By removing most special cases it not only simplified the function but also fixed most issues. I did a side by side comparison with the original game and couldn’t find any difference. (1)

What was different though is that I couldn’t interact with anything in the ship’s hold anymore. Apparently it hasn’t been working for a while.. Originally an uninitialized object was identified by checking if its name string points to NULL but I misread it as a check if the first char is 0 (empty string). Unfortunately, objects without a name can still be valid what messed up the traversal of objects in a room. (1)

After running in circles for a while a few important features like RTL and save/load support were implemented. Unfortunately, state serialization still shows puzzling behavior that I haven’t figured out how to solve yet. (1,2,3)

(Source: https://www.youtube.com/watch?v=XfLRdzZyPZE)

• The biggest change this week was fixing the terminal next to the sleeping pods and the clearing of previous input. (1,2)
• Also, the game text was converted to it’s original encoding (CP437) to correctly render special characters. (1)
• A long standing rendering bug that caused certain sprites not to be rendered was finally found and fixed. (1)

There were a few smaller commits that will help to finally fix timing issues and a few rendering problems with dimming the ship’s light next week.

The remaining game logic has been implemented with a few exceptions but still needs to be thoroughly tested. (1, 2)

Speaking of testing, there are still bugs in the first act that need to be ironed out before a pull request can be issued like rendering artifacts and problems with the terminal next to the sleeping pods.

For next week, it is planned to fix up act 1 and to finish implementing the dialog system as the second and third act are now available for testing.

This week was about implementing the terminal in the sleeping cabin room, adding the death screen and clean up.

Unfortunately it’s still quite buggy and needs more investigation. Currently, the game restarts on death to the beginning as there’s no full implementation of save/load yet.

For next week I will keep implementing missing game code for the remaining two acts and make it just compile for now. Most key functionality is already implemented but with the other acts in the engine the dialog functionality can finally be tested.

So most of the week was spent on rewritting the game loop to implement animations as shown above and fixing the bugs introduced by it..

The animation system is simply implemented by setting a timer that when it expires causes the room specific animation() function to be called. (1, 2, 3, 4)

Besides that,

• definitions for the remaining rooms were added
• initializes game state on startup (brought more surprises than anticipated)
• implement monitor scene

An interesting discovery for me was that although the text rendering code is correct it shows artifacts when drawing umlauts. As the game strings were copied directly from the cp437 encoded source I expected the character value to be encoded in a single byte as characters in extended ASCII don’t exceed 0xFF. What I didn’t consider is that UTF8 is only backwards compatible to basic ASCII (0x00 – 0x7F) and anything above takes at least two bytes, what causes the corruption as seen above (“Triebwerke funktionsunfähig”). That’s why umlauts in the German original will be substituted by their equivalent octal value as already done here.

For the coming days the first act should be made playable by implementing edit() so the sleeping cabin section works correctly and adjust the game loop for dying and implement the ‘death screen’.

After that implementing the remaining game logic takes priority especially now that the animation system is working.

Sorry for the delayed report this week but I really wanted to add a video finally walking through the empty hallways of the starship Supernova. Moreover…

? The GUI is finally working! ?

(Source: https://www.youtube.com/watch?v=RJhB39q3_kI)

Probably the most noticeable improvement is that GUI is now actually rendered correctly and interaction with the environment is finally possible without everything glitching out. Although, there are still issues with Give and Use command that will be addressed next.

Besides the GUI there have been improvements on every front.

• Wrapping GUI elements in a state class (1, 2, 3)
• Fixing font rendering
• Correctly handling and rendering object state (1, 2, 3, 4)
• Improvingn mouse input handling (1, 2)

Coming up next will be implementing timers for handling events and animations. There have been talks about starting translating but depending on how things go it either has to be pushed back or will have a lower priority.

Of course fixing a bunch of bugs along the way would be nice too 🙂

Sorry for the delayed update due to injuring myself while sleeping..

The input code still gives me quite a headache and will be partially rewritten when there’s time. I am now in a place I wanted to prevent ending up in but for now, getting things running has priority. As I said getting the input code working was my goal for this week additionally to fixing some GUI issues and rendering static GUI elements like the minimap, inventory, command row and status line.

I expected this part to be more difficult, as the original code is quite intertwined but I completely misjudged the effort needed for porting and debugging.

For this week I want to keep fixing the GUI and implement stubs so the remaining game logic can be implemented without too much trouble.