Category: Uncategorized

I finally got around to do a proper playthrough of EMI in ResidualVM with all the fixes and improvements made during GSoC. I’m happy to say the game can now be played from start to finish with only some minor issues 🙂

The biggest issue that remains right now is a bug with the lava boat puzzle on Monkey island. The boats on the lava field are not visible, which makes the puzzle difficult to complete. JoseJX is working on this issue though, so hopefully it will be resolved soon.

Other than that, there are a few graphical issues. Animations occasionally still seem to snap to the wrong frame. This happens for example when jumping out of the bank window on Lucre island, and when Guybrush gets a grog at the Micro-Groggery on Jambalaya island. Also, the text in the end credits flickers and the text color doesn’t match the original.

Pathfinding is not perfect yet. Actors tend to zig-zag around before reaching their destination, which looks silly sometimes.

On the audio side, footstep sounds are sometimes wrong. For example, when walking on the Jambalaya island beach, it sounds like Guybrush is walking on wood. Also, voiceovers sometimes tend to get cut short, which is a bit annoying.

These issues will have to be resolved after GSoC though. I’m pleased that all of the issues I spotted are fairly minor, and the game is very much playable in the current state. I’m confident that support for EMI could be included in the stable release of ResidualVM fairly soon.

As GSoC soon comes to a close, I want to thank my mentors and the ScummVM/ResidualVM community for your support and for providing me this great opportunity to bring another classic adventure back to life. This has been a fun summer!

With all the major feature implementation tasks completed, this week I focused mainly on fixing any remaining bugs I could find. Klusark was kind enough to do a full playthrough of the game in ResidualVM, which revealed a number of issues of varying severity. Some of these issues were blockers which prevented progression in the game at certain points. These should all be resolved now, and on current master the game should be completable from start to finish again.

Here’s a brief summary of the issues I resolved this week:

• Guybrush got stuck when attempting to lure the fish on Lucre island into the scupperware box, preventing progression in the game
• The game got stuck when giving the earrings to the Dainty Lady figurehead
• Saving and then restoring a savegame caused lighting to break
• Saving and restoring caused a crash due to broken shadow initialization
• Guybrush faced the wrong way after the Voodoo Lady summons the items from the heirloom chest
• Some of the ships and the water at the harbor on Melee island were rendered incorrectly
• Guybrush was invisible in the heirloom cave, and the light shaft effects were drawn incorrectly

The last two issues were the most interesting ones, so I’ll discuss those a bit next. Let’s look at a before-and-after comparison from the Melee Island harbor.

Several issues can be seen in the screenshot of the old version on the left. The color of the waves is too dark, the water splash sprites are drawn over the ship and the rowboat, and the bottom of the ship is not clipped correctly.

In debugging these issues, I used Apitrace in order to figure out how exactly the original game forms the final picture. Since the game has an OpenGL renderer (activated with the parameter -gl), we can use Apitrace to extract the list of OpenGL function calls as well as the OpenGL state on each drawn frame. By examining the trace I found that ResidualVM was drawing some of the sprites in a different order than the original game, and for some sprites depth testing was enabled while it wasn’t in ResidualVM. Also, the original game draws mask sprites to clip the ship and the rowboat to the correct shape, which were not drawn in ResidualVM at all.

When actors (including sprites) are about to be drawn in EMI, they are first sorted into descending order by their sort order, which is a per-actor integer field that can be controlled from the game scripts. When an actor is attached to another, the actor’s sort order will be overridden by the parent actor’s sort order. This is what happens when the water splash sprite is attached to the ship sprite. However, after the sprite is attached, the scripts then set a new sort order for the sprite. In the original engine this value overrides whatever was derived from the parent, but in ResidualVM the sort order was not updated, which caused the sprites to be drawn in wrong order. Updating the sort order even if the actor is attached fixed this issue.

The rest of the issues were mostly related to incomplete parsing of sprite and model data files. I discovered flag fields in the sprite data, which control whether depth and alpha testing should be enabled for that sprite, and whether additive blending should be used or not. I also found a similar flag that controlled the blending mode in the model data. I discovered these mostly through trial-and-error, by changing suspicious-looking bits in the data files slightly and then loading up the original game to see how the changes effect the end result. Setting the correct blend mode for models fixed the water color.

To enable drawing of the mask sprites, I had to disable a piece of old, temporary code that simply skipped drawing of any sprites with “mask” in the name. I assume this was done because if incorrectly rendered, the mask sprites looked ugly and only cluttered the image (see the image above). Now with the blending and sort order fixes in place, the masks did produce the correct result, though.

This is the heirloom cave before and after my changes. The left one is fairly obviously broken: Guybrush is invisible, the light shaft effects look ugly, and shadows are missing. Fortunately the light shaft effect was corrected by the previous model blend mode fix.

Guybrush’s invisibility was caused by another seemingly temporary hack in ResidualVM, which disabled drawing to the color buffer if an actor’s sort order was greater or equal to 100. In the cave, Guybrush’s sort order is set to 100 in the scripts, which caused him to become invisible. The original engine doesn’t seem to do anything similar, so I concluded that this behavior was wrong.

The cave set also led me to discover another previously unknown field in the set shadow data. The field is a string, which specifies a name of a light in the set. When this string is present, the shadow projection point should be the light’s position, instead of whatever is stored in the shadow data. After parsing the light name and implementing this behavior in ResidualVM, the shadows now work correctly in the cave also.

After last week’s progress, the sound in the PC version of EMI when run in ResidualVM was close to perfect. However, the PlayStation 2 version of the game still wasn’t playing any music. Although I haven’t focused much on support for the PS2 version so far I decided to look into the PS2 music issues, because I was ahead of schedule and this was one of my stretch goals.

Much of the groundwork for PS2 support has already been done by previous contributors. For example, the SCX sound format used for the sounds in the PS2 version is supported (although the implementation currently doesn’t stream from the disk). However, the music wasn’t playing because a mapping from music state numbers to filenames was missing.

EMI (like other iMUSE games before, I presume) assigns an integer state number for each music track. The game scripts tell the engine which state should be playing at any given time by calling the engine function ImSetState, which takes a state number as parameter. In the engine, we then need to be able to map the state number to the correct music file.

In the PC version this mapping is stored in the file FullMonkeyMap.imt, but unfortunately in the PS2 version the mapping is hardcoded into the PS2 executable.

To locate the data inside the PS2 executable, I used the excellent PS2 disassembler tool ps2dis. Firstly, after starting up ps2dis and opening the executable, I invoked the static analyzer from the Analyzer menu. This will add function labels and track data references, making the disassembly more understandable.

Next, I opened the “Jump to Labeled” dialog with Ctrl+G. This dialog lists the strings stored in the binary. Scrolling the list down a bit revealed this:

The strings that end with “.scx” are the music filenames. If we jump to the location where one of the strings is stored, We can then use “Jump to Next Referer” with F3, which will take us to a location in code that references this data. For the music filenames there is only one location referencing them.

This looks promising! The code here grabs the filename plus some unknown data and passes this information to a function. There are exactly 125 references to the filenames here, which is the same as the number of music states in EMI. My assumption is that the code here builds the state to filename mapping entry by entry counting up from state number 0.

With the state number to filename mapping figured out and implemented into ResidualVM, the PS2 version now plays music as well. These changes are now included in PR #972. With some additional fixes introduced in PR #975, the PS2 version is now fully playable, although some (mostly graphical) issues still remain.

For the past week I’ve been working on a number of improvements to the EMI sound system in ResidualVM. My change set in PR #972 improves especially the music playback in the PC version of the game.

Firstly, the music playback now starts at the correct position of the track, and the correct sub-section of the track is looped. This is because the music player now respects the cue point data which is stored in files with the file suffix “.jmm” alongside the actual MP3 music files.

The .jmm files may contain ‘start’, ‘jump’ and ‘end’ cues. The ‘start’ cue specifies a position in milliseconds where the music track should start playback. The ‘jump’ cue specifies two positions on the track. Once the first position is reached, the playback will jump to the second position. This cue is typically used to loop a sub-section of the music track. If the JMM file does not specify a ‘jump’ cue, the track does not loop (an example of this is the chapter screen music). The ‘end’ cue seems to always have a value of -1 in EMI, so I chose not to implement handling for it at this time.

Secondly, music syncing is now implemented. In some parts of the game, when the music track changes the new track should start from the same position where the previous track ended. Typically such synced tracks are slight variations of the same composition. For example, when Guybrush is close to the fountain in Melee town (set mel), the music changes to a version of the Melee town music with a sound of flowing water mixed in.

The information whether a music track should be synced with another track or not is stored in a file called FullMonkeyMap.imt. This file maps music track id numbers (which are referenced from the Lua scripts) to sound files, and specifies some parameters for each track. One of these parameters is called “sync”, and has an integer value associated with it. A value of 0 seems to indicate that the track should never be synced. Otherwise, if the sync value matches the sync value of the track that was playing previously, the track will start from the same position where the previous track was at the time of transition.

Another improvement to music playback is when the music track changes, the music now transitions with a linear cross-fade, instead of instantly cutting to the new track. This makes the transitions sound less abrupt, and matches the behavior of the original game.

After writing the previous blog post about shadows, I discovered that my shadow implementation caused a crash in the House of Sticks at Lucre island. The format of the shadow data in this particular set did not seem to match the format I described in the previous post.

It turned out that my interpretation of the shadow data was wrong. The shadow color is not part of the ‘sector reference’ structure as I thought before. Instead it is part of the ‘shadow’ structure. The color is stored after the array of ‘shadow reference’ structures.

It appears that in most cases a ‘shadow’ structure only contains one sector reference. This is why my initial implementation seemed to work. However, the House of Sticks set file contains a ‘shadow’ structure that references multiple sectors, and in this case my implementation tried read too much data, since it attempted to read a color for each sector reference.

This discovery indicates that a single shadow will always have a uniform color. However, if multiple shadows are drawn on top of each other, this can give the appearance of a single shadow with multiple colors. This happens for example in the Place of Prostheses.

Since the shadow color is uniform, rendering the shadows is simplified. PR #956 now contains much less changes to the renderer code, as the EMI shadow rendering is now very similar to that of Grim.

Feeling relaxed after a week of vacation I’m now finishing up work on implementing shadows for EMI. This work is now in PR #956.

EMI has two types of shadows: simple shadow sprites (internally called “lame shadows” or “dumb shadows” 🙂 ) and planar projection shadows. The projection shadows look more realistic, but are only used in a few sets. If shadow quality is set to low in the options, only shadow sprites are used.

The screenshots below are taken from the original game. In the screenshot on the left, shadow sprites can be seen under Guybrush and Elaine. The screenshot on the right shows projection shadows cast by Guybrush and the brazier on the deck of the ship. Initially with ResidualVM both shadow types were not drawn at all.

After my previous sprite fixes, enabling the shadow sprites turned out to be very simple. Previously ResidualVM’s actor drawing code contained a hack that skipped drawing the costume containing the shadow sprite if shadows were not explicitly activated for that actor with a call to ActorActivateShadow from Lua. After some testing in the original game I found out that the ActorActivateShadow function doesn’t affect the shadow sprites at all in the original (instead it is used to toggle projection shadows). After removing the hack, the shadow sprites showed up as expected.

I then moved on to tackle the projection shadows. This turned out to be slightly trickier. Fortunately projection shadows have been implemented previously in ResidualVM’s renderers for Grim Fandango, so most of the rendering code could be reused also for EMI. To render projection shadows we need a shadow color, the light position as well as the planes on which shadows are cast on. The first problem was that I didn’t know where the original game got this information from.

In Grim, the shadow color, the light position and the shadow planes are set up through Lua code, but in EMI I couldn’t find anything similar in the game’s Lua scripts. The information had to be stored somewhere else.

The next place I decided to look in was the .setb files, which contain scene-specific information such as camera positions, sectors and lights encoded in binary form. The shadow color seems to change between sets, so it seemed like a fair assumption that the information I was looking for would be stored in the set data.

This led me to the discovery that for sets that have projected shadows enabled, there was indeed some data at the end of the .setb file that was not read by ResidualVM (marked with red in the pic above). This previously ignored block of data contains an array of structures that I call ‘shadow’ structures here. Each ‘shadow’ structure contains a name, the light position to be used when projecting the shadow and an array of ‘sector reference’ structures. The ‘sector reference’ structure contains the name of a sector within the set as well as the shadow color to be used when a shadow is projected onto that sector.

Once I discovered this data, it was fairly easy to enable the pre-existing planar projection shadow rendering code in ResidualVM also for EMI. The main difficulty was that previously the renderers only supported one global shadow color, but in EMI the shadow color can be set per-sector. I’ll discuss this issue and how I resolved it in detail in the next blog post.

With all the issues resolved, it’s very hard to tell ResidualVM and the original game apart in most scenes. The screenshots below are from ResidualVM. Compare them with the screenshots of the original above 🙂

For the past week I’ve been working on enabling head tracking. Head tracking allows actors to orient their head towards any point in the game world. This work is now included in PR #947.

The head tracking works by reorienting one of the bones of the skeleton after keyframe animation is applied. The bone is oriented to face the direction the actor wants to look towards. The look direction is interpolated, so the change in orientation is not instant.

The original Lua code uses the functions ActorSetHead and ActorSetHeadLimits to set up head tracking. The former takes as parameter the name of the bone that should be animated, while the latter takes three parameters that specify horizontal and vertical angle limits. The angle limits ensure that the head won’t be oriented unrealistically.

My current implementation looks good in most cases, but I did find one case that looks noticeably different from the original. Mr. Cheese’s head is at a weird angle when he looks towards Guybrush in the Scumm Bar. The screenshot on the left is ResidualVM, while the one on the right is the original.

The head tracking animates Mr. Cheese’s neck bone, but the neck bone has children which are animated by keyframe animation, and the vertices of the head mesh follow the child bones. Although the neck bone is facing Guybrush, the head mesh is not, because of the animation applied to the child bones.

I spent a significant amount of time trying to figure out the difference between my implementation and the original during the week, but none of my solutions produced the same result as the original in this case.

One of my attempts was to override the animation of the neck bone as well as it’s children bones completely whenever head tracking is active. This way the head was oriented exactly towards the point the actor looks at. That, however, didn’t match the original game either. Mr. Cheese’s head does seem to be affected by the animation also in the original.

For now I decided to leave this be. Mr. Cheese is currently the only case I’m aware of where the result is different from the original. I may return to this at a later stage once I’ve finished the other remaining tasks.

My recent sprite fixes PR includes a change that fixes the appearance of Stan (see the GitHub issue here). I found the method they used to produce the unmoving plaid effect quite interesting, so here’s a quick summary.

This is the final composition for reference. Note Stan’s trademark: the pattern on his jacket does not obey perspective. Although the effect looks simple, achieving this result with fixed-function OpenGL is actually not very straightforward. Let’s see how this image is produced.

If we render the first background layer only, we can make an interesting observation. A rectangle covered with the check pattern of Stan’s jacket is part of the background. It covers all of Stan’s screen-space area. Stan moves back and forth during dialogue, but never leaves the area covered by this rectangle.

Here is a render with only the character models on a black background. We can immediately see that the jacket is not part of the model. Instead, Stan has a fully transparent texture where the jacket should be, so we can see through him. The transparent jacket is still drawn to the depth buffer, though.

If we combine the first background layer and the character models we get a result like above. Since Stan’s model is partly transparent we can see the check-patterned background through him. Obviously something still needs to be done about the rest of the rectangle surrounding him.

Above is an additional render with the rest of the background layers enabled. The desk is part of a background layer that is drawn over Stan, covering his legs. This image is similar to what ResidualVM produced before my fixes.

Here’s the trick. A sprite is rendered behind Stan, but over the background layer. I highlighted the sprite with red color in the above screenshot to make it stand out from the rest of the background. The sprite contains a chunk of the “real” background without the check-patterned rectangle. Since the sprite is drawn with depth test enabled, the sprite only covers Stan’s surroundings but not Stan himself.

This sprite was previously not drawn at all in ResidualVM, so the check-patterned rectangle was not replaced by the correct background around Stan.

This method works nicely since the backgrounds are static and the camera never moves. Other than a bit of overdraw, there’s also no additional rendering cost.

After finishing the lighting task, I had some time to focus on fixing bugs before moving on to my after mid-term tasks.

One of my GSoC mentors, chkr brought up a bug regarding the candles in Governor’s mansion. In ResidualVM, the candle sprites didn’t show up at all. Look at the chandelier in the screenshots below to see the difference. ResidualVM on the left, original on the right.

This seemed simple enough to quickly fix before moving on to other tasks I thought, so I got to debugging.

When looking at the original Lua code that sets up the candles, my first observation was that the candles were first being attached to the chandelier actor and their position was set afterwards. Perhaps something was going wrong with the offsetting of attached actors even after JoseJX’s recent fixes? This turned out to be a false lead when I noticed that no calls were being made to sprite drawing functions at all for the candle sprites. Even if the sprites were positioned out of view, the current ResidualVM implementation should attempt to draw them. Later it turned out that the candle actors were in fact in the correct positions.

In the EMI engine, sprites are components of chores. A chore is basically a track of instructions for the components that are executed in sequence. Only the components of active, playing chores are updated and drawn. Chores are components of costumes, which in turn are components of an actor. I found out the candle actors and their costumes were present like they should be, but the costumes were not playing any chores with sprite components, so no sprites were drawn.

This seemed weird to me. The Lua code is supposed to start the “flame12fps” chore for the candles, but the call to start the chore was never made to ResidualVM. The next step was to figure out why.

The original Lua only starts the chore in the case if no chores are already playing. But surely there shouldn’t be any chores playing if the actor was just created? Well, it turned out that when the game is run in ResidualVM, there is.

By adding a breakpoint to the IsChorePlaying engine opcode that the check in Lua calls for each chore, I found out that the opcode returns true because the “wear_default” chore of the costume “dumbshadow” is playing. Because of this, the sprite animation is never started.

In the original engine the chore is not playing. This is easy to check by running custom Lua code with Ctrl+E after enabling the hidden debug functions in the game (I could explain how to do this in a later blog post). If the chore is forced to play, IsChorePlaying will also return true in the original engine, so there is no special handling in the engine to always return false for this particular chore.

Ok, so what is the “wear_default” chore of “dumbshadow” and why is it started then? The “dumbshadow” costume is used to produce fake shadow effects such as the ones that can be seen under Guybrush and Elaine in the above screenshot of the original game. The shadows currently don’t show up in ResidualVM, but even if they did, we probably wouldn’t want the candles to have shadows. Thus it seems likely that the correct behavior would be to not activate the shadow, like the original engine seems to do.

The Lua code that starts the shadow chore can be found in _actors.lua. The chore is started as soon as the actor is put in a set, if certain conditions are fulfilled. Among these conditions is one peculiar one. If the opcode GetActorPuckVector returns nil, the shadow won’t be activated. When GetActorPuckVector is run on the candles in the original engine, the returned value is nil. In ResidualVM the opcode always returns a non-nil value.

Finally we’re getting to the gist of the bug, after seemingly drifting so far. GetActorPuckVector is an opcode used to get the forward vector of an actor projected to the actor’s current sector (or walkplane) to be used for movement. It appears that in the original engine the GetActorPuckVector opcode always return nil until a call is made to SetActorFollowBoxes, which constrains the actor’s movement to the walkable sectors. Most likely the purpose of the nil-check is to only activate shadows for actors that are constrained to walkable sectors.

Replicating this behavior in ResidualVM’s implementation of GetActorPuckVector fixes the candles, as expected. Phew!

One more quick post about the lighting modes before I move on to other subjects.

Looking at the comparison shots in the previous post gave me an idea about the FASTDYN lighting mode. The FASTDYN lighting appears smoother when compared to NORMDYN, and it seems to also light the back side of models slightly. These properties reminded me of the Half Lambert lighting technique used in the Half-Life games.

I decided to quickly implement the technique to see how it compares with the original. Surprisingly in some cases the result is almost an exact match, but in others the shading seems too bright. For example, see Otis  in the screenshots below. Left is the original renderer (FASTDYN mode), and on the right is my implementation with the Half Lambert technique.

The lighting is close to the original when Otis is at about a 45 degree angle to the light coming from the windows. However, if we rotate Otis slightly this no longer is the case.

Otis is now at about a 90 degree angle to the light, but the shading seems much darker in the original. If we rotate Otis 45 degrees further, the lighting is once again very close to my Half Lambert implementation.

Otis is now facing away from the light coming from the direction of the windows.

This behavior seems fairly strange to me. There must be some additional term in the FASTDYN lighting model that I’m not aware of yet.

I may return to this at a later time, but right now I’m happy with my current implementation that matches the NORMDYN mode of the original game exactly. This was just recently merged to ResidualVM’s master, so give it a try.