Month: July 2025

This week I worked on a bunch of small bugs with the saving Director Movies PR, and the movie cast member. This was one of the least productive weeks up till now.

The save files in ScummVM are stored at a specific location given by the configuration manager at
ConfMan.getPath("savepath")
This is the only directory that is writable. The game directories are not writable. The save file needed to be stored with the format:
<target-name>-<savefile-name>.dir
However, since the file is now stored at a location different than the _gameDatadir, while loading the movie, we need to add the movie(s) in the SearchSet which is a list of all movies in the game directory sorted by priority of loading. The priority of the saved movies should be greater than movies in the game directory. Which was an easy enough of a concept to understand. However, Director engine already has an implementation for game quirks, in the CachedArchive which tries to load game quirks (specific archives, text files). @sev wanted an implementation similar to that. But, instead I spent a lot of time trying to implement my own approach. That resulted in a lot of time wasted. Finally, @sev intervened and explained to me in detail what was to be done.

The method was to create a separate sub class of class Archive, named SavedArchive and store a hashmap of expected paths (e.g. INTRO.DIR) mapped to saved movie paths (e.g. tkkg1-win-INTRO.DIR) and save this archive with higher priority in the SearchSet at the startup of the engine. So, when asked to load a file, the SearchSet first looks in the saved archive before checking the game files.

Apart from that there were a bunch of small fixes to the PR:
– Instead of saving the _mainArchive, save the _currentArchive
– Start writing multiple casts, and their cast members
– Delete the generated resources to avoid memory leaks
– Implement the stubbed computeChecksum() function
– Write external bitmap resource correctly with zero size
– Create a separate file `archive-save` for the saving code

Also, I opened a very small PR for fixing a bug while duplicating  a cast member from a one cast to another. The problem was that we were looking for the source cast member in the target cast.

On the movie cast member task, I made little progress. Until last week, I was processing lingo scripts for the movie cast member correctly. I got to testing trektech-win this time, and it turns out this was not enough. The game was crashing as soon as the movie cast member finished loading and processed its first script. The reason was that in Director, the lingo execution is global. The movie cast member shares the global variables of the main movie. Hence, the scripts of the movie cast member must also be processed as if they are part of the overall lingo execution (with reference to the movie cast member). Hence, Window class (which processes input events and holds the lingo state) and lingo processing are too intricately tied together. Creating a separate SubWindow for the movie cast member messes this up. After the movie cast member jumps to a frame, returns from Lingo::execute() it freezes its state as follows:

After freezing, it switches the lingo of the global window.However, since we’re processing the movie cast member, the lingo state of the movie cast member’s window should be the one to freeze. But instead it freezes the main window. This results in numerous segmentation faults. This will require in-depth knowledge of the lingo execution, hence instead of coming up with a wrong way to solve this problem, I asked @sev to switch me to another task.

One thing that I did end up fixing was when we switch the movie in the movie cast member, the previous movie needed to be deleted before loading the next movie.

After that, yesterday @sev gave me a set of three new tasks. Out of which, the first one was that one of the tables in the ImGui visual debugger for Director was getting generated without IDs causing a runtime error. I couldn’t reproduce the bug,  so I moved to the second task.

The second task was that the lingo scripts being dumped with --dump-scripts flag on, were misnamed. The problem was that we were writing filenames as: <movie-name>-<scriptType>-<castId-of-script>.lingo But the castId of the script was being written wrong. Instead of 1 we had 65537 (65536 (2^16)+1), instead of 2 we had 65538 (65536 (2^16) + 2). I suspected that the castId must be being read incorrectly. Sure enough, the castId was being read as: /* 44 */ castID = stream.readSint32BE(); Upon inspecting the hex dump of the movies I found that, the castId was a 16 bit-entry rather than 32-bit. The 16 bits before that were unknown and changed between 0x1 and 0x0.

D5 MovieD4 Movie

In each of the screenshots, the 32 bits being read previously as castId.
Hence, it needed to be changed to
/* 44 */ unk43 = stream.readSint16BE();
/* 46 */ castID = stream.readSint16BE();
Which works perfectly.

Despite this, overall, it was a pretty disappointing week. It’s ironic that this is the longest blog I’ve posted till now (by word count).

@sev also pointed out that I’m not reporting my progress regularly, which I intend to improve upon in the upcoming week. Hoping for a better next week.

This week I made a bunch of minor fixes to the movie saving functionality, that was almost done last week. By that time, the movie was being saved, with correct offsets and sizes, the movie was loading but with some issues, e.g.
1. Losing the formatting of the ‘STXT’ text
2. Last few pixels of the bitmap not being saved, etc.
Fixed those one by one.

I worked with workshop movie saveMovie amongst the dictionary movies for Director 4/5 M/W (Those for Windows Director 5 are missing), made sure that that was working properly and made the pull request open for review. I am fairly confident that it should work properly but I need to make sure of that by testing with actual director targets. I’ll be doing that this week.

After this, @sev reassigned me the task of loading movie cast members. The loading of movies is already there (exactly same as the main movie) and also rendering exactly the same as filmloops (already supported in ScummVM Director). Plus the progress from the first two weeks isn’t all to waste.

However, much of the refactoring I did, separating Lingo Context Keeper from Window, had to be scrapped. @sev proposed a new solution of creating a subclass of the Window class and create the movie by passing an instance of this subclass for all the event/lingo processing. All the rendering methods will be overwritten by this subclass to do nothing. Separating all the rendering functionality from the event processing in Window will accomplish very little. Hence the workaround.

With this method, I was successful in getting the linked movie to load. However
1. It was not showing up in each frame
2. It wasn’t processing its own scripts (acting like a filmloop, looping frames)
3. It was overwriting the palette of the main movie.

To over come the 3rd problem I had to add a boolean _isCastmember to the Movie class and check it before setting the palette for the entire window. But now this results in the linked movie itself not having any palette, but I think the solution is easy, we just have to somehow set a palette for the movie’s bbox.

For the first problem, I had to write a isModified() function for the movie cast member (similar to all the other cast members) which marks the movie dirty for re-rendering.

The second problem however, was especially tricky. Because the lingo Scripting in ScummVM director is global, it only processes the scripts of the movie given by g_director->getCurrentMovie(). To solve this, I had to set the linked movie as the current movie using g_director->setCurrentMovie(_movie), before stepping its Score.

While loading also it processes the event kEventMovieStart, where I had to make sure that the correct movie is processing the event. But finally I got movies to work with scripting (at least for my test movie).

Overall, seems I’m on track to correctly load movie cast members. This week I’ll actually test this on the ‘trektech’ target, and try to progress it as much as possible.

This week my seventh semester started. So, there was a lot of stuff with credit registration and Hostel allotment, etc. So, couldn’t give the usual amount of time to ScummVM. From next week I’ll be consistent again. I also passed my midterm evaluation this week. Thank you, @sev and everyone for helping me, would’ve been lost without them.

This week was the most productive out of all the weeks until now (or maybe it feels like it). This week, I combined everything together. Until now, I was writing the logic of each resource to be written separately, dumping the file in the ./dumps folder and writing only that chunk in that file. So, I was checking the loading of only one resource at once. I wasn’t even recalculating the offsets.

This was good for testing but for the final writing, I needed to recalculate the sizes and the offsets of the resources. Which is what I did this week.

First thing was to remove the STUBbed saveMovie lingo command. Then I had to rebuild the RIFXArchive::_reosurces array which contains the struct Resource (size, offset, flags, index, castId, etc.) of each  resource. I added any new cast members to the resource list along with their children. I was under the impression that each cast member has separate children. Which made me think I’ll have to rebuild the indices of the cast members as well. But that is not the case: one child resource can have multiple parent resources (e.g. one ‘BITD’ can be child to multiple ‘CASt’ resources). This parent-child relationship also needs to be updated in the RIFXArchive::_keyData. My first approach was to rebuild the existing _resource array, but I quickly realized that I’m writing over important data here. So, I switched to building a new resource array and passing that to different functions. The bare bones of the writing of Memory map (‘mmap’), key data (‘KEY*’) and Cast data (‘CAS*’) were already done.

At first, there were a lot of issues with this. The data was being written correctly but the offset was wrong, the size returned by my function: getSCVWResourceSize() was off by two bytes. So, the entire loading of filmloop was failing. Also while writing ‘BITD’, ‘CLUT’, ‘STXT’ and ‘SCVW’ resources, I had to first find their parent cast members, which wasn’t being handled correctly. I also later realized that ‘BITD’ writing for 1/2/4bpp was off by one pixel. While reading the cast info in the ‘CASt’ resource, the strings: name, filename, directory name and type are all pascal strings. So, their first byte is not read (that byte marks the lengths of the string, which is redundant, we already have their length, so we ignore it). This caused the written cast information to be missing the first byte. The indices of ‘CASt’ resources written in the ‘CAS*’ were not written in the correct order, causing them to have the wrong CastIds. There was also an issue where if the lingo script goes saveMovie "filmloop_saved.dir", at first I was trying to write the file by constructing a path like Common::Path("filmloop_saved.dir"); but later realized that it also needs a parent directory, so started saving like Common::Path("./" + "filmloop_saved.dir");

Through all of this (and many more issues), I was finally to sort each issue one by one, and finally write the movies correctly. Now, the saveMovie {argument} lingo command saves the exact copy of the currently loaded movie to the path specified by the argument.

After this, I quickly finished writing score (very similar to Filmloop) and Rich Text.  This marks writing of all the modifiable resources in Director (that I know of). I have to check a few things before calling this ‘Done’. What happens when I try just duplicate cast "Original Cast" and if I call it repeatedly, will it be able to write all the duplicated casts? I also need to play around with puppetSprite. Also externally linked bitmaps are something to look into. @sev suggested trying out actual games that use this functionality to make sure it is fully functional.

This task is nearing its end. I hope to complete it soon. After that I want to work on my initial task of loading movie cast members.

Continuing from last week’s progress, this week I worked on saving some more Cast related resources: palette data (‘CLUT’ resource), bitmap data (‘BITD’ resource), text data (‘STXT’ resource) and the filmloop data (‘SCVW’). Most of the time was spent on testing these implementations. A test movie for each was made in Basilisk emulator (for Mac) and 86Box emulator (for Windows) using original Director 4 and Director 5.

The ‘CLUT’ resource was straightforward, since it only stores the colors in the palette as RGB values in bunches of 3. But we convert the 16-bit colors to 8-bit colors by only keeping the upper 8 bits only. Hence while saving these colors back, I had to left shift the byte by 8 bits and save it back.

The ‘BITD’ resource and ‘STXT’ resources were particularly tricky.

The ‘STXT’ resource contains the text data for Buttons and Text Fields. But the text is encoded in font encodings like Mac Roman and Mac Japanese for Mac or Windows-932 for Windows. This is specified in the ‘STXT’ resource. We detect the encoding, extract the part of the text that is encoded and convert it to a U32 string. While saving back however, we need to save the text in its original encoding. A single text resource can be encoded in more than one formats and at random offsets even. Thankfully, we do not ignore this information, we store it in a separate string as a header, like follows:
Common::String format = Common::String::format("\001\016%04x%02x%04x%04x%04x%04x", _style.fontId, _style.textSlant, _style.fontSize, _style.r, _style.g, _style.b);
_ftext += format;
The task was to read the header \001\016 in this string and write the font style that followed. Encode the U32 string back into the encoding given by _fontId, and write the raw string.

In case of ‘BITD’ resource (bitmap data), depending on the number of bits per pixel, we load it differently, naturally. The bitmap may be external or internal. @sev told me to focus on the internal loading for the moment. Most movies have their bitmap data compressed, i.e. instead of saving the data pixel by pixel RGB values, they store R/G/B values together,  hence if neighboring pixels have the same R values, instead of writing the said R value n times, we store the R value and the number of times it is repeated – n, which is Run Length Encoding for compressing bitmaps. However, I’m ignoring that for now, and storing the pixels in a row. Testing this was a struggle and a half. I had to create movies with bitmaps in all formats 1bpp, 2bpp, 4bpp, 8bpp, 16bpp and 32bpp and test each one separately.

Lastly, I worked on writing filmloop data which is the same as the score (‘SCVW’ resource). The resource stores each frame in the filmloop, its sprite channels and the associated cast member id. Since filmloop don’t have their own cast members, but use the cast members loaded in the cast, this was easier to save than the rest of the resources above. I simply had to write it in a format that is recognizable to the Director engine. This still took some time to write and test since the data loading for filmloops for D4/D5 is slightly different.

There is a slight recurring problem that the sizes of the resources need to be calculated before writing the resource itself because, the size precedes the data. This could be avoided by calculating the size as we write it, but that requires storing the offset of where the size is supposed to go, calculating the size, jumping to the offset, writing the size and then jumping back to the current position, which is an ugly solution.

Overall, the progress this week was good, but I’m worried whether I’m taking too much time testing. The midterm evaluation is coming up in 7 days, and I’m unsure I’ve done enough to count it as a success. I hoped to have completed this task by the midterm, but… Alas!