Author: superlemon

This week again, I continued working on the ImGui visual debugger in Director, mostly working on small bugs and broken features caused by the work in the previous weeks.

The Execution Context window shows the lingo scripts being executed. To make it show multiple handlers from the same script at once, I had made changes that broke the navigation. Stepping in, stepping over and stepping out in the control panel didn’t work properly, jumping multiple instructions at once. The problem was that startOffsets of the script weren’t getting stored, since I needed to render multiple handlers, each with a separate ImGuiScript object. These startOffsets store the offsets of each line of script. While stepping in/over/out, the lingo program counter is compared with these to find out which line is getting executed at the moment.  I had to make sure that the ImGuiScript object associated with handler currently being executed persisted and was passed by reference to the renderScript() function so that it will store the startOffsets properly. Also I had to make sure that the _script->_dbg._isScriptDirty flag is set after all the scripts are rendered. Previously it was set to false after rendering the first script.

In the lingo scripts in the Execution Context window, if there is a call to a function, you can press the function to go to its definition. However, the search for the handler was flawed. The value we were passing to the function that fetches the ImGuiScript as the castId of target script, was wrong. It took me some time (and some messing around with ProjectorRays) that this is the Id (or index) of the script (in cast of a localcall, this is the index of the script in the script context (consisting of scripts in the same file/’Lscr’ resource), whereas in an extcall it was the index of the script in the global context (consisting of all scripts)).

This is obviously wrong, hence I had to find out the actual castId of the script which contains the handler, and pass that, which worked as expected.

During this, I also realized that for some reason, some of the scripts weren’t showing up in the debugger at all, apparently for no reason at all. Upon some investigation, I found out that the way the scripts were fetched was, that the LingoDec decoder reads these scripts and the corresponding data. One of which was the castId of the script, which it turns out isn’t reliable. It may be wrong, it may have duplicates, can’t be trusted. Hence, instead, from the HashMap that maps the castID of a cast member to its script id, I fetch the castID and use that to fetch the script, which worked like a charm. This PR should explain the issue in detail. This also solved an issue where, when asked to dump the scripts, the names of the files dumped and the scripts didn’t match at all.

I had to also solve scrolling issue in the Scripts window. After jumping to a handler, if the script contained only one handler, the scrolling worked fine, but in case of multiple handlers, it used to jump to the last handler. Had to introduce a _scrollTo flag which was set only when rendering the current script.

I also got rendering of text and shape previews this week. In order to render them, I was at first checking which channel had the text/shape to be rendered, and pass that channel and a ManagedSurface to Window::inkBlitFrom(), and showing the surface in the preview. However, obviously, this meant that if a shape/text is not on the stage at the moment, it won’t show a preview. Hence, I had to prepare a dummy channel with the correct sprite and relevant properties (e.g. thickness, foreground color, background color, pattern, etc.) and then blit it to a temporary surface for preview. This honestly took some time to figure out.

Other than this, there were some minuscule things, too unnecessary to mention here.

Hence, overall a pretty normal week again. Next week is the final week for coding after which there is the final submission. I hope to have done enough to count the project as a success.

This week also brought good news in the form of a job offer. In most engineering colleges in India, recruiters visit the college to hire final year students as graduate trainees. I was lucky enough to be selected at one such offer. Now that I have this security, I can continue to contribute to ScummVM even after GSoC. Not as regularly but over weekends and holidays. Working on ImGui has been fun but I wanted to directly work on making a game work in ScummVM’s Director, similar to how @rvanlaar was working in SafeCracker. Couldn’t unfortunately succeed at that during GSoC but I’m planning on doing so afterwards. Looking forward to that!

This week, I continued my work on Director ImGui debugger from last week. Working on ImGui has been bit of a fun exercise since, it is an open canvas. There are a lot of opportunities to improve upon, and it all depends on what a developer might find useful while working on Director engine.

First of all, there were a bunch of impurities with the ImGui debugger. e.g.

• Many child widgets have the same problem as the breakpoint window error pop-up from last week. i.e. widgets having the same label and id causing the ImGui throw an pop-up error saying changes made to one widget will be reflected to the other. I solved the same problem in the following places:
• i.   Vars Window
  ii. Watched Variables
  iii. Scripts handler widget
• I also improved the search for handler in the getHandler function. Instead of searching for the handler in all casts, we can single out the cast that has the handler in it.
• When displaying Prop list in the execution handler window, there was a missing line, which I promptly added.

And a bunch of other small stuff, too small to be worth noting here.

Another feature that was stubbed in the debugger was being able to load and save the current state of the debugger into a persistent file. So, we can save a position in the game we are currently working on, and load the same position every time we start  a debugging session.
We went with the JSON format to store the state. We are currently storing the following things (and loading them back):
1) Window Positions
2) Score frame number
3) Windows that are open/closed
4) Global, Local and Watched Variables
5) Breakpoints
6) Log
7) Colors
Here, @OMGThePizzaGuy and @lephilousophe showed me `ImGui::SaveIniSettingsToMemory/ImGui::LoadIniSettingsFromMemory` which made my job much easier.

In the Watched Vars window, I am also showing Local Variables and Global Variables with different colors. On top of that, Local Variables that are out of scope are shown with different color.

All variables are now shown with `renderVariable()` which allows variables to be clicked on to add them to the WatchedVars list and also show their current value.

Previously, the Scripts window only showed the handler being executed at that moment. However, for better navigation, @sev suggested that all the scripts from the same lingo context should be shown in the same window. This took me a while. Since, now not only was I rendering multiple (ImGui) widgets for multiple (Director) windows, I was also showing rendering multiple handler in the same Scripts widget. This caused a bunch of problems with stepping over e.g. pressing step over caused the script to move multiple steps forward, and with showing the current line in the handler (with a little yellow arrow), because the _state->_dbg._isScriptDirty was getting set to false immediately after the first script was rendered. Also, the scroll to the current line was also not working. This took some time and debugging to fix.

When @sev started review on my PR, he immediately pointed out that I was making a separate ImGuiState::state for each window, which in hindsight I realized was a mistake. So, I reworked it to only make a separate instances of the objects that are needed to hold the execution context of a window. This also took a while to get right. There was a bunch of unexpected behavior including, the scripts were not showing up the first time the ‘step’ button was pressed in the control panel.

I also fixed a deprecation warning that occurred while compiling the Director engine. I had to create a custom copy constructor for the MacWindow class since the synthesized copy constructor was calling a deprecated copy constructor for the ManagedSurface class. Instead I had to use a copyFrom method as suggested.
There was another instance of use of deprecated method operator= for the ManagedSurface class, used copyFrom there as well.

Overall, a somewhat normal week. I hope to make even more improvements to the ImGui debugger. e.g. Adding a trace for watched variables is next on the list.

This week I worked on some improvements on the ImGui visual debugger in the Director engine created by @sev. It allows us to see the lingo scripts being processed, cast members and their properties, score and a bunch of other stuff.

There was an extremely minor problem with the scripts window. The script window allows us to put breakpoints into the movie. However, the check boxes for the breakpoints were all being rendered with the same ID. ImGui has an label+ID system to name UI elements. Each new UI element must have a unique label+ID combination. Which was a simple mistake, rectified immediately.

The second thing was in case there are multiple windows, i.e. there are (Director) windows present other than the stage, then the ImGui just clubs all of the scripts being processed into a single (ImGui) Scripts window. So, we need to have a separate pane for the script being processed in each (Director) window.

For this, my idea was to create a separate `ImGui::_state` for each (Director) window and store them in a hashmap. Each time `showExecutionContext()` is called, we take the `ImGui::_state` for each window, and render them.
_state = getWindowState(stage);
updateCurrentScript();

Although, I ran into a multiple problems:
1. The execution context (ImGui) window once turned on, should stay turned on, but it was disappearing as soon as the (Director) window changes.
2. The control panel was debugging the script for the wrong (Director) window.
3. The other (ImGui) windows were being turned off whenever the (Director) window changes.
4. The multiple windows were showing the same script because the LingoState wasn’t changed in the global lingo context g_lingo.
5. The other panes were not showing any scripts because the `getHandler()` function wasn’t able to find the lingo script with the correct cast ID.
Solved these one by one and created a pull request yesterday.

Also, the callstack is currently a separate (ImGui) window. We also need to combine that with the scripts into a new (ImGui) window named the Execution Context. I did the refactoring. The Execution Context now looks like follows:

There was also a minor font rendering issue in the Score (ImGui) window. Apparently there were multiple breaking changes in the 1.92.0 update for ImGui regarding Font changes. However, it didn’t take long to figure out that it was just a one line change. I just had to set the `ImGui::IO::DefaultFont`, which is NULL by default. Previously when called:
ImGui::PushFont(ImGui::GetIO().DefaultFont)
The PushFont function would see that DefaultFont is NULL, and automatically push the first font added to using ImGui::AddTTF****. However, after the update, if the parameter to ImGui::PushFont is NULL, the font is unaffected and only the font size is changed. Hence, I had to manually make sure that ImGui::IO::DefaultFont is not NULL.

Overall, I didn’t feel stuck at any point. The process of figuring this out was smooth. However, this still took me a week to finish because I took an off day in the middle of the week. Also, due to my house hunt (which was a pain and a half) I used to start late @10PM IST and get tired by 2AM IST and doze off! Although I had written about it in my proposal, that I’ll have less time once my college starts, this is less than I had anticipated. This makes me worry about my GSoC project’s success. This week will be better, now that I (hopefully) have a stable abode. I’ll start consistently @6PM IST and put in 6-7 hours before calling it a day.

Here’s to hoping for a better week (again)!

This week I worked on a bunch of small bugs with the saving Director Movies PR, and the movie cast member. This was one of the least productive weeks up till now.

The save files in ScummVM are stored at a specific location given by the configuration manager at
ConfMan.getPath("savepath")
This is the only directory that is writable. The game directories are not writable. The save file needed to be stored with the format:
<target-name>-<savefile-name>.dir
However, since the file is now stored at a location different than the _gameDatadir, while loading the movie, we need to add the movie(s) in the SearchSet which is a list of all movies in the game directory sorted by priority of loading. The priority of the saved movies should be greater than movies in the game directory. Which was an easy enough of a concept to understand. However, Director engine already has an implementation for game quirks, in the CachedArchive which tries to load game quirks (specific archives, text files). @sev wanted an implementation similar to that. But, instead I spent a lot of time trying to implement my own approach. That resulted in a lot of time wasted. Finally, @sev intervened and explained to me in detail what was to be done.

The method was to create a separate sub class of class Archive, named SavedArchive and store a hashmap of expected paths (e.g. INTRO.DIR) mapped to saved movie paths (e.g. tkkg1-win-INTRO.DIR) and save this archive with higher priority in the SearchSet at the startup of the engine. So, when asked to load a file, the SearchSet first looks in the saved archive before checking the game files.

Apart from that there were a bunch of small fixes to the PR:
– Instead of saving the _mainArchive, save the _currentArchive
– Start writing multiple casts, and their cast members
– Delete the generated resources to avoid memory leaks
– Implement the stubbed computeChecksum() function
– Write external bitmap resource correctly with zero size
– Create a separate file `archive-save` for the saving code

Also, I opened a very small PR for fixing a bug while duplicating  a cast member from a one cast to another. The problem was that we were looking for the source cast member in the target cast.

On the movie cast member task, I made little progress. Until last week, I was processing lingo scripts for the movie cast member correctly. I got to testing trektech-win this time, and it turns out this was not enough. The game was crashing as soon as the movie cast member finished loading and processed its first script. The reason was that in Director, the lingo execution is global. The movie cast member shares the global variables of the main movie. Hence, the scripts of the movie cast member must also be processed as if they are part of the overall lingo execution (with reference to the movie cast member). Hence, Window class (which processes input events and holds the lingo state) and lingo processing are too intricately tied together. Creating a separate SubWindow for the movie cast member messes this up. After the movie cast member jumps to a frame, returns from Lingo::execute() it freezes its state as follows:

After freezing, it switches the lingo of the global window.However, since we’re processing the movie cast member, the lingo state of the movie cast member’s window should be the one to freeze. But instead it freezes the main window. This results in numerous segmentation faults. This will require in-depth knowledge of the lingo execution, hence instead of coming up with a wrong way to solve this problem, I asked @sev to switch me to another task.

One thing that I did end up fixing was when we switch the movie in the movie cast member, the previous movie needed to be deleted before loading the next movie.

After that, yesterday @sev gave me a set of three new tasks. Out of which, the first one was that one of the tables in the ImGui visual debugger for Director was getting generated without IDs causing a runtime error. I couldn’t reproduce the bug,  so I moved to the second task.

The second task was that the lingo scripts being dumped with --dump-scripts flag on, were misnamed. The problem was that we were writing filenames as: <movie-name>-<scriptType>-<castId-of-script>.lingo But the castId of the script was being written wrong. Instead of 1 we had 65537 (65536 (2^16)+1), instead of 2 we had 65538 (65536 (2^16) + 2). I suspected that the castId must be being read incorrectly. Sure enough, the castId was being read as: /* 44 */ castID = stream.readSint32BE(); Upon inspecting the hex dump of the movies I found that, the castId was a 16 bit-entry rather than 32-bit. The 16 bits before that were unknown and changed between 0x1 and 0x0.

D5 MovieD4 Movie

In each of the screenshots, the 32 bits being read previously as castId.
Hence, it needed to be changed to
/* 44 */ unk43 = stream.readSint16BE();
/* 46 */ castID = stream.readSint16BE();
Which works perfectly.

Despite this, overall, it was a pretty disappointing week. It’s ironic that this is the longest blog I’ve posted till now (by word count).

@sev also pointed out that I’m not reporting my progress regularly, which I intend to improve upon in the upcoming week. Hoping for a better next week.

This week I made a bunch of minor fixes to the movie saving functionality, that was almost done last week. By that time, the movie was being saved, with correct offsets and sizes, the movie was loading but with some issues, e.g.
1. Losing the formatting of the ‘STXT’ text
2. Last few pixels of the bitmap not being saved, etc.
Fixed those one by one.

I worked with workshop movie saveMovie amongst the dictionary movies for Director 4/5 M/W (Those for Windows Director 5 are missing), made sure that that was working properly and made the pull request open for review. I am fairly confident that it should work properly but I need to make sure of that by testing with actual director targets. I’ll be doing that this week.

After this, @sev reassigned me the task of loading movie cast members. The loading of movies is already there (exactly same as the main movie) and also rendering exactly the same as filmloops (already supported in ScummVM Director). Plus the progress from the first two weeks isn’t all to waste.

However, much of the refactoring I did, separating Lingo Context Keeper from Window, had to be scrapped. @sev proposed a new solution of creating a subclass of the Window class and create the movie by passing an instance of this subclass for all the event/lingo processing. All the rendering methods will be overwritten by this subclass to do nothing. Separating all the rendering functionality from the event processing in Window will accomplish very little. Hence the workaround.

With this method, I was successful in getting the linked movie to load. However
1. It was not showing up in each frame
2. It wasn’t processing its own scripts (acting like a filmloop, looping frames)
3. It was overwriting the palette of the main movie.

To over come the 3rd problem I had to add a boolean _isCastmember to the Movie class and check it before setting the palette for the entire window. But now this results in the linked movie itself not having any palette, but I think the solution is easy, we just have to somehow set a palette for the movie’s bbox.

For the first problem, I had to write a isModified() function for the movie cast member (similar to all the other cast members) which marks the movie dirty for re-rendering.

The second problem however, was especially tricky. Because the lingo Scripting in ScummVM director is global, it only processes the scripts of the movie given by g_director->getCurrentMovie(). To solve this, I had to set the linked movie as the current movie using g_director->setCurrentMovie(_movie), before stepping its Score.

While loading also it processes the event kEventMovieStart, where I had to make sure that the correct movie is processing the event. But finally I got movies to work with scripting (at least for my test movie).

Overall, seems I’m on track to correctly load movie cast members. This week I’ll actually test this on the ‘trektech’ target, and try to progress it as much as possible.

This week my seventh semester started. So, there was a lot of stuff with credit registration and Hostel allotment, etc. So, couldn’t give the usual amount of time to ScummVM. From next week I’ll be consistent again. I also passed my midterm evaluation this week. Thank you, @sev and everyone for helping me, would’ve been lost without them.

This week was the most productive out of all the weeks until now (or maybe it feels like it). This week, I combined everything together. Until now, I was writing the logic of each resource to be written separately, dumping the file in the ./dumps folder and writing only that chunk in that file. So, I was checking the loading of only one resource at once. I wasn’t even recalculating the offsets.

This was good for testing but for the final writing, I needed to recalculate the sizes and the offsets of the resources. Which is what I did this week.

First thing was to remove the STUBbed saveMovie lingo command. Then I had to rebuild the RIFXArchive::_reosurces array which contains the struct Resource (size, offset, flags, index, castId, etc.) of each  resource. I added any new cast members to the resource list along with their children. I was under the impression that each cast member has separate children. Which made me think I’ll have to rebuild the indices of the cast members as well. But that is not the case: one child resource can have multiple parent resources (e.g. one ‘BITD’ can be child to multiple ‘CASt’ resources). This parent-child relationship also needs to be updated in the RIFXArchive::_keyData. My first approach was to rebuild the existing _resource array, but I quickly realized that I’m writing over important data here. So, I switched to building a new resource array and passing that to different functions. The bare bones of the writing of Memory map (‘mmap’), key data (‘KEY*’) and Cast data (‘CAS*’) were already done.

At first, there were a lot of issues with this. The data was being written correctly but the offset was wrong, the size returned by my function: getSCVWResourceSize() was off by two bytes. So, the entire loading of filmloop was failing. Also while writing ‘BITD’, ‘CLUT’, ‘STXT’ and ‘SCVW’ resources, I had to first find their parent cast members, which wasn’t being handled correctly. I also later realized that ‘BITD’ writing for 1/2/4bpp was off by one pixel. While reading the cast info in the ‘CASt’ resource, the strings: name, filename, directory name and type are all pascal strings. So, their first byte is not read (that byte marks the lengths of the string, which is redundant, we already have their length, so we ignore it). This caused the written cast information to be missing the first byte. The indices of ‘CASt’ resources written in the ‘CAS*’ were not written in the correct order, causing them to have the wrong CastIds. There was also an issue where if the lingo script goes saveMovie "filmloop_saved.dir", at first I was trying to write the file by constructing a path like Common::Path("filmloop_saved.dir"); but later realized that it also needs a parent directory, so started saving like Common::Path("./" + "filmloop_saved.dir");

Through all of this (and many more issues), I was finally to sort each issue one by one, and finally write the movies correctly. Now, the saveMovie {argument} lingo command saves the exact copy of the currently loaded movie to the path specified by the argument.

After this, I quickly finished writing score (very similar to Filmloop) and Rich Text.  This marks writing of all the modifiable resources in Director (that I know of). I have to check a few things before calling this ‘Done’. What happens when I try just duplicate cast "Original Cast" and if I call it repeatedly, will it be able to write all the duplicated casts? I also need to play around with puppetSprite. Also externally linked bitmaps are something to look into. @sev suggested trying out actual games that use this functionality to make sure it is fully functional.

This task is nearing its end. I hope to complete it soon. After that I want to work on my initial task of loading movie cast members.

Continuing from last week’s progress, this week I worked on saving some more Cast related resources: palette data (‘CLUT’ resource), bitmap data (‘BITD’ resource), text data (‘STXT’ resource) and the filmloop data (‘SCVW’). Most of the time was spent on testing these implementations. A test movie for each was made in Basilisk emulator (for Mac) and 86Box emulator (for Windows) using original Director 4 and Director 5.

The ‘CLUT’ resource was straightforward, since it only stores the colors in the palette as RGB values in bunches of 3. But we convert the 16-bit colors to 8-bit colors by only keeping the upper 8 bits only. Hence while saving these colors back, I had to left shift the byte by 8 bits and save it back.

The ‘BITD’ resource and ‘STXT’ resources were particularly tricky.

The ‘STXT’ resource contains the text data for Buttons and Text Fields. But the text is encoded in font encodings like Mac Roman and Mac Japanese for Mac or Windows-932 for Windows. This is specified in the ‘STXT’ resource. We detect the encoding, extract the part of the text that is encoded and convert it to a U32 string. While saving back however, we need to save the text in its original encoding. A single text resource can be encoded in more than one formats and at random offsets even. Thankfully, we do not ignore this information, we store it in a separate string as a header, like follows:
Common::String format = Common::String::format("\001\016%04x%02x%04x%04x%04x%04x", _style.fontId, _style.textSlant, _style.fontSize, _style.r, _style.g, _style.b);
_ftext += format;
The task was to read the header \001\016 in this string and write the font style that followed. Encode the U32 string back into the encoding given by _fontId, and write the raw string.

In case of ‘BITD’ resource (bitmap data), depending on the number of bits per pixel, we load it differently, naturally. The bitmap may be external or internal. @sev told me to focus on the internal loading for the moment. Most movies have their bitmap data compressed, i.e. instead of saving the data pixel by pixel RGB values, they store R/G/B values together,  hence if neighboring pixels have the same R values, instead of writing the said R value n times, we store the R value and the number of times it is repeated – n, which is Run Length Encoding for compressing bitmaps. However, I’m ignoring that for now, and storing the pixels in a row. Testing this was a struggle and a half. I had to create movies with bitmaps in all formats 1bpp, 2bpp, 4bpp, 8bpp, 16bpp and 32bpp and test each one separately.

Lastly, I worked on writing filmloop data which is the same as the score (‘SCVW’ resource). The resource stores each frame in the filmloop, its sprite channels and the associated cast member id. Since filmloop don’t have their own cast members, but use the cast members loaded in the cast, this was easier to save than the rest of the resources above. I simply had to write it in a format that is recognizable to the Director engine. This still took some time to write and test since the data loading for filmloops for D4/D5 is slightly different.

There is a slight recurring problem that the sizes of the resources need to be calculated before writing the resource itself because, the size precedes the data. This could be avoided by calculating the size as we write it, but that requires storing the offset of where the size is supposed to go, calculating the size, jumping to the offset, writing the size and then jumping back to the current position, which is an ugly solution.

Overall, the progress this week was good, but I’m worried whether I’m taking too much time testing. The midterm evaluation is coming up in 7 days, and I’m unsure I’ve done enough to count it as a success. I hoped to have completed this task by the midterm, but… Alas!

This week’s progress was somewhat better than the previous. Continuing from last week’s writing task, I was successfully writing the initial map, memory map, the key data and the cast data (‘CAS*’ data) successfully. Until then, we’re outputting the data as it is, without any modification. The goal was to be able to write movies for now even if there aren’t any modifications. This week I got to actually writing data that will be modified.

The first thing was the cast config data (‘VWCF’ chunk) data, this was pretty straightforward. I just had to save all the data that was being ignored and write it back in `saveConfig()`. Although, it involved calculating checksum for the config.

The next thing was the cast member data (‘CASt’ chunk), this was the tricky part. As I understand it, a ‘CASt’ chunk is made up of three sections, the first is the headers (which includes the metadata, like flags, cast member type and size of the rest of the two sections), there is info and data. The info has “strings” inside which contain information about the cast member like the name of the cast member, the file name (if it’s external), id of the script attached to it, font information, etc. Depending on the cast member, this number of strings varies. Hence, the size of the info also varies. Since I need to write the size of the info and data sections in the header, I have to calculate it from the loaded data. Hence, I ended up creating three methods getCastResourceSize(), getCastInfoSize(), and getCastDataSize(), this is useful since, I’ll need the ‘CASt’ resource size when I’m writing it in the memory map. Thankfully, this structure of info is consistent across all ‘CASt’ resources. So, I can create a single method in the base class and it works for all the cast members. The only section the cast members differ in is the data section for which I created a custom method in each derived class. (Note: Here the base class is CastMember and the derived classes are TextCastMember, PaletteCastMember, BitmapCastMember, etc.) The data section contains the cast member data (the bounding box dimensions, background color, flags, etc.). Now that I think about it, data and info are arbitrary names. Both contain different data about the castmember, but data still.

Majority of this weeks’ time was spent on testing chunks being written(even though there were only two (‘VWCF’ and ‘CASt’)). For debugging, I’ve also introduced a new channel (kDebugSaving). Some of the cast members are still haven’t been tested thoroughly.

These cast members can have children resources. Data like bitmap (pixel data in ‘BITD’ resource), text (‘STXT’ resource), are stored in these children resources. After the ‘CASt’ resource, our next step is writing these children resources which I think are going to be easier to write and test than the ‘CASt’ resource. Sidetracking, I tried writing the ‘CLUT’ (Palette data) and ‘STXT’ (Text data).

Even though, I was occupied the entire week, this still isn’t that much progress. It is still some time before this task is on its way to completion. I’ll need to speed up if I want to complete it within time.

Also, I realized my blogs are not as information rich as I’d like them. So, I’m planning on writing a mid-week blog to explain in detail, what I’ve done till now (shouldn’t take long!).

Quarter of GSoC period is gone… Wow, time flies!

This week I switched my task to saving Director movies in ScummVM. This was my main task of focus in my proposal. @sev explained to me that some games need to save .dir movies.

A director movie is in a containerized format. Chunks of different types of files are stored under a resource header. We can load and parse the movies in ScummVM but we cannot save them back since we throw away the things we don’t know yet. However, it is not as simple as reading the data in the file and writing it back to file. Because these games may modify the data inside these chunks. Also, the loading functionality is scattered throughout the Director engine. The ProjectorRays project by @djsrv is a great resource for this task.

So, the task is to keep the unknown data for the sake of saving the movies and to figure out how I can save the chunks that will be modified. All the other chunks that won’t be touched will be written verbatim.

This week, I started by writing the “reverse functions” to the archive loading functions in the Archive class readStream(), readMemoryMap(), readKeyMap()and readCast()  namely writeStream(), writeMemoryMap(), writeKeyMap() and writeCast(). The former load an Director movie archive whereas the latter saves the said archive. The rest of the chunks as described in the memory map (chunk ‘mmap’) are written as they are. For now, the movies are being written properly (except for a minor hiccup which I will check out today). Although I have done much testing on the windows, Director 5 movies I have, but @sev asked me to test this with a variety of movies before opening a pull request. Another idea is to store all the data for a specific chunk in a struct for consistency just like how ProjectorRays does. However, ProjectorRays doesn’t handle all chunks.

My initial idea to saving director movies was to move all the loading functions to the Archive class so that it can store them. It will have all the data and write it whenever necessary. However, @sev pointed out, that is a substantially flawed approach. There are far too many chunks to move to the Archive class, they don’t belong to the Archive class and it will be an unmaintainable mess. Thankfully, I didn’t spend much time on it.

Now, this week, I need to start writing Cast data (reversing loadCastData()).

I had to go to Pune again, for the publishing of our college magazine. Finally, that responsibility of mine is over. This week, I was logging in late, which moving forward, won’t be happening.

I’ve finally started to pick up the pace (although still slow). Things have started to look little brighter now…

This week, I actually got to implementing the Movie castmembers in ScummVM. The idea behind this was to use the MovieCastMember as a wrapper around a Movie member. This Movie class variable will handle the updating of frame number, execution of Scripts, handling events, etc. just like how a the main movie handles it. The tricky part was that it does not have a window. The main movie has a dedicated window where it can render its Sprites. The Movie castmember needs to be rendered in a box inside this main window. For this purpose when the main window updates (renders the next frame in the main movie) it also asks the MovieCastMember for its updated sprites using a function called getSubChannels(). So, the idea was simple, before returning these sprites, we ask the Movie castmember to “step”, update its own state.

But, this simple idea lead to me facing many difficulties. First of all, I had to include a check at each place the _window member was accessed in all functions in classes Movie and Score. The class Window also tracks the the Lingo State, which was now inaccessible for our MovieCastMember. So, the Window class needed refactoring.  I separated all Lingo State keeping functionality into a separate class LingoStateKeeper and migrated all the relevant functions  from Window. I added a member _stateKeeper to Window and Score. There was also a question of whether MovieCastMembers can switch movies using the go to movie ... command. I created a movie as a test in the original Director to check this. @sev asked me to add it to the director-tests repository.

This task is still incomplete. I’m able to see externally linked movies in ScummVM’s Director but they are stuck. They still do not allow execution of their own scripts.

What was thought to be a two day job, ate my entire week. I didn’t have my Director basics down. I hadn’t read the Manuals that were given to me a weeks ago.  I made obvious logical mistakes, I got stuck at multiple points… to the point that @sev has asked me to take a step away from this task.  I was working on processing Lingo script events the last. He will continue from here. He asked me to focus on my main task of  saving Director Movies. Hopefully, I’ll learn enough to come back to this task and be able to finish it.

I’m worried at this pace, whether I’ll be able to complete my project or rather make a meaningful contribution to ScummVM in a reasonable time period. I wanted to work yesterday (Sunday) but I thought it’d best to take the day off and read the Interactivity Manual (even though its a bit late now) and start working today (Monday) with a fresh mind.

This week I’ll focus on studying ProjectorRays to start writing director files. Study the different chunk types in a .dir file.

So… the first two weeks for me have been much less than perfect. I’m too focused on the greener side of the river forgetting there is deep water in between, gotta pass through that first or I’ll drown… But I’m optimistic. Even though I couldn’t keep the promise of having a good week from last week, I hope to have a comeback week starting today.