Ramyak's GSoC Blog

Tuesday

This week I finally got SSH access to john.scummvm.net, the machine that runs the Director buildbot. I’d seen its output for weeks (those BUILDBOT: warning lines I kept triggering) but had no idea how it actually worked. Getting access and reading through the code was very satisfying because I finally met the machine that had been yelling at me for the past couple months.

The Buildbot Architecture

How a build starts

The first thing that clicked was how the build is scoped. The buildbot watches the ScummVM GitHub repo via a webhook, but it doesn’t rebuild on every commit. master.py checks whether any changed files are in engines/director or graphics/macgui. If yes, it kicks off a build. If not, it ignores the commit entirely. Simple filter, makes sense.

Triggering the tests
Once the ScummVM binary is compiled and uploaded to the master, build_factory.py calls steps.Trigger(schedulerNames=["Director Tests"]). This tells the buildbot master to fire the Director Tests scheduler, which is a Triggerable scheduler defined in master.py. That scheduler then queues up all the test builders at once; one for each entry in targets.json, so they all start running in parallel on the available workers. The build step waits (waitForFinish=True) for all of them to complete before marking the overall build as done.

Running the tests and error checking

The test runners themselves are defined in targets.py. Each one downloads the binary, rsyncs game files from /storage, and runs ScummVM against a list of movies. Screenshots are saved to /home/director-buildbot/screenshots/ on every run. Error checking is done in steps.py, which watches the output for lines matching “BUILDBOT: incorrect check for line:” – that’s the log-replay mechanism from the director-tests repo, where expected output is recorded directly into the test movie file, and on each buildbot run the live output is compared against it line by line.

Reading through this, the separation of concerns became clear: master.py handles scheduling, build_factory.py handles the build pipeline, targets.py defines what gets tested, and steps.py defines how a single test run behaves. Once I had that mental map, the rest followed quickly.

The ImageDiff tool

One thing the buildbot produces but doesn’t analyze automatically is screenshots. Sev pointed me to ImageDiff, a tool built to catch visual regressions, cases where the engine produces slightly different output without triggering any log-based errors.

It’s a Flask web app that reads from the screenshots directory and shows a frame-by-frame diff between any two builds for a given target and movie. The core logic uses PIL’s ImageChops.difference to compute a pixel-level diff. If there’s any difference, the diff image is rendered alongside the source and comparison frames so you can see exactly what changed. Results are cached to disk so repeated comparisons don’t recompute.

I temporarily deployed it on john.scummvm.net on port 5002. The two targets currently generating screenshots are director (from the director-tests-* folders) and theapartment-mac, both of which showed up with their full build history. It’s a simple tool but it fills a gap that log-replay can’t, visual changes.

Bugs

This week I did not work on any game bug fixes. I only made changes to the visual debugger and its bugs.

The gus games bugs are mostly fixed already and the remaining one bug has not been very consistently reproducible. So, this week I’ll try to find out how to replicate it.

Here is a brief on the DT changes:

Windows Panel

• Added a new Windows panel showing all currently loaded windows and all .DIR movie files in the game directory
• Clicking a movie navigates to it via func_goto

Search

• Added variable search mode to the search bar
• Improved search with new modes (Handlers, Variables, Body) and a cleaner 3-column results table with keyboard focus on open

PRs:

https://github.com/scummvm/scummvm/pull/7564

https://github.com/scummvm/scummvm/pull/7553

last week’s changes: https://github.com/scummvm/scummvm/pull/7563

What I am currently working on

• Currently I am working on some more DT changes.
• Working on the checksum function.
  • Some Director movies have a VWCF resource with a checksum that our implementation fails to verify correctly, causing a crash when navigating to those movies in the debugger.
  • Sev suggested – before diving deep – running the mismatched movies through ProjectorRays first to see if it also miscalculates, that way we know whether the bug is in our implementation or the movies themselves. The code once completely written, will be identical to the Projector Rays checksum code.

P.S. The buildbot integration will be re-worked soon by one of our devs rvanlaar, so the current architecture might not be relevant in the future

Sunday

Coming back to a large codebase after a break is disorienting.

I returned this week after a 3-week gap and barely recognized the code I’d been working on. So I planned the week around small, finishable tasks. Nothing ambitious. Just things I could complete and feel good about, to rebuild momentum.

Start small

Sev assigned me my Planka board (the task tracker we use) and pointed me toward a set of Gus games: Gus Goes to Kooky Carnival, Cybertown, and a few others. I went through the tickets, broke each issue into its own card, and we had a Zoom call where he helped me set up the environment properly.

This call is worth describing because it’s a good example of what mentorship looks like here. We talked about Dumper Companion (a tool for inspecting Director movie internals), useful ScummVM debug flags, debugging workflows. The non-technical conversation ended up being just as helpful for reorientation as the technical one. If you’re new and a maintainer makes space for this kind of call, take it.

cast viewer in the visual debugger

It wasn’t loading shared cast members. I fixed that and removed some duplicate code while I was there. Small, satisfying, done. That’s the point of starting small.

---

Bug 1: Crashing on empty cast slots

The hard part of a fix is often defining the boundary correctly, not writing the code

In original Director, if you set a property on a cast member slot that exists but is empty, it fails silently. ScummVM was throwing an error, which crashed games that relied on that behavior.

Imagine cast slots are numbered 1 through “b”, where “b” is the last populated member. Accessing slot “a” (within bounds but empty) should fail silently. Accessing slot “c” should throw an error. The bounds are defined by the last populated cast member, not the total allocated size.

The fix seemed obvious: skip the error if the cast member isn’t found. But that would also swallow errors for genuinely out-of-bounds IDs, real bugs you’d want to catch. The problem wasn’t the error. It was defining “in-bounds” correctly.

test movies

This is also where Sev introduced me to test movies, minimal Director movies that reproduce a single buggy behavior in isolation. Instead of loading an entire game to verify a fix, you create the smallest possible movie that triggers the issue. Much faster, much cleaner.

The solution: teach ScummVM to distinguish between an empty cast slot within a valid range and a genuinely invalid cast member ID (error).

---

Bug 2: Sprite dragging not working

The symptom and the cause can live in completely different systems

In one of the Gus games, clicking a puzzle piece should let you drag it around the screen. In ScummVM, clicking did nothing. The piece stayed frozen.

I assumed the drag logic was broken. It wasn’t. The drag code was fine. The real issue was buried in the event pipeline, and finding it meant tracing the entire event flow from click to handler.

ground truth testing

I needed to confirm how Director 4 actually behaves, so I ran the original Director inside Basilisk II (a classic Mac emulator) to observe the real immediate sprite property. You can’t rely on documentation alone (shocker for me), some features are marked obsolete but still functional, and behavior varies between versions.

Here’s what immediate does in Director 4 (5 & 6 too):

normally, mouseDown fires on press and mouseUp fires on release. When immediate is set on a sprite, both mouseDown  and  mouseUp  fire on the press.

The drag handler works by running a repeat while the stillDown loop inside on mouseUp. If mouseUp only fires after the physical release, stillDown is already false and the loop exits instantly. Nothing moves.

The fix was in queueEvent(): when a press arrives on an immediate sprite in D4, queue both mouseDown and  mouseUp  events immediately, then suppress the physical mouseUp later to prevent double-firing.

version boundaries

sev pointed out something: does immediate behave the same in D3? Does it stop working in D5? 

Turns out immediate works in D4, D5, D6. Not tested D3 yet. Will update this after.

the director-tests repo

Instead of loading full games to test a behavior, we use the director-tests repo, a collection of minimal test movies that verify specific Director engine behaviors. You create the smallest possible movie that reproduces just the thing you care about, and it lives in the repo as a permanent regression test.

---

What I’m aiming for next week

• Work through more Gus bugs. 

• Look at build-bot issues. 

• Go deep on one subsystem end-to-end.

• Improve tests and stress the visual debugger.