Last week was the last official week in the GSoC work period. I plan to submit my final evaluation after I post this. There you will be able to see some of the highlights from my time with ScummVM this summer.
In the past week, though, I didn’t start work on any new features. I spent a while understanding the data path location mechanism that the Mac version of Journeyman Project 2: Buried in Time used, and I ultimately discovered some buried XFCNs. (As @djsrv helpfully explained, XFCNs are external functions from (HyperCard)[https://en.wikipedia.org/wiki/HyperCard] that can often be called as regular Lingo.) One of these just looped through all the volume names on the system, looking for the proper CD name. This Journeyman Project is split across three CDs, so some more additions will be needed.
I also fixed an obscure issue decoding the hex format strings for our Macintosh text implementation. Now, the main screen of Majestic looks almost perfect – with proper cursors, which I also patched to load properly – and consistent text formatting.
[]
Additionally, I investigated and fixed several bugs with my good old friends the widgets. Now Chop Suey also looks very accurate, and there is no more flashing cursor or cut-off text boxes. Text borders are now (almost) working properly again, which makes the Lingo Dictionary movies more pleasant to look at.
[]
For being the rendering guy this summer I haven’t posted nearly enough pictures, so I put those two here for you to enjoy. I’ve enjoyed working with ScummVM, and I plan to start contributing again once I get comfortable in my university schedule.
Last week, I finished implementing the direct-copy mode for our Macintosh window manager. Now, with the Director engine, there is no intermediate blitting onto a screen surface that is then copied to the physical screen. This restored transition speed and avoided duplicate work on every frame. Here’s the opening of Spaceship Warlock with the new interface and transitions:
For a while I was stuck on implementing border transparency without an intermediate surface, but @sev helpfully reminded me of g_system->lockScreen(), which no longer has performance issues on some systems.
I also looked at several long-standing bugs this week. As @djsrv probably noted, in investigating one we discovered that the goto Lingo should not run the exitFrame event. In the opening scene of Majestic, the two middle bitmap text channels previously held the mouse-over highlighting for “Restore Old Game” and “Exit” in the main menu. Clicking “Start New Game” went to the correct place, but not before the main menu’s exitFrame event disabled visibility of those channels again. Now, though, everything looks right:
Tomorrow, it will be time to figure out why the cursor implementation I made doesn’t work with Majestic… and also why the bitmap cursor in Chop Suey still has a strange flashing behaviour.
Last week, I added palette support for Director 3 movies, so several more games had their colouring issues on their opening movies resolved. Then, I spent much of Tuesday and Wednesday fixing issues with our path-loading code that prevented several of @trembyle’s testing titles from running – no less than 7 when you account for Windows and Mac versions. Lingo provides several ways to interact with the filesystem. Some games asked you for a drive letter and would just loop endlessly if they would not find the file they wanted. Others were a bit smarter – on Windows, for instance, they would look for a hardcoded file name on all the drive letters and crash if there was no match.
In some cases, our file finder was not trying all the possible Director movie extensions on Macintosh, and others had filenames improperly converted from Macintosh long file names to the FAT 8.3 format. I actually just now saw that @trembyle notated several new titles with path/file errors, so I will probably look at those once I finish my current set of tasks:
Before @djsrv implemented the desktop mode, the Director transition drawing code was taking direct control of the screen (via copyRectToScreen). In the desktop mode, however, this would pay no respect to window layering or positioning. So I am working on improving screen access through the window manager. Currently the WM keep a copy of the screen as a surface, but I am exploring how to avoid this extra copying. This can cause performance issues, even in non-desktop mode, since the surfaces must be copied twice – from the Director sprite, onto the _screen, and from _screen onto the display device. Fixing this duplication will help the effort to support QuickTime video from Director movies.
In the last part of the week, I got distracted by my starting in earnest my first from-scratch RE of another engine. This is not for GSoC, just to satisfy my own curiosity. It’s the engine used by the one title in the Living Books series (D. W. the Picky Eater) that doesn’t use Mohawk. I understand the file format fairly well, and so the next task is extracting resources and then working toward the script bytecode. See my repo for more information.
This will be short, as last week was rather slow. I mostly finished shoring up all the areas where Director palettes can be tested and set – in the frame palette channel, with the puppetPalette command, with the movie-wide default palette, and also with the palette of cast Lingo. As I look back at my commit log, I realize that this took me many more commits than I expected. There are still some issues that some of our new test targets, including the 1991 title The Riddle of the Maze, recently revealed. As of this morning, though, this game’s elegant artwork has been restored:
Apparently, before Director 4, palette castmembers were forced into the same order that they are stored. If you reorganize them in the cast window, they will actually revert to a default position on save – and this position seems to be in the order of palette creation. Also, Director 3 Macintosh format has a slightly different way of storing palette channel information in the frame. (I have still to push the patches for titles that have this variation.)
I also finally got some of the titles that @trembyle had mentioned on the Director wiki, including the aforementioned Maze. @trembyle has done some immense work testing many new titles for us, and today I enjoyed investigatinCEg some new bugs in new titles. Playing with these new games was a nice breath of fresh air from the targets that we had been testing since the beginning – like Spaceship Warlock and Chop Suey and The Apartment.
My other big task last week was working on the text selection interface. When I think back, I believe my first experience with Director happened when I was pretty young; it was a typing game that also depended heavily on text selection. I remember this game as Director because it crashed once and I distinctly recall the Director projector icon. @sev had done most of the algo work in the MacGUI side, but I needed to write a bit more interface and do lots of testing to discover what was behind a selection offset offset bug.
Once I finish up the palettes and add a few new targets that I found, I will be working on supporting movies that have >8-bit colour. If you look in the graphics code, there are tons of const byte * declarations everywhere, so those will be changed out for a more general implementation. There are also bitmap decoding issues with greater colour depths that I need to look into. This task, a fairly big one, will be a nice way to round out my GSoC work on our Director engine.
This week, I worked on implementing a few Lingo commands that required some significant backend work, but not much in the rendering code where I have spent so much of my time. I mentioned that I had worked on a custom cursor implementation, and I spent the first part of the week fixing bugs there. Now cursor bitmaps like the starfish on the island are properly displayed at the proper position, whereas after my initial work just a black box showed.
Early in the week, I fixed a subtle rendering bug that had plagued Spaceship Warlock for a while. You’ve seen how the Stambulian policemen in Warlock were bright green; well, this was about the same as the keying colour that the MacGUI was using for border transparency. Once I spotted this, the fix was easy – and also prevented a redundant surface copy. (Yay for performance improvements!) Here you can see how the erroneously applied transparency actually gave a nice look to an otherwise bland wall in Stambul:
I also finished the implementation of custom colour palettes. For a long while the opening to Chop Suey looked all psychedelic. It turns out that a recent refactoring swapped around the order of palette loading so ScummVM was given the wrong palette to use. My partial fix for this revealed that our recently-added target Majestic used a castmember palette, which had been loaded before – albeit improperly. So over the weekend I expanded our Director palette manager to support custom palettes alongside the half-dozen default Mac ones. This also brought along basic support for the puppetPalette command, which controls the palette from Lingo. Now, Chop Suey and Majestic are looking quite handsome:
This project came later in the week, however. I first implemented sound fading, which more intense than I first knew. Since multiple movies can be running at once, my knee-jerk blocking fade loop didn’t work out. Instead, I needed to integrate the sound fade with the existing score stepping methods. I’m amused that that this was one of my largest commits for the week. (Thankfully, though, transitions do seem to block in the original – that would be a pain to refactor.)
Finally, I scratched my head for a while on an issue that @sev has since begun looking into. An interesting Director target is Macromedia’s own guided tour, which takes you “behind the scenes” at their studios to introduce new features of Director. Our renderer implicitly assumed that bitmap sprites had the same dimensions as the underlying castmembers, unless the dimensions had been modified from Lingo. Well, our smartly-dressed friend from the guided tour dismissed that theory:
What’s on the left is the original sprite (plus arms), and on right is the original castmember. ScummVM draws them both the same size. I would never have noticed, except that the sprite for his moving mouth appears all out of place when he isn’t the right size. Director does indeed have an option to scale individual sprites in the score, and we were reading this information in for bitmaps… but when I tried to use it there were puzzling discrepancies in the dimensions we expected and what the file clearly said. I still haven’t figured out why.
It’s been a good week over all, with interesting tasks both in and outside GSoC. You’ve perhaps seen on my bio that I like playing music. One of my friends recruited me to play piano for her upcoming violin competition, so when I need a break from coding I spend a few hours with the interesting sonorities of the great American composer Samuel Barber. And, at university on the weekends, I am working on using machine learning to control dielectric elastomers – smart materials that show much promise for soft robotics.
Just a few weeks ago, Chop Suey ran at an almost unplayable crawl in ScummVM. Part of this was its reliance on Matte inks, for which I implemented a simple surface caching scheme and shaved about 20% off our buildbot’s target test time for Spaceship Warlock. This also made Chop Suey run much faster, though it still consumed CPU
Last week I called Chop Suey a Lingo-heavy game, and I was referring to how much it controls animation via puppets and the updateStage command. Because, as you saw, its cursors are bitmaps and certainly do not fit in the standard 16×16 Macintosh cursor box, Chop Suey introduces its own mouse update code and calls for the stage to be updated several dozen times each frame. Most of the inefficiencies were here.
I spent the early part of the week in much trial-and-error, working out the pieces of the renderer that were most inefficient under such repeated application. The idea is to do a little bit of work up front – checking flags and so forth – so the expense of redrawing a region of the screen is saved. (As I have realized, even when working on Chop Suey, very subtle bugs can arise from forgetting to check a rendering flag.) Even at usual framerates without much Lingo that doesn’t matter very much, but Lingo-heavy games like Chop Suey have shown dramatic improvement.
I also implemented another feature that is notable in Chop Suey by its absence. I said that Chop Suey does its own cursor handling. Well, the window manager was still drawing a regular cursor atop the bitmap, which looked pretty ugly. Near the end of the week, though, I added a flexible cursor class for the three cursor types Director can use: built-in cursors, cast (bitmap) cursors, and resource cursors. The last type is important for Majestic: Alien Encounter, but I haven’t seen custom cursors used much elsewhere. It was fun to implement, though. As a nice byproduct in Chop Suey, the default cursor is now properly turned off.
Oh, and I also spent most of a day trying to discover why some textboxes in Spaceship Warlock weren’t rendering properly, along with some other nettling MacGUI issues. The issue actually lay in the cast loading code, which I hadn’t touched much, but it’s always satisfying to squash a bug and learn more about the codebase in the process – even if your “fix” breaks other stuff. 🙂
After last week’s post, I nearly finished with the last major piece of the ink types – applying foreground and background colours on the fly. We had some idea of when this colour-application process took effect, but deducing the rules and working out the patterns took a while. I have gained a new appreciation for the power of bitwise operations through this process. My next step will be extending all this logic to other colour depths, but I probably won’t get to that this week.
It’s getting late now and I can’t find the test movie I had made to demonstrate this work. I’ll look at it tomorrow and post a video demonstration of my work then.
There is still some optimization that must be done, including eliminating my reliance upon the window manager’s slow colour finder and implementing some sort of colour caching. (The inked operations are often not applied directly to the colour value; it must first be split into its components and then recombined in the palette space.)
I also performed some fine-tuning on the data structures used for rendering, to ensure that our pixel-based callback is quite fast. Lots more refactoring went into that. Finally, last week I implemented some obscure features, like the constraint and stretch of sprite. The latter allows a sprite to be arbitrary scaled to another bounding box, and implementing that feature largely finished my goal of having self-contained blitting routines for our engine.
Many of the tasks currently in my Trello channel are concerns with MacGUI elements that don’t impact most real games. These bugs will still take some work, but as I am working on those I also want to get the graphics in Chop Suey (and other Lingo-heavy titles) running smoothly and efficiently. Today I chased bugs and inefficiencies in the rendering pipeline. This is a hefty task, as Chop Suey often uses about 70-75% of my system’s CPU when it is running in ScummVM. With the discoveries I made today, however, taking the slack out of the renderer should be eminently doable.
Another week has gone by quite quickly. After taking some time off for Independence Day here in the US, I can’t believe that another Monday has passed. Last week I talked about a major update to our renderer, and I finished that pretty early last week. There was some major refactoring involved, but my work – splitting out our renderer into a Stage class and moving to traditional dirty-rects handling – worked well with some of the major refactoring that djsrv had been wanting to do for a while. Going into this week, our engine’s code is far more modular (thanks primarily to djsrv), and the graphics are looking good. All this we showed off to John Henry Thompson last Thursday.
First, I finished implementing the puppetTransition command, which allows you to call transitions from Lingo. As part of this, I reworked the transitions pipeline to support only drawing a transition on the changed area of the screen. This took a lot more work (and many paper sketches) more than I was expecting, as some of the transition algorithms had interesting behaviour when I tried to clip them.
I also did some more work on the ink types, including making an efficient interface for the ink types that use implicit mattes or masks. Over the weekend I worked on what was for a long time a big irritation of mine – sprites with scripts did not highlight (or, in Director parlance, hilite) on click. When I tried to work on this before, I found that simply testing for a script on the sprite was far too broad; there were many erroneous inversions. It turns out that for bitmaps there was a special flag, and for shapes (as far as I could discern) only one ink type where this effect showed. I hadn’t looked much at the Director file format before, but I found where that flag was and also uncovered some other unknown fields in bitmap casts (primarily palette information). Working there led me to some nice condensing of our sprite code, which I pushed this morning. That’s one more step to making The Apartment realistic!
Overall, though, last week felt like mostly under-the-hood changes and small optimizations to individual movies. Right now I’m working on the graphics code for efficiently setting the foreground and background colour of puppeted bitmap sprites, and rendering this on-the-fly.
This week wrapped up the fourth week of coding, which means that there are about 8 weeks left until GSoC finishes and it’s time for me to go back to school. This week, I spent the early part of the week fixing up our rendering pipeline. It is fun to work on, since it’s so fundamental to the engine, but many bugs are quite difficult to track down. A big part of this was more closely integrating Director’s internal state memory, now kept in the Channel class that I wrote, with the “widgets” that the window manager knows about. A tight interface with the window manager will become essential quite soon, when we implement MIAWs (movies in a window).
When I got tired of tracking down obscure artifacts and bugs near the middle of the week, I revamped the interface that we use to draw Director transitions. There are about 50 different transition types, which sev had implemented before, but rather than having to keep a copy of the old frame and then draw the new frame back over it – as we had to do before – I put in a more natural approach of rendering the next frame over the current one, in accordance with the transition. This cut out some unnecessary surface blitting that was happening every frame. Working with the transitions took way longer than I expected, and I’m still not quite sure why.
I also spent some time reviewing our Trello board and picking out easy tasks for a break or marking complete those reports of broken movies, etc. that some side effect had fixed along the way. A nice 5-minute fix was enabling our text renderer to use different colours in each chunk (font run). Now, Warlock is looking even more realistic, and just that little bit of white text helps so much.
I hear that later this week sev, djsrv, and I will do a catch-up meeting with John Henry Thompson, the inventor of Lingo. In this meeting I plan to show off some of the work I have done with Director’s ink types. That was the other major project I worked on last week. The inks are all the different ways that Director can draw and composit sprites on the screen. They range from a simple copy of the whole bounding rectangle area to mattes to an inversion and composition with what’s underneath. Many of them have poorly documented behaviour, at least in edge cases, and when transforming colours the depth must always be kept in mind. Director itself had different render loops depending upon the colour depth. I don’t think we will quite need that, but the bitwise operations I put in will require much more testing and fine-tuning across the spectrum. I’ll be working on this more in the next few days. Here’s a little demo of the ink types in action. There are still some bugs in here.
I’d better get back to work now, because I am preparing another update to the renderer that should deal with many of the issues that have lurked in the background in the last few weeks. Stay tuned for some news on that!
I started off last week attempting to get the infamous zoomBox working with the new rendering pipeline I wrote for our Director engine. A zoomBox effect is the classic window movement animation in classic Mac OS: When you open or close a window, you see many spectral rectangles in between the window’s origin and destination. Even though I haven’t seen any games actually use it, this function was the first motivation I was given reworking the rendering pipeline. Before, we could not look ahead into the next frame to get the dimensions of a sprite.
Moreover, before, we had to cache a copy of the screen so it could be restored after the animation was finished. I used the same trick that it seemed Mac OS used with many interface elements, though. As the animation was drawn, the pixels of the rectangle were inverted, and then on a second pass they were inverted back. Just like matrices, invertible animations are quite nice.
I also modified much of my code from the previous weeks to get a major part of our benchmark kit, Macromedia’s own The Apartment, working properly. Normally, when a sprite is puppeted, its position is not updated from the score. So animated puppeted sprites are impossible, which made my job a bit easier. There is another case, however, that required a major rework of the rendering code I had already written. I didn’t realize that with a sprite simply declared moveable, rather than a full puppet, it can be animated. Thus, we could no longer store the current point in each sprite as it flew by over the playback head – that would give us very strange jitters. Instead, location information needed to be consistent across frames. So, another level of abstraction was needed, a Channel class that would keep track of this intraframe information.
It feels strange to summarize into a single sentence a task that took me several hours to complete. But, I suppose this is the very nature of software engineering. Over the past weeks, I have learned very much about being patient with myself and taking breaks when I need to. I’m going to try to be a bit better about working for a long time on the weekends, even though I find it very enjoyable.
On Wednesday, I think it was, djsrv fixed a Lingo issue and I fixed several rendering\widget issues that allowed me to get more than a quarter of the way into Warlock – and even then, it was my choice to stop and not a misplaced null pointer’s. 🙂 This is an exciting progress milestone for us. Ah, the beautiful Belshazzar!
On Friday, I didn’t commit much because I got distracted by an engine I had mentioned briefly before, the Media Station engine. Many of the Disney Animated Storybook CD-ROMs were developed with this engine, as well as one of the few Living Books titles that didn’t use Mohawk. Strangerke warned me a few weeks ago that the executable was very complicated, but I found some success using moralrecordings’ Mr. Crowbar to peer inside the assets. They seemed like easy RIFF, but nearly all the chunks seem to be named some variation of “igod” and there’s some extra structure hidden inside the chunks. Bitmap and waveform data is clearly visible, though, so there’s probably not far to go to understand the format. After GSoC is over, I would hope stay on and keep working with this engine – plus Director, of course.
Anyway, I’d better get back to Director. I’m now working on getting all of the inks (blitting modes) playing nicely together. Graphics programming has proven pretty neat, even if it often goes slowly.
]
]
