Month: June 2020

Hello again!

This week wrapped up the fourth week of coding, which means that there are about 8 weeks left until GSoC finishes and it’s time for me to go back to school. This week, I spent the early part of the week fixing up our rendering pipeline. It is fun to work on, since it’s so fundamental to the engine, but many bugs are quite difficult to track down. A big part of this was more closely integrating Director’s internal state memory, now kept in the Channel class that I wrote, with the “widgets” that the window manager knows about. A tight interface with the window manager will become essential quite soon, when we implement MIAWs (movies in a window).

When I got tired of tracking down obscure artifacts and bugs near the middle of the week, I revamped the interface that we use to draw Director transitions. There are about 50 different transition types, which sev had implemented before, but rather than having to keep a copy of the old frame and then draw the new frame back over it – as we had to do before – I put in a more natural approach of rendering the next frame over the current one, in accordance with the transition. This cut out some unnecessary surface blitting that was happening every frame. Working with the transitions took way longer than I expected, and I’m still not quite sure why.

I also spent some time reviewing our Trello board and picking out easy tasks for a break or marking complete those reports of broken movies, etc. that some side effect had fixed along the way. A nice 5-minute fix was enabling our text renderer to use different colours in each chunk (font run). Now, Warlock is looking even more realistic, and just that little bit of white text helps so much.

I hear that later this week sev, djsrv, and I will do a catch-up meeting with John Henry Thompson, the inventor of Lingo. In this meeting I plan to show off some of the work I have done with Director’s ink types. That was the other major project I worked on last week. The inks are all the different ways that Director can draw and composit sprites on the screen. They range from a simple copy of the whole bounding rectangle area to mattes to an inversion and composition with what’s underneath. Many of them have poorly documented behaviour, at least in edge cases, and when transforming colours the depth must always be kept in mind. Director itself had different render loops depending upon the colour depth. I don’t think we will quite need that, but the bitwise operations I put in will require much more testing and fine-tuning across the spectrum. I’ll be working on this more in the next few days. Here’s a little demo of the ink types in action. There are still some bugs in here.

I’d better get back to work now, because I am preparing another update to the renderer that should deal with many of the issues that have lurked in the background in the last few weeks. Stay tuned for some news on that!

Hey guys!

I started off last week attempting to get the infamous zoomBox working with the new rendering pipeline I wrote for our Director engine. A zoomBox effect is the classic window movement animation in classic Mac OS: When you open or close a window, you see many spectral rectangles in between the window’s origin and destination. Even though I haven’t seen any games actually use it, this function was the first motivation I was given reworking the rendering pipeline. Before, we could not look ahead into the next frame to get the dimensions of a sprite.

Moreover, before, we had to cache a copy of the screen so it could be restored after the animation was finished. I used the same trick that it seemed Mac OS used with many interface elements, though. As the animation was drawn, the pixels of the rectangle were inverted, and then on a second pass they were inverted back. Just like matrices, invertible animations are quite nice.

I also modified much of my code from the previous weeks to get a major part of our benchmark kit, Macromedia’s own The Apartment, working properly. Normally, when a sprite is puppeted, its position is not updated from the score. So animated puppeted sprites are impossible, which made my job a bit easier. There is another case, however, that required a major rework of the rendering code I had already written. I didn’t realize that with a sprite simply declared moveable, rather than a full puppet, it can be animated. Thus, we could no longer store the current point in each sprite as it flew by over the playback head – that would give us very strange jitters. Instead, location information needed to be consistent across frames. So, another level of abstraction was needed, a Channel class that would keep track of this intraframe information.

It feels strange to summarize into a single sentence a task that took me several hours to complete. But, I suppose this is the very nature of software engineering. Over the past weeks, I have learned very much about being patient with myself and taking breaks when I need to. I’m going to try to be a bit better about working for a long time on the weekends, even though I find it very enjoyable.

On Wednesday, I think it was, djsrv fixed a Lingo issue and I fixed several rendering\widget issues that allowed me to get more than a quarter of the way into Warlock – and even then, it was my choice to stop and not a misplaced null pointer’s. 🙂 This is an exciting progress milestone for us. Ah, the beautiful Belshazzar!

On Friday, I didn’t commit much because I got distracted by an engine I had mentioned briefly before, the Media Station engine. Many of the Disney Animated Storybook CD-ROMs were developed with this engine, as well as one of the few Living Books titles that didn’t use Mohawk. Strangerke warned me a few weeks ago that the executable was very complicated, but I found some success using moralrecordings’ Mr. Crowbar to peer inside the assets. They seemed like easy RIFF, but nearly all the chunks seem to be named some variation of “igod” and there’s some extra structure hidden inside the chunks. Bitmap and waveform data is clearly visible, though, so there’s probably not far to go to understand the format. After GSoC is over, I would hope stay on and keep working with this engine – plus Director, of course.

Anyway, I’d better get back to Director. I’m now working on getting all of the inks (blitting modes) playing nicely together. Graphics programming has proven pretty neat, even if it often goes slowly.

Last week, I finished up the tasks that had dragged me down, and then I moved on to sundry other topics. Now our Director engine renders The Apartment, a demo movie that MacroMind and later Macromedia developed for the early Directors, in greater fidelity. One of the issues that took a while to figure out was that, in the Director 3 version of The Apartment, GUI buttons had a special type that were not stored in the usual place. To our engine, they just looked like regular texts, but they were not. I also realized how tedious it is to fiddle with a few magic numbers until the GUI elements came out looking right. I still need to get QuickDraw shapes fully transported over to widget-based rendering, which I plan to do later this week.

After my PR was merged, sev granted me write access to the repository, so I could commit fixes on-the-fly. This was very helpful because, as you can see, I have fixed many little bugs in my code since then. Over the latter half of last week, I grew well acquainted with the technical methods in the Mac text-rendering class that sev wrote, and I did some major refactoring to merge the two editable- and non-editable variants. Then, with this, I began implementing some Lingo entities that permit some Director-based customization of text fields. Some of this, such as changing fonts and line widths and such as a movie plays, I have not yet pushed.

An issue that I spent several hours investigating last week involved text background colours in Spaceship Warlock. Ultimately, with sev’s help, it turned out that our loading code was not reading the palette information for text cast members correctly. Thus, the hack had been to give all texts the background colour of the whole Director stage. This worked well sometimes – but sometimes individual movies in Warlock had a white stage covered by the black rectangles, and sometimes they had a black stage. So, I confronted strange inconsistencies like this:

Once we got to the root of the issue, texts rendered correctly, no matter the underlying stage colour:

(I was confronted by those cops so many times one day, I had a dream about them.)

Tracking down strange rendering bugs like these has been the majority of my work over the past few days, and I will get back to it again today in tracking down some long-standing issues in The Apartment. Debugging is often frustrating, especially when I judge myself for going too slowly, but it is quite pleasant to see my fix for an issue pushed – and often the solution was far away from where I thought it would be. Well, I’d better get to it for this week.

Last time, I promised I would give an update on QuickDraw and the Macintosh GUI emulator. This second major part of reworking the rendering pipeline, which i thought I could finish in the first week of coding, has taken more time and gone deeper than I expected. When I am coding I am well aware of wasting time in shaving the yak. I mentioned in our Discord the other day that I might have experienced mission creep. Let me explain.

MY initial goal was to move over Director’s text rendering to the enhanced Mac text class that sev wrote for us earlier this year. I spent the early part of that week implementing the different kinds of text properties that Director permits putting on text – straightforward effects like shadows, framing, and proper text alignment. (I now realize what complex work goes into graphics management for even a simple text editor.) Then, I created a new class (widget) specifically for buttons and did the similar work for the classic Mac’s three button styles. This took me to about Wednesday last week, and this part of my work went fairly quickly. I even got several movies mostly working, including the stageColor movie (a simple one that flashed the emulator background color very quickly – I won’t show it because I myself am photosensitive). This showed off saveral pieces of the work I had done: Drawing text with transparency and extra features, as well as reading button states properly.

I then realized that in order for the window manager to process widget events in the proper order, it needed to have some concept of priority – a mirror of channels in Director. So then, I thought, we have only four major visual cast types in our Director engine: text, buttons, shapes, and bitmaps. Why not have all those be known to the window manager as widgets? This would accomplish another of the refactoring tasks I had been given, reducing the bloat of the Stage class by moving drawing methods away from there.

Then, I got rather bogged down later in the week in understanding the minutiae of the window manager, and I realized that much of the outward-facing code in our Director events manager could be handled better in the MacGUI manager instead; this was added to my list of items to finish. (You see what I mean about shaving yaks.) There were a few other version-specific irritations that I also tried to figure out; this took me into the weekend.

I soon had a lot of deeply connected tasks that seemed rather overwhelming as the first week ended; I realized that I had expanded the range of my next PR far beyond where I initially started. Until the weekend, I thought I was making good time. After I did not do much over the weekend to catch up with mechatronics research lab at where I work at university, I realized that trying to change everything at once is an excellent way to get behind. I do not want this to happen, so I talked to sev yesterday to help get me back on track.

I will post some screenshots later this week, once my improvements are complete enough to merge at least some of them. I believe our prime test move, The Apartment, will soon yield some good results in previously broken movies. I have known for a while my tendency toward over-engineering and trying to “go solo,” so confronting such mission creep in the very first week of coding was quite helpful for me.

Don’t worry, you’ll hear from me again soon!

Google Summer of Code 2020 officially started today, although throughout May I worked regularly on the Macromedia Director engine’s rendering pipeline. In the middle of May I replaced the long-standing frame-based renderer – which redrew the entire screen every frame – with a channel-based approach. This first part of the project brought me in touch with most of the other ScummVM Director devs, which was a nice touch for GSoC’s Community Bonding Period.

Modern operating systems use a desktop metaphor, with folders and notepads and option drawers. Director uses a cinematic metaphor, which works like this: Each movie has some cast members (shapes, bitmaps, text, etc.) which can be instantiated as sprites to appear on the stage (user viewing area). Each sprite inhabits a channel in the score. Rather confusingly, in fact, the terms sprite and channel are almost used interchangeably. One channel can display many different cast members throughout a movie. As you can see, this jargon (I dare not say lingo) departs from the usual understanding of sprite.

Just as its musical connotation implies, the score compactly describes the activity of each sprite and provides ready access to Lingo scripts and other customizations.

As you might imagine from this screenshot of a Director 4 (1994) score, scores can be viewed as large matrices indexed by frames on the vertical and channels on the horizontal. Each entry (cell) of this matrix thus describes one item on stage at one point in time. On each frame, Director draws the sprites in ascending channel order.

At first, I naively thought that like Photoshop I could just toggle “layers” and be on my way. I quickly discovered, however, that in Director this convenience was exactly the functionality that we lacked. The score needed keep track of exactly what sprites were on stage at any given moment. It would no longer be enough to blindly draw whatever the current frame said its sprites were. Our engine needed to think more horizontally, rather than vertically.

After creating numerous test movies to understand Director’s rendering rules, I devised a simple mask-based approach. I took stock of all the dirty areas of the screen and just iterated over the channels to redraw the parts of sprites that intersected there. Nothing could be simpler, but it took me quite a while to understand all the eccentricities of the code and complete the refactoring required to get the engine alive again.

My approach gives fairly complete consistency with the original. Many difficult commands, like zoomBox, can now work properly. Moreover, my work enabled puppeted sprites – sprites controlled by Lingo, and not by the instructions in the score – to work properly. If a sprite is puppeted, its channel in the score is simply not updated with sprites from new frames. The same sprite stays on stage until its puppet is disabled. This was not possible when every frame brought a new set of sprites to draw. But it was trivial when the score kept track of channels and sprites and chose when to update and redraw them.

My changes did not enhance much visually – in fact, there was a regression that drew hideous white boxes instead of transparent hotspots. However, this morning I got to the bottom of those issues, and now Captain Hammer looks as fearsome as ever.

Stay tuned for my adventures with QuickDraw and our Macintosh GUI emulator!