FREESCI ENGINE FOR SCUMMVM

Here’s where things stand:

• I’ve figured out how to make the SCI engine use the game info provided by ScummVM, and implemented those changes.
• FreeSCI now does something sensible for configuration settings – querying the ScummVM Conf Manager when it needs to do so, otherwise setting parameters to match what ScummVM expects its output to be and leaving the rest to the backend.
• The FreeSCI debug console now communicates through a ScummVM console

I’m now in the process of testing, debugging, polishing and documenting everything.

Suffered a pretty nasty hard drive crash last week, but I think I’ve finally fixed everything. I’ve been working on dealing with configuration data. This tends to fall into a few categories:

• Options that exist in both FreeSCI and ScummVM but are handled by the ScummVM backend. Here, you just want to figure out what values will clamp the FreeSCI output to what’s “expected” by ScummVM.
• Options unique to FreeSCI: Here we query the ScummVM configuration manager for the setting, but anticipate failure and do something reasonable when it happens.

• Options that exist in FreeSCI and ScummVM and require the engine to modify its behaviour : These are the trickier ones, as they tend to involve mapping parameters onto each other and the occasional switch() construct.

I’ve also been reading over the detection code to see what improvements are needed.

As promised last week, you can find a tarball of the latest version at:
http://www.its.caltech.edu/~szerrade/scummsci.tar.bz2

Disclaimer: This is mostly for the benefit of people curious as to the current state of the project, and probably shouldn’t be used by anyone actually looking to play a game.

There are about 1700 lines of new code in this build (as compared to the repository jvprat had at the beginning of the summer). About 1200 of those are in new sound and file IO drivers, and the rest are scattered throughout the dozen or so files that had to be rewritten to use the new drivers.

The testing has been limited to King’s Quest IV and a couple of game demos on my Ubuntu PC- cross-platform/cross-game testing is reserved for the final stage of the project. To build, do:
./free_configure
make

The new files are described below. All of them are in engines/sci/src/.

scicore/file_hander.c and include/file_handler.h : This routes the file access system calls to the relevant drivers. It’s patterned after the FreeSCI exe.h/exe.c drivers. These calls are sort of a compromise between the FILE* methods used in FreeSCI and ScummVM’s methods. Getting them to fit together has sort of been a square-peg round-hole problem.

scicore/scumm_file_driver.c: This is the part that turns various FILE*-style calls into scummvm commands. The implementation is not always 100% faithful – for example, I was not able to figure out how to translate scanf into something the Scumm savefile drivers could work with – but I’ve tested it (mostly on King’s Quest IV) and it works well enough to access resource files and save/restore games.

Files are divided into save files (for which read/write is allowed) and resource files (which can only be read). This is necessary because some backends (Playstation 2 comes to mind) force you to use memory sticks for save state and read-only disks for game data. Which type of file you open depends on the driver method you call. Generally, if you want to open a file for writing you should explicity use the sci_save_write_open method.

scicore/unix_file_driver.c : This maps abstract driver commands to C FILE* methods. Currently disabled in favor of:

sfx/seq/scumm-adlib.c: This implements a sequencer that uses ScummVM’s Midi drivers to play Adlib music. The hardest part of writing this was debugging the custom instrument patches.

sfx/device/scumm-midi.c: This implements the Scumm drivers as a FreeSCI midi device, allowing them to be used by FreeSCI’s GM and MT32 drivers. You can change the driver used by modifying the invocation on line 85 – since the next stage in the project is to overhaul the config system, I didn’t see much point in setting up a runtime control for this only to have it replaced in a week.

Here’s the current state of things:

• I’ve written an abstraction layer that allows for multiple file access “drivers”. This means one driver for Unix-style FILE* files, and another for the methods in common/savefile.h and common/file.h
• The ScummVM file driver is finished. The UNIX one is mostly done, but I’m still trying to figure out how to have it deal with some of the path information. The driver is defined in engine/sci/src/include/file_handler.h and file_driver.h, and the actual driver and implementation are in engine/sci/src/scicore/file_handler.c and scumm_file_driver.c.
• The FILE* invocations scattered throughout the FreeSCI source have been replaced with calls to the driver. There are a few exceptions to this, but as far as I can tell none are very critical.
• There’s now a sequencer that uses ScummVM’s Adlib midi driver, and a midi device that uses ScummVM for output. The instrument setup in these is unfinished.

I was hoping to be done with files and audio at the beginning of this week, but the time spent making the file handling modular has put us somewhat behind schedule. The revised plan is to put in overtime and finish all the File/Audio stuff within the next three days, bundle the finished product as a stable build, and then compress the next step (configuration file management) from fourteen days to nine or ten. This should put us back on track by the time midterm evals roll around. We also have three weeks of general-purpose “debug and test” time at the end of the schedule to work with.

Nothing especially interesting to report lately. I’m still working on the sound and file implementations, and hoping to have them done by the end of this week. Spent a while today trying to figure out how to create a wrapper for Scumm’s write drivers that mimicked fprintf, so that I could simply find-replace away some of the fprintf calls in the FreeSCI source. Eventually discovered the vsprintf/vfprintf functions, which made things a lot easier.

Then we move on to dealing with configuration.

I’ve started uploading some of the my code to a darcs repository.

I ordered a few games to test things with. In the meantime, I’ve been using the Space Quest III Astrochicken demo to test the basic sound code. This means I’ve spent a lot of the past week listening to chickeny music, puncuated by the occasional explosion.

Wikipedia’s article on “Astro Chicken” begins thusly :
This article is about the hypothetical self-replicating spacecraft concept. For the side game from Space Quest III, see Astro Chicken.

So we do that, and learn the following:

In the fictional storyline of SQ III, the Two Guys from Andromeda were enslaved by ScumSoft. ScumSoft, in the game, is an evil software company, maybe partly inspired by SunSoft^{[citation needed]}, but also the Lucas Arts adventure games (the engine was called SCUMM)^{[citation needed]} and Microsoft (some claim Elmo Pug was a caricature of Bill Gates^{[citation needed]}). At ScumSoft HQ, they were forced to create games such as Astro Chicken.

So that’s kind of ironic.

Adlib driver seems to be working pretty well now, so we just need to modify the open() callback for each of the sound card/patch formats.

Here’s a diagram jvprat made. It shows the basic flow of MIDI info:

I picked FreeSCI’s file handling as an initial target because it looked like it would be fairly straightforward: replace anything in the FreeSCI code that used file IO with invocations of the ScummVM file and savegame classes. In the process, it would be possible to get used to the ScummVM and FreeSCI source code, making future tasks easier.

This gets complicated when one adds the additional goal of keeping the source code as close to the FreeSCI main branch as possible. Simply going through and replacing anything with a FILE* near it results in a lot of changed source code. This means that incorporating any future changes to the FreeSCI main branch into our code will be a bit of a pain.

One option is to build an abstraction layer into FreeSCI; this layer would encapsulate both the FILE* implementation and the use of ScummVM classes, so that one could choose between them. This is similar to the approach that FreeSCI’s graphics code uses to allow for different means of outputting graphics. An alternative is to keep copies of the relevant files and pick which one we want at compile time, depending on whether we’re compiling as a standalone copy of FreeSCI or a ScummVM engine.

Either way it involves a change to the FreeSCI base, so we’ve been getting feedback from other developers. Until it gets sorted out, I’m devoting some attention to the sound output.

Today marks the official start of Google Summer of Code. For those who asked, here’s what the tentative schedule for the next few months looks like:

May 26 – June 8 : Official start of GSoC. Alter FreeSCI’s file I/O routines and any other OS-specific code to use ScummVM’s OSystem class.

June 9 – June 22 : Implement a ScummVM sound driver for FreeSCI. This driver will use ScummVM’s sound methods as its backend, replacing the existing platform-specific drivers.

June 23 – June 29 : Debug and document existing code, finish all file and audio work.

June 30 – July 13 : Modify the engine to use configuration settings specified in the ScummVM options menu whenever possible. For example, setting the music driver in ScummVM should cause our engine to choose the appropriate resource files and output driver. In any instance where the engine would query FreeSCI’s configuration data, we modify it to instead consult the ScummVM configuration manager.

It should be noted that the the FreeSCI configuration system has some options that cannot be set using ScummVM’s menu. For these options, the engine should still query the ScummVM configuration manager. This way, anyone seeking to add a more general configuration utility later on will not need to modify the engine extensively.

July 14 – July 20 : Modify game detection code. Currently, the FreeSCI engine uses its own detection algorithms to decide which interpreter to invoke. We want to modify it to use ScummVM’s Md5-based game detection, resorting to the FreeSCI code as a fallback

July 21 – July 26 : Integrate FreeSCI debugger as a subclass of the ScummVM console.

July 27 – August 4 : Polish the user interface. Test on multiple non-Windows/Linux platforms. Resolve any outstanding portability issues.

August 4 – August 18 : Debug and document code.

Hi, readers.

Over the next few months, this blog will serve to document my efforts to complete the transformation of the FreeSCI Virtual Machine into an engine for ScummVM. I’m undertaking this as part of the Google Summer of Code program, under the mentorship of Jordi Prat, who began the project.

During the next week, I plan to post some information on the current state of the project and what we hope to accomplish. In the meantime, feel free to comment with general suggestions and comments.