Back and gone

Okay, that was quick. Haven’t happened before when swapping a HD from one laptop to another that everything goes so smoothly that you don’t have to reconfigure anything at all on the software level (some kind of Celeron -> Pentium 2 and a modular precompiled Debian kernel).

Unfortunately I have also lost interest in continuing working on my project due to slight instability of mind, so I am quitting.

For anyone who might want to continue this project, it would be helpful to merge the “agi” branch of my github scummvm-tools repository to master. It contains two finished extraction tools and a disassembler that should work with any V2 logic resources and also with the V1 logic resources found in Black Cauldron. I just wrote a simple command line parser for it so that anyone can readily try it without modifying the source.

Work on my scummvm “booters” branch is incomplete but as far as I know it shouldn’t break anything with any V2 games, so I think this would also be useful to merge. Of course it could take some additional testing before the next release.

Finally I added the partly reverse-engineered disassembly files of the three games with my few comments. These are all produced by running the games in DOSBox and using its debugger to dump the code and data segments while AGI is running, then disassembling the binaries with ndisasm (shipped with nasm). I added the MD5 sums of the disk images to the corresponding files. The repository is just a handy place to store them temporarily so there they are.

Personally I think it was an interesting ride, I have even considered buying some games for trying them after seeing cool things like the Dreamweb asm->C++ converter. I part from IRC but you can of course still contact me by email (pitkajus, paju.oulu.fi) if you have any questions about the code I have produced.

Bye and have fun.

Taking a shortish break

Power supply of my laptop accidentally broke down (hope it didn’t break anything else), and after talking to my mentor, I decided to take a short break and to try to come back up with some kind of development environment after the next week or so.

Status update

EXTRACTION TOOLS: There are two extraction utilities that are finished, one for Black Cauldron booter and another for DDP booter. These extract the game resources from the disk images to the common VOL, LOGDIR, PICDIR, VIEWDIR, SNDDIR, OBJECT, WORDS.TOK files that are found in all AGI v2 games. The DDP extraction utility does not extract OBJECT or WORDS.TOK files because these do not exist or serve any function in the game.

V1 DISASSEMBLER: I wrote a disassembler for V1 logic scripts based on the game resources of Black Cauldron. Actually I first wrote a V2 disassembler taking the instruction tables from ScummVM, and then added another instruction table for V1. Both V1 and V2 handle the IF (FF … FF) and ELSE (or GOTO; FE) constructs in the same way. The disassembler tries to detect IF-ELSE constructs in one pass and indents the code based on the output of this detection. To correctly detect them in all cases would need another pass through the code, which is not difficult to add, but is currently left undone. In the case of incorrect detection only the indentation of the output gets wrong because I decided to always output GOTO statements even though they would be detected as ELSE. In the output the current offset is written to the left of the code and to the right into a comment the bytes that were matched as the instruction. The output could be made more C-like by translating instructions like assign(a,b) to a = b, and increment(a) to a++, and the same for some condition commands in IF expressions (this is what the V2 disassembler from Sarien does). The V1 disassembler translates the logic scripts of Black Cauldron mostly correctly. Many of the instructions are identical to their V2 counterpart, some have a somewhat different functionality (program control instructions), some have different number of arguments (random comes to mind), and a few others do not exist in V2 anymore. To make it more friendly a few command line options could be added, that is, to select between V1 and V2, and to control output formatting (this is taken in account in the design, but the few lines of code for parsing the command line are not written, so that you currently need to recompile for changing these).

AGI ENGINE IN SCUMMVM: I have refactored the AGI engine so that another instruction table for V1 has been added. Also the disk image detection code and game resource loaders (based on the extraction tools) are written for BC and DDP. It turned out the program control is handled differently in V1, that is, how different logic scripts get executed from the logic.0 base script. I have an experimental implementation in my local branch, but there are still some details to be worked out, involving a few internally changed variables that are used by the scripts (I estimate one or two days of work to get it about right). I actually got the intro running by fiddling with some of the variables in the debugger. 🙂 After this it should be mostly fixing a correct implementation for a few instructions that behave differently in V1, since all other game resources are identical to V2 and the code for them is thus already written (with the exception of SOUND resources, but I have already implemented a player for those, one that could maybe still be cleaned up a bit).

DOCUMENTATION: I have written something about how game resources are organized in DDP. This still needs to be done for BC as well (see the extraction tools). Then maybe the V1 instruction table could be written down, along with the differences to V2 and information how the internal variables are handled. Also I fixed the information about V1 SOUND format.

KING’S QUEST: The interpreter in this game seems to be different, but I haven’t delved into the details yet. I guess all other game resources have identical formats to V1 and/or V2.

Everything in DDP is bad for children!

Now that I am more familiar with Donald Duck’s Playground, I can tell you my honest personal opinion about it. In short: it is the worst game I have ever played, not less because it is supposed to be an educational game for children. The only thing it does is to teach your children to become the worst kind of brainless consumer robots on planet Earth by insisting the player to iterate through the most dull game mechanics I have seen, that is, by having Donald Duck to work odd jobs to earn money in order to buy uninteresting toys from a few stores for his nephews. The only good thing about this game is that the player cannot die in it, unlike many other Sierra games where the player infamously dies around every corner unless you magically know the exact sequence of correct steps to proceed.

If my job here was to redesign the game the first thing I would change would be to make Donald have to take his nephews, along with a few bottles of milk prepared by their loving mother, and make an idyllic trip to local landfill to collect useful items from the dump.  These items would then be brought to Gyro Gearloose  who would help the nephews in building a space rocket out of the recycled parts. The rocket would then be launched to space in order to go teach manners to three-headed negro monkeys of developing planets.

Those items from the dump that couldn’t be used for building the rocket could be used for positive terrorism and thrown against the windows of the now-useless stores. Maybe it would be even better if the stores were replaced by only one store with coffee and alcohol in millions of kinds of bottles and packages. Special coupons for use in this store would then be distributed for everybody, and the actual use of these coupons  would be closely observed by all goodwilling, respectable citizens, and if even one unused coupon was found the co-citizen in question would have to face a year-long correction camp for his own good. Yeah, the nephews, being technologically advanced citizens, could use the extracted alcohol as fuel for their space rocket.

Game resources in Donald Duck’s Playground

I have now figured out how the game resources of Donald Duck’s Playground (DDP) are mapped to the disk and created a tool to extract them. The resource formats are identical to what they are in AGI v2 DOS-based games, except that the SOUND format is different. Also, there might be slight differences in the LOGIC scripts since this game is using the AGI version 2.001. I also started to document the booter games in ScummVM wiki.

The SOUND resource format is actually very straightforward although it is less structured than it is for the later interpreter versions. It consists of varying sized sequences or “rows” of bytes that each end with a zero. These bytes are simply the values that are to be sent to the sound chip in the IBM PCjr (SN76489; three square wave generators and one noise generator). Each row contains the data for all the generators and is fixed length in time. Since all the bytes are supposed to be sent to the same I/O port in the IBM PCjr, there are no restrictions to how they should be ordered (with the exception that some updates need two bytes and the chip expects them to come in order). In fact, a row often contains only one zero, which means no data is sent to the sound chip, effectively skipping the row.

I suspect this is the format for AGI v1 SOUND resources as well. I will update the wiki about the SOUND resource format once I have verified this.

What comes to the reimplementation, I have added the detection for this game to ScummVM’s AGI engine. I have also implemented the player for SOUND resources. The game works but there are problems in some parts because ScummVM does not yet support some AGI v2.001 specific things. Will come back to this at some later time.

Briefly about boot sequences

Generally, once the BIOS, the start-up program of PC, has performed its initialization procedures and found a bootable device, in this case a floppy disk drive, it loads the first sector of a floppy to the address 0000:7C00 and transfers execution to that address (or any other address that points to the same physical location, I guess, which would mean that you cannot be sure of the exact Code Segment at this point). After this, the boot sector program loads the main program into memory, typically by using INT 13H to read sectors from disk, and finally begins to execute the main program.

I expected that the one 512-byte boot sector would be enough to load the main program into memory, but alas, all of the three games seem to be wastefully loading an intermediate program from another sector (or even sectors) to do the loading! By the way, the data on the disk is ordered so that the “head” is flipped after a track (9 sectors) is read. The disk format is 5.25-inch, 40-track, double-density (40 tracks, 2 heads, 9 sectors per track) with the total capacity of 360KB. There are probably a lot of details to be found about the loader program, some of them maybe explaining why it spans multiple sectors, but I figured it is easier to just let the loader do its job, and when the game is running use the DosBox debugger to dump the one 64KB segment, containing the whole main interpreter program, to a file.

A tested algorithm to find the AGI core with pointer tables to instruction handlers is to do the following in DosBox debugger: (1) Set breakpoints to memory locations that contain INT 13H (used to load data from disk), (2) in the game move to another room so that new data needs to be loaded, firing a breakpoint, and (3) step through, possibly using breakpoints, until the core is found.

Introduction

Hello, and welcome to my GSOC 2011 diary.

My project proposal to improve ScummVM’s AGI engine has been accepted as part of the GSOC 2011 program. My task is to add support for the three bootable floppy games that are listed below. These are some of the games that were created early in the phase when Sierra’s AGI engine (Adventure Game Interpreter) was still rapidly maturing together with the games that were being developed. After this period, most of the energy was put into creating content and only little changes were made to the underlying engine.

  • King’s Quest I (AGI “v0”)
  • Black Cauldron (AGI v1.12)
  • Donald Duck’s Playground (AGI v2.001)

On this blog I will reflect my progress on the project. Two topics that will be discussed almost surely are the reverse-engineering and the reimplementation of these games. For details, you can follow my github repository where I push the code I produce, and my ScummVM wiki page for pointers to more specific documentation about the internals of the engines and games mentioned.

Well, let’s hope it will be a nice productive summer together with the main ScummVM team!