Category: Uncategorized

GSoC is nearing its end,  today is the “suggested pencils down” date, means there shouldnt be much more happening that some cleanups and bugfixes. Im already doing exactly this for a couple of days so I guess its time to round up what got done (and what didn’t)

TFMX

The modplayer is fully implemented and accurate in regards to the player in the MI-Executeable. Actually it has some more effects and parameters implemented and can play alot of TFMX/Pro Modules, albeit with some issues (TFMX was often modified for games so this is no surprise).

Integration in scumm engine
uses 2 seperate Player instances as this allows music and effects to peacefully coexists without the latter stealing channels from the former, and this also means the mixer-settings (Sfx, Music Volume) are automatically supported.
Downsides are the additional memory required for the second player-instance (music-data is shared) and higher CPU-load if more than 4 channels are active. Neither is a serious drawback as the additional Memory is less that 700Bytes, and the CPU-load should be manageable. When I tested MI Amiga on a PSP, running music had an impact of additional ~10% load and that was with a downclocked CPU (111Mhz instead of 222) and highest Samplerate possible(48KHz).

MaxTrax

The modplayer is complete, it implements everything the original player (sources from UADE) did, even some optional effects (microtonal and modulation, both can be enabled with compiler macros). I cant vouch for the accuracy, I only tested the songs from Kyrandia so far and they only need a subset of the player. Especially the optional effects would need testing. There are some audible differences as effects are played in the intro as channels from the music are replaced – this is not 100% like the original player.

Integration in kyra engine
Since there are rarely more than 2 channels used for music, Im only using 1 instance for music and effects. The game is a bit of a problem as support for the Amiga-Version is currently incomplete in ScummVM, lots of tables are hardcoded in the executeable and even different sections of the game behave (slightly) differently when starting music/effects. I have implemented support for all sections, but I am unable to thoroughly test it. For sfx 2 tables are needed which are directly in code ATM, but should find their way into kyra.dat later.

Desktoptracker

no progress, the modules aren’t in the documented format. See this blogpost.

I had alot trouble getting myself motivated, probably a reason for this is that I lack a clear goal on how far I should rework Paula, all the classes using it and the scummvm-players which would need to be adapted aswell.
After all, everything I do has to be merged back into ScummVM`s main development branch, and modifications need to be tested aswell.

A probably bigger reason for the lack of progress is that I moved to the countryside/suburbs for a week, with the good intention of continuing work while having relatives and friends nearby. This dint work out too well and last weeks extremely hot weather dint help one bit =). I`m moving back back in the city later today and this should allow some more serious attempts at continuing work.

Progress on Maxtrax has slowed down as the player is in good shape already. I looked a bit at Desktoptracker an am now pondering what to do next.

The problem with Desktoptracker:
I exchanged some mails with Kirben, he knows a bit about the Acorn-Version of Simon.

• the files arent matching the available specifications, either ALL the music files have similar errors when transferring them from disks (the OS dint complain about anything when transfering, so thats unlikely), or they arent simple Desktoptracker Files but compressed or otherwise modified variants
• He tried running the game under an Emulator, but getting only noise instead of music (he cant rule out emulator issues tough)
• I tried the modules under xmp, which doesnt recognise them directly. If I fix up the headers it quits or crashes.

So I consider this task dead.

Improve Paula (Amiga-Sound-Chipset Emulator).
During implementation of TFMX and Kyra I already added some missing features and slightly changing behaviour to the existing “Paula” class. The scumm-engine also has a “Player-MOD” class which does the exact same thing in a slightly different manner. The first step of the task would be to create a single class which enough functionality to be sufficient for all Amiga-Games supported in ScummVM… or even better just be able to serve as base for any kind of Amiga-Soundformats which use some halfway sane methods of generating sound. Primary this will require an interface to allow direct hardware-access or OS-level abstraction (which allowed queing up many commands).
The second step would be to make every Amiga-Format use this class and, as far as I am able to, verify everything is still working. This is the bare minimum regarding deliverables for this task, further improvements are possible like providing high-quality interpolation or emulating the switchable lowpassfilter.. and then there could be ample time wasted trying to optimise the interpolate for speed without sacrificing noteable quality =)

C64 Soundsupport
I seen this issue creeping up in the devlist talks. Not sure how much work this would be and in what state the gamesupport itself is. If there is a pressing need for that I could look at it.

The player is incomplete as it misses a few commands, but they arent used by the intro and its almost certain that they arent used elsewhere. As Maxtrax has an eerie resemblance to MIDI (down to using the same structures in files) I looked at a few classes and it might be possible to actually use one existing MIDI Emulator instead of using Maxtrax to emulate MIDI-Commands on Amiga-Hardware and then emulate the Amiga-Hardware. Im not 100% sure its possible, but I will have a good look at it later.

The integration into the kyra-engine is good enough for the whole intro – which is as far as the Amiga-Version runs currently. Kyrandia has several seperate tables for section of the game (intro, game, final atleast), each with seperate tables for sfx, tempo and even seperate code handling music/sfx commands. Finishing support will surely drag beyond the Summer of Code

I will post another blog-entry today or tomorrow on whats next.

• ~23 May – 1 June
  The History
  I chosen to start at the hierarchical top of the Format then work my way down. ie. Song -> Tracksteps -> Patterns -> Macros. I used ScummVM classes right from the start which saved some time vs my initially planned approach to make a small selfcontained project at the start, there where some troubles with the linux buildsystem but nothing major.
  As I progressed I realized that the TFMX Format is way more complex than I initially though, creating songs for it is more like programming than composing (maybe one reason why only a handful musicians used it). I felt like I`m falling behind my schedule more and more. I did however reverse parts of the executeable and used this to verify some of he weirder stuff in tfmxplay, some things turned out to be different in MI
• 1 June – 4 June:
  I now can follow the “path” of the modules. Follow is a bit of a big word, 4-5 patterns just outputted their current command and parameters – and interleaved with each other. Impatience and some disappointment showed up. I now worked feverishly to get evenough macros implemented to be able to hook it all up audio-output.
  The first versions had ugly, distorted noise but you still could figure out the parts of songs.
• 4 June – 6 June
  Now I had a full player, only I needed to detect the bugs that kept it from outputting something recognisable. This proved to be difficult and took me 2 days just to find some rather laughable bugs (storing 16bit in a byte?). For the next task (maxtrax) I plan to start at the lowest level, so I can verify new code soon and not code for 2 weeks and then have to hunt around endlessly. It might need some more code for beeing able to instantly test new additions, but I think steady progress is more important than beeing halted for indeterminate time searching bugs.
• 6 June – 12 June
  Things were clearing up again and after finding the few showstopping bugs I quickly could pad out the player, find playback issues and verify nonworking code with the MI-Executeable. Guess I made up the time lost this week. I also managed to put hook the player into ScummVM so the game would work with Sound.
• 14 – 18 June
  I was busy with University so I wasnt around much. Nevertheless I good a good look at how sfx are handled. Turns out that TFMX has functions to queue both external macros or patterns (channel 7 is reserved for external patterns it seems), complete with priority levels and “lifetime” to determine when and if other effects/music should be replaced. Luckily ScummVM is only using macros and priority 0, its however pretty messy keeping track of them (probably not at all).
  I implemented a first version with sfx by using 2 Tfmx instances, 1 for music and 1 for effects, this allows hooking up both to the right mixer channels. Not sure if I might use only 1 instance in the end.
• 18 – 20 June
  Actually played a bit of “Monkey Island”, found out that the mansion-scene was quite bad in the last implementation. Some effects depended on beeing stopped or muted by the next effect, and I had to implement that for this scene. The game should work well now.
• 22 June – 1 July
  This week I was occupied primary with University. Still found some time for a few fixes.
• 1 July  – 11 July
  Merged in changes from trunk before starting work on Maxtrax.
  I initially thought about reversing the format the hard way (disassembler) as the sources linked in ScummVM Wiki are already adapted for a Player.. and have wuite some code decated to some pluginframeworks. I wanted to base my player on the original source. Reversing proved to be difficult as Kyrandia uses the Amiga`s libraries and I havent found a way to fed the definitions to the disassembler.
  Compare this to TFMX which hits the Hardware directly, finding the related code was merely a matter of looking at the exception-vectors (eg. which code gets called on Audio-Interrupts) in an emulator. Of course it also helped that TFMX is structured rather nicely and placed right at the beginning of the executeable.
  Without fast progress I was trying to locate the origins of the assembler-sources and found out that the original, almost unmodified sources are available in UADE.
  Progress was steady since, even though I dint worked very dutifully this week I had hit the milestone of beeing able to iterate through songs.
• 12 July  – 14 July
  Some cleanups tiresome debuggin work later, the player now reproduces the intro-song. Need to test some more but it appears to be pretty truthfully aswell.
• 14 July  – 18 July
  The Now
  What Im doing now is looking at how sound-effects are played (and which kinds of effects are used in Kyrandia) and figure out how to add them into the game. I plan to have support for that at the end of the week.
• 19 July  – 10 August
  The Mystery
  The following weeks are a bit of a mystery, I will have a look at the Desktoptracker-Format, but from what information is available I fear its not enough to base an implementation on it.
  So Im still uncertain on how to use this time, more on this and some possible options at another blog-entry
• 10 August – 17 August
  Expect 3-4 days to improve TFMX/maxtrax
  Remainded is codecleanups, testing and preparations to merge the code back into ScummVM

State

• Monkey Island + TFMX
  The game should work fine with both effects and sounds, player implements everything the game uses. Differently to the original I however use 2 instances of the TFMX-Player for sound and effects which should be noticeable in the game even if its rare that both are used at the same time.
  There are some cleanups necessary, primary for making both instances share data, currently the music + samples get loaded twice in seperate blocks of memory.
• Legend of Kyrandia + Maxtrax
  Commandline player can load and play songs (called scores in this format). Output appears to be thruthful, but more comparisons (only tested intro-music so far). Effects are totally missing currently. The original player implemented communication by using 2 OS-conform “servers” – the game sended commands to those servers and the commands got processed asynchronously. Need to figure out which commands where used and how to hook the player into the kyra-engine.

I managed to implement the Maxtrax Format far enough to be able to play Kyrandia’s tunes, albeit there still are some issues like hanging notes. Which means my timeline still holds up as this state was planned for the end of the week =)

The planned next step would be to improve and integrate the player into the Scummvm so Kyrandia can play its sound and music. This will be an interesting issue, as I learned that the Amiga-Version still isnt running stable and it actually crashes atleast with my version during the intro. So testing sound effects and music wont be easy.

I started working on Maxtrax yesterday, means I`m pretty much in the same spot like in the beginning. Things are a bit different this time though as there is an commented assembler-source available directly from the author, and the format itself is simpler than TFMX.
Still it will take some time to get familar enough and code a player, which means there wont be much new blogposts in the near future.

For “Monkey Island” I compiled another set of Windows and Linux Binaries (Editor’s notice: This file was hosted on the now defunct hosting provider hotfile[.]com. It is not available anymore.), this Versions allows choosing of the color-palette (as there are different versions of MI in the wild). Default is the palette of the later versions, to chose the other palette set rendermode to EGA.

I cleaned up the code and a few bugs and I`m satisfied with the current state, so I`m releasing the binaries today.

Some common terms for all versions:
The binaries posted are only intended for the Amiga-Versions of “Monkey Island”, use the regular ScummVM for everything else.
To report bugs, first make sure they are bugs – means check if the Amiga Version behaves differently by running it on an emulator or on the real hardware. Differences to MSDOS or other versions dont count.
A special note to music-quality: The Amiga had a lowpassfilter which is emulated by UAE/WinUAE (you can disable it in the options), but not within ScummVM. Also those Emulators have several expensive interpolation-methods which again ScummVM is lacking. Especially the missing lowpass filter is noticeable, but apart from that the output should be very close.

So, without further delays, here are executeables for Win32/Linux32/Linux64:

Nothing stops you to compile it yourself of course, look at the previous Blog-Post for the Link

Update: While playing a bit further in MI, I found out that the effects in the mansion where a mess. I fixed them and compiled a new set of binaries (I really hope these will last longer): (Editor’s notice: This file was hosted on the now defunct hosting provider hotfile[.]com. It is not available anymore.)

Update2: Please report any bugs in the ScummVM-Forum

Support for soundeffects dint take much of my time, but I`m in the middle of exams and tests right now so work on ScummVM slowed down alot. Im still improving the code and will put out some new binaries to test soon.
Since I seen requests regarding the code in the comments, I`m working on a branch of ScummVM, so you can use this to look at the source or compile it yourself. By default it will compile ScummVM with working TFMX Support, defining TFMX_CMDLINE_TOOL will result in a spartan commandlinetool to playback TFMX Modules (aslong they are called “mdat.monkey” and “smpl.monkey”).

As said I will post some binaries soon, and since “Monkey Island” is working with sound and effect, this will conclude my initial work on TFMX and next up is Maxtrax/Kyrandia. I would ask you to use this time to thoroughly test “Monkey Island” and report any weird behaviour or inaccuracies regarding sound/effects. I`ll probably make a thread on the forums for this, seems more suited for this than commenting on a blog.

I dint so much coding work, but I researched a bit on the inner workings of ScummVM and Monkey Island.

A TFMX-Song doesnt need to have an end, it can loop a section indefinitly or even just repeat the same sample all over. Yet Monkey Island has a coulple of scenes like the intro where the game waits a preset time to show up the MI-Logo and then waits for the song to stop before loading the next scene.
Yet the corresponding TFMX-Song (#4) has no ending, the only reason it does end is because I added some additional checks that did work nicely in the case of the intro. The same checks fail on another Song (#3) which runs endlessly. So its time to explore how this kind of timing works in the original Amiga-Version and how ScummVM expects it to work.

Timing on the Intro is twofold, I could not find any reference in the TFMX-Player that would signal the time the song “really starts” (after just some ambient sound, sorry cant express this better 😉 ), even if that would be easy to archive it hasnt be done. Instead the original executable must have a own counter which runs independent of the music-playback, but Im not going to reverse the whole executeable to find out how this is done exactly.
ScummVM does poll regulary for a “musictimer” and counting 1/25 seconds since songstart seems to just about work (guess the timer is a multiple of the 50 VBlank interrupts per second). Still would be good to hear from some ScummVM Devs if there are some custom hacks for MI Amiga already in place.

Ending of songs however is something thats controlled indirectly by the TFMX-Player. A TFMX-Song has special commands to talk back to the application – writing values to a shared block of Memory, perfect to notify if a specific pattern has be reached. And the songs in question (#3 and #4) both use them to signal the end of the Song. They do so by writing a 1 to a shared Memory Block at start and writing a 0 to the same position at the end. (and song #3 just loops and overrides the value with 1 again!?).
Since ScummVM aint providing a function which directly correspond to that I will have to figure out a way to make this work seemlessly, it already polls if a song is still running so I will try putting a check for this flag. Still that might not be enough as not all Songs set this flag to 1.