Sev's ScummVM notes

In the years I had the numerous requests to add a Patreon page. There you go: https://www.patreon.com/_sev. It is a way to support me personally (if you care) and if there will be at least several patreons, I plan to use this platform for helping me to choose the area where I spend my time.

In the meantime, I will be posting assorted reports on the things I added to ScummVM. There are so many topics to work on and so limited time. Perhaps support from patreons could help me use my time more wisely.

After a long while I finally managed to get first graphics displayed in ScummVM…

Of course, this also includes full code for scene loading. I would say, that the engine is way over-engineered.

It is the very first screen shot. Hkz is working on it. ScummVM portability is demonstrated at its most.

Today LucasArts re-released some of their adventure classics on Steam. Immediately people started to wonder about ScummVM. Let’s see…

It came out of the blue, and nobody ever expected such turn. We even had internal jokes that LEC would rather release “Jedi Fights the Green Tentacle” than another Monkey Island game or any of their old adventures.

I think ScummVM did its job really well. We kept a big fan base of the games alive and even growing. We released our software for number of platforms, including such hot devices as iPhone and Wii. Finally LucasArts seemed to realize that there is market for 2D point and click adventures, and their new re-releases are geared towards just that. Well, kind of.

There was a certain speculation on the technical side of the re-releases, basically whether they will be ScummVM-compatible or not.

Thus, we already checked the binaries of some of the games, no surprise there, they use Windows port by Aaron Giles, not ScummVM. As of the ScummVM compatibility, we see that here were some changes that make the games incompatible with the current ScummVM version. They are very trivial changes, though (moving the content of one file into another), and do not seem to constitute any kind of copy protection. Of course one can wonder if they did this deliberately to break compatibility with ScummVM, but since it is so trivial to deal with this change, this seems somewhat unlikely.

Thus, it is matter of five minutes hack for adding support of these games to ScummVM, as basically it is just a slight format change, not any kind of copy protection or encryption. Adding such kind of support would remain completely legal, just as ScummVM was always legal with running other game versions, and nobody rights will be violated.

However, we decided to abstain from it at least for now. It seems that LucasArts’ folks are pretty aware about us, we’ve been even mentioned on their twitter account, and there is a chance that we could be contacted. We do not want to interfere with their sales and/or any upcoming plans for other platforms. Also we are really open for cooperation in this regard and will be more than happy to make any upcoming releases run with use of ScummVM.

I am sure you saw the news post item about certain GPL violation.

Let me present here some more details about the case.

Disclaimer

Fingolfin and cyx are bound by the agreement signed by the lawyers. They are not allowed to talk about the case. Other developers are not, and there is pretty handful of them who know at least beginning of the situation. We are not happy about the outcome, cyx and Fingolfin included.

Short story

As you may imagine, we are closely watching the market for ScummVM-supported games. One of the main reasons is provide an up to date information on our Where to buy games list. Thus, we were pretty glad and interested to hear that there are upcoming Humongous Entertainment titles on Wii. Thus, as soon as the games were on sale, couple of developers bought them.

Suddenly something quite unexpected happened. I got 2 independent reports from 2 completely different sources, even from different countries that apparently there is ScummVM inside. I was not surprised by the fact of ScummVM usage, as there are already Revolution games and GOG.com who sell ScummVM as part of their distributions, still a surprise was that there were the SCUMM-based games. Thus, I quickly requested to inspect the box for copyright statements, and as such, there came a surprise.

Clearly, it was a violation. Further inspection showed that it was even a blatant violation. I addressed to my friend, google.com search engine, and I found following facts:

• Apparently Atari contracted porting work to their long time partner, Majesco Entertainment.
• Majesco Entertainment in turn subcontracted the work to Mistic Software, a Canadian-based game shop
• Mistic Software has its development office in Chernigov, Ukraine, locally presented as Mistic Soft LLC who did the actual work
  

These Mistic Soft guys got the game assets and sources (this is obvious from the fact that they were able to insert their names into game credits seamlessly), but instead of performing actual work they went a cheap way.

Inspection of the game binary (we dumped it on SD Card on Wii with homebrew channel, it’s pretty small) showed that they based their work on ScummVM 0.9.0, inside there were full credits to ScummVM Team (with couple placeholders for their names, though they left them empty), as well as such goodness as “This might be broken — please report where you encountered this to Fingolfin” or “Tell sev how to reproduce it”. Judging from the symbol names they rolled out their own backend, and used Nintendo tools for that.

Then we performed some playtesting on the real thing in order to find clues that ScummVM is inside. Pretty fast we found a 0.9.0-era bug in Freddi Fish. Let me demonstrate it:

Note that gfx glitch on Freddi’s left eye. It is right at the beginning of the game, when Freddi reads a letter from the bottle, and is easy to spot on the console. There could be other bugs, but that was enough. Now this bug is fixed in ScummVM.

The development

We were quite shocked. It looks like Mistic Soft took money for all the porting efforts but instead of that just stacked together a ScummVM build! Of course, it was easier (my estimate, that it requires about 2 full time weeks to port ScummVM to new platform), since as I suspect, original interpreter is not very alignment-safe and is not endianness-agnostic as ScummVM is.

Thus, we started to discuss what should we do with the case, and decided to approach gpl-violations.org. We got a lawyer which specializes on such violations, and pretty soon we sent a letter to Atari in Germany (the fact that the games being sold in Germany eased the work considerably).

First they started to deny everything. This is a pretty common thing among the lawyers, and also as I understand, they were not aware of the issue. Consider this: These programmers from Mistic Soft just provided the binaries to their Quality Assurance department, and the games worked well (as ScummVM does). They found no bugs, passed the work to Majesco, and those reported about successful project completion to Atari. Of course nobody except the developers (and perhaps their immediate managers) knew whose code is in there.

So, first Atari lawyers thought that we are going to sue them for zillions of Euros, and they prepared well for the fight. Once they got to know that only thing we are demanding is stick to GPL, they relaxed and the negotiations went much better.

We discussed that they could put stickers on those games which are being sold, and put ScummVM copyrights on new prints, mention us on their site. And it was really nice at first, but pretty soon the lawyers found that Nintendo explicitely prohibits use of open source software together with their Wii SDK, and as such it was a big fault from Atari side.

Thus, instead of being nice they decided to go and hit us badly.

At this very point let me go a bit astray and explain some things about ScummVM. ScummVM is a collection of game engines. There are two major categories of them. Part of the engines is based on original source codes provided by the IP right holders, and another part is partially produced with reverse engineering. Reverse engineering, and particularly disassembling is legal in majority of countries, Germany included. Another approach for reverse engineering is so called Chinese Wall technique is legal in all countries, US included.

SCUMM engine, later versions of which is used in Humongous Entertainment games is one of the engines which was reverse engineered, and that is what the lawyers tried to blackmail us for. In result this all turned into a completely different, much more complicated case.

Thus, there were two routes: 1. Go to court. Defend ScummVM, defend our techniques, spend a lot of time, money. 2. Arrange some kind of settlement agreement.

The finals

Thus, the facts were:

• There is a GPL violation (their denial has to be proven in a court, strings in executables and the bug above clearly show it)
• Atari could not release source codes because of Nintendo NDA
• Atari could not put GPL clause because of Nintendo NDA
• Atari could not “buy out” ScummVM from us
• There is no possibility to double license ScummVM, at least SCUMM engine
• We do not need any money as a “bribe to keep silent”

And we started to discuss. The rough details of the final settlement were: Fingolfin and cyx can post an agreed “press release.” They are not allowed to talk more about it. There is a period of time in which all current copies have to be sold. Any copies beyond this period or any reprints get fined with quite high fine for each new/remaining copy. The remaining stock has to be destoryed. There will be no single usage of ScummVM for any of upcoming games without our knowledge. Atari makes a significant donation to Free Software Foundation. Atari covers all expenses on gpl-violations.org lawyers.

Of course, we are not happy about this, as GPL is being violated, but it was better than to dive into many months long trial case.

So, when you will see Pajama Sam, Freddi Fish or SPY Fox games for Wii now in stores, know, that ScummVM is in there. And when you hear about ScummVM, remember, it is a commercial grade software which even Atari is not ashamed to put their label on.

In the last years ScummVM became one of the most known open source projects. But it was not always like this.

Today eriktorbjorn stumbled upon the very first mention of ScummVM on Slashdot. What is amazing about this that some comments have links to screenshots posted, and one of them is still alive! I’m attaching the mentioned screenshot, as the server which hosts the original is quite slow. The screenshot is dated Sun, 04 Nov 2001 07:44:03 GMT which was “2nd pre-alpha release,” or 0.0.2. It was after 45 commits to public CVS!

Of course, it was not just the beginning of the development, as with his first commit Ludde submitted an already semi-working engine, still you may see this piece of our project history.

At the time ScummVM consisted of 13,658 SLOC (physical source lines of code) and was able to run 6 games, including monkey1, monkey2, indy4, indy4 demo, DOTT and Sam&Max. Compare it with today’s 663,214 SLOC, and I am not ready to tell how many games we do support :).

I think you’re aware of James Woodcock’s Music Enhancement Project. James was applying his talent to rearranging music for several titles supported by ScummVM.

One of his projects included complete soundtrack for Discworld.

During last several weeks he renewed his work on this project, and finally there are noticeable results. This is a huge step forward in the game atmosphere (though some may complain about not being faithful to origial), and I expect that many of our users will choose play this marvelous classic on new highs of audio experience.

James recorded a sneak peak preview of the soundtrack in action which you may see in the attached video.

The work performed is huge. It is over 100 minutes of music significantly rearranged and recorded on a modern Yamaha-branded sound equipment. It took almost 2 years for him to work over all 107 tracks. Besides, the work has been blessed by Terry himself, so this is going to be first sound pack officially supported by ScummVM and distributed from our site.

You will be able to play any version of the game with this new sound track, it will be just matter of extracting the archive into the game directory. Of course, original untouched MIDI music will be still present for those seeking true nostalgia (Who may also prefer to play fullscreen with 320×200 resolution:) ).

Stay tuned!

And now is the promised video:

(Source: https://www.youtube.com/watch?v=ZsJ0iutFCvA)

As you perhaps know, for last 1.5 years ScummVM serves as an Ultimate AGI engine, especially since Kari Salminen aka Buddha^ joined us first as a GSoC member and then as a full-blown developer.

To the date ScummVM covered more AGI games than any other open AGI interpreter. The missings are AGI0, AGI1, AGI2.01 games as well as Tandy CoCo3 conversions.

Though there was a thing which bothered me for quite time, that is the fact that famous AGI Specifications is not fully converted on our Wiki. Important sections were missing, and not all existing text was in a good shape. So, finally I got some time and completed the project.

Now you may go and explore it on our Wiki. The big help for me was a wonderful online HTML to Wiki Converter – tables tool written by Borislav Manolov. I highly recommend it.

I wanted to tell this story for a quite long time. A draft of it was sitting on my HD for couple of years. And now, there finally it is. A piece of ScummVM story, a saga.

All of this started back in May 2000. Daniel Balsom was so delighted in furry Inherit the Earth game, so he began to peek into its executables in order to make it run on his Windows XP. YARN (Yet Another Remake? Nifty!) was born. The aim of this project was to run both Inherit the Earth and I Have no Mouth and I Must Scream, though practically straight away he switched to The ReInherit Engine, supporting just the first game.

I may say that it was a horrible time. Daniel ran original game with DOS Debugger and tried to figure out what is going on there, staring into a hex editor and documenting it on sheets of paper of different sizes and then attempting to curl his head in order to write C code. He told me that there was a big pile of those papers and he even offered to scan them. Ugh!

In some time the game intro was showing with handful of bugs, but next real world event killed the work. Original Inherit the Earth director, Joe Pearce, announced that he acquired full rights on the game and that there is a re-release in the works, compatible with Windows XP. Thus, whole purpose of the ReInherit project was voided, and Daniel was discouraged to continue any work on it. It was around December of 2003 when last commit to the engine was performed.

As soon as in March 2004 I found this abandoned project and offered Daniel Balsom to join our team. He was pretty busy at that time (and still is), but offered his help as a consultant. We got his sources, ported it over to OSystem (core of ScummVM hardware abstraction layer), then started to map out functions in Interactive Disassember.

In parallel I tried to contact Joe Pearce of Wyrmkeep and inquiry about the game engine soruces. Daniel told me that he already tried to contact Joe about a year earleir, but that was exactly the time when Joe was in process of the game rights clearance, so obviously, he was not able to pass any game materials. We thought it was a fat chance. Mr.Pearce was not fast with replying my mails, but suddenly, at end July 2004 a miracle has happened. We were given complete sources for ITE!

Thus, we closed our IDAs and started to look straight into C/Asm code of the original. According to the accompanying license we could not use more than 5% of original code verbatim, that’s why instead of porting original code we continued to work with ReInerit codebase and fix it along the way.

I found a guy from Russia, Andrew Kurushin who joined our team in December, 2005 just for helping us with the engine. After a year of work, in August of 2005 we finally announced support for Inherit the Earth game.

After completing support for ITE we decided to pursue the ultimate goal of YARN, that is to support both games of the engine. Thus, work on I Have no Mouth and I Must Scream started. This time we did not have any source code. The engine developed quite a lot, but still it was the same. I started very slow and time consuming work of mapping it out. Pretty quickly we stuck, as there were subtle changes in actor handling which prevented game from playing, and actor code is one of the most complex in the engine.

The work was practically on hold for 2.5 years until June 2007 when Filippos Karapetis joined us to resume development. During that period I tried to encourage developers to help me with the engine, but due to complexity of the task, I did not succeed. Andrew became superbusy with his real life, and I had much of other work to do with ScummVM.

Filippos lead the development by a huge leaps. No longer that just in 3 months I Have No Mouth was announced as supported and that is without source code (!).

This was story about SAGA (Scripts for Animated Graphic Advenures) engine. But it continues even today. There exists SAGA2 engine, on which games Dinotopia and Faery Tale Adventure 2 are based, and Filippos recently started to0 work on them. We also got source code for FTA2 on a similar license as ITE, and I hope that pretty soon yet another couple of games will be added to ScummVM collection of fine games.

Basically that means that there are just a few days that Tinsel engine will be in SVN. No, 0.12.0 will not contain it, as the engine is too raw, and there are still some legal questions to resolve.

I really hope that this will not interfere much with the testing season, as it will still not appear in the daily builds. But for our porters it will mean that we the compile-testing 😉 will start on it for all supported platforms which will let us work on the portability issues.