Category: Uncategorized

The GSOC coding period is now officially over, and the tool(s) are pretty much done too so. Only bugfixes left to be done, and still testing is an issue.

I’ve been plagued by disease since friday unfortunetly, and been suffering debilitating  canker sores, which hurt LIKE HELL, tensed up so bad I got headaches and whatnot and got some medication which kills all feeling in my throat. So I’ve spent most of my time sleeping (which was the best cure according to my doctor). This means I’ve not been able to get as much done as I’d hoped.

There have been some minor changes in the GUI the past week, and  several bugfixes. Fingolfin helped out with Mac OSX gui, which was very helpful since it’s hard to design UI for a platform you’ve never used ;).

The first page has changed look to include a few buttons with the activities, rather than the checkboxes (note that this is not final yet however), also note the helpful ‘help’ button:

Also, the About dialog has been improved upon, with a much better look:

In part due to my downtime the last five days, I’ll probably continue to submit patches during the coming week (and after that maybe?).

I’ve spent the last two days fixing small things that’s been bothering me for awhile, and a few nasty bugs (especially one where files opened using a File constructor would not set xormode properly, resulting in all data being read to be corrupt.

Also, I’ve made some improvements to the wiki page on the tools, however it’s far from complete yet. The changes there will then be merged into the README. Still missing are details on the tools, and the formatting could certainly be improved.

Also, the recent changes in trunk has been merged again, so all functionality added during GSOC should now be present in the branch.

By now, the master tools work fine, both in their CLI and GUI versions, and using them sure is better than the previous methods ;).

This is how it looks when you run the CLI (under Ubuntu), compression tools don’t have much fun output (it’s running compress_queen):

And the GUI (under Windows Vista this time), a small detail that has been added (and is visible in this screenshot) is the name of the tool displayed on the line at the bottom:

Almost all things mentioned in the last post have been fixed, but rather than an ‘advanced’ dialog. You now select Target Platform instead, which loads some defaults optimized for the target platform, instead of forcing you to set defaults yourself.

Tools now filter filenames more effectively, and should in most cases be able to guess exactly which tool you want, or atleast offer the choice of two or three tools. The only tool that is not auto-detected is compress_tucker, which must be run in advanced mode (or for the cli, using the –tool option), since it accepts an input directory, which can’t be effectively selected. A suggestion was to redo the tools so they all accept input directories, since the user might not now what file to select. However, making that work without hickups would require significantly more research into every game than making the user pick the correct file.

Documentation is also not complete (is it ever), and tomorrow, I’ll improve the current  tools page to include instruction on how to use the new GUI and CLI instead.

Another thing that does work yet is the optional –mac parameter for compress_agos (and to a lesser extent, -f to compress_gob and  –speech/music-only to compress_sword1), the simplest option would be to simply create two instances of the “compress_agos” tool, one with the –mac parameter set and with slightly different names, which would then appear as two separate  tools to the user.

Also, I rightly deserve admonitions with the screwup three days ago (with the CLI not compiling), I’ll try to make sure it does not happen again.

For tomorrow, I’ll try to improve the wiki documentation, and possibly some small fixes to the GUI / CLI, such as help strings. The README (and help pages) will be updated after the wiki, as the formatting (and publicity) of it makes it easier to track errors.

Just a pick of how the tool looks like while running under Ubuntu.

Note that especially the file picker controls (not on the screenshot) are still horribly malsized. I will fix that soon. ?

Now, the tool works from finish to end!

But there’s still plenty of work left to do (it never ends)

• Ensure that all the tools still works, since there has been some very large code changes, and it’s easy for subtle (and blatantly obvious) bugs to sneak in.
• Write documentation on how to write new tools, and how to maintain the current one, since it differs greatly from the method used previously (I would say it’s much easier now, but how to use all the features might require an extra page).
• Write the master CLI interface, so the tools can be used with the GUI, and make sure the GUI can be used from the command line (which also brings the benefit of making it possible to “open with” the gui tool).
• Add GUI options to set default settings under an advanced menu.
• Rewrite all tools to output to a directory, so the user doesn’t need to bother with knowing the output filename, as it not important for most tools.
• Add a way to run the tools that accept directory input, as that right now doesn’t work. This is a tricky one as there is no control to select either a file or a directory, and displaying one control for directories, and one for files will be very confusing to the user. Any ideas anyone?
• Add proper filename filters to all tools, so that the GUI can filter out tools as effectively as possible. This is very important to be able to entirely skip the page were you select the tool to be used. As all tools that take input on the form “*.*” are displayed in the list of alternatives, also, for the tools that accept files with the same extension as another tool, add some binary peaking or similar to resolve it automaticly).
• Improve the UI, for example, display a short description of the tool when it’s selected in the list in a small infobox, and display extra text next to the file input fields explaining what is expected.
• Add progress bar support to as many tools as possible.
• Add extra pages for the tools that have extra parameters, like compress_agos which has an extra –mac parameter, note that some of these parameters are specific and can be ignored for the GUI, like the ability to extra only a single file using extract_kyra.
• …alot more…?

For anyone who wants to test the tools (on linux), checkout https://scummvm.svn.sourceforge.net/svnroot/scummvm/tools/branches/gsoc2009-gui and run make. Note that you must have wxWidgets with atleast version 2.8.9 installed.

For Windows / OSX, the project files  have not been updated yet, but if you want to compile it, it’s enough to stick all the .cpp and .h (except the degob, dekyra, decine, desword2, de<anything>) files into one project and compile it. And you need wx installed on Windows too, obviously.

I suggest you test some of the tools, as I’m almost certain many of them have bugs, and reporting which one’s would aid the debugging greatly!

Been 14 days since my last update, so it’s long overdue, but it’s been fourteen days of progress!

The tools themselves has went through a major redesign [in code], going from simple C IO to a much better File class, all tools now also have a common base class which implements functionality such as parsing arguments etc., which makes that part much easier than before, only requiring the code actually related to compression / extraction to be included in the tool .cpp file, plus some hints to the GUI.

Also, the tools no longer use exit() to indicate failure, but throws exceptions instead. This way they can be run inside the GUI process, rather than in a separate process. And it also handles cleanup through RAII classes, however, many tools still leaks alot if an exception is thrown. Still, that is not a great issue as they will typically only be run once per process, and the OS will handle any cleanup.

Due to these changes, the tools are now forced to be included into the GUI (or some other application), and can no longer be compiled individually (actually, you can compile with -DSTANDALONE_MAIN to force a main for each tool, but it cannot be done with the standard make). A master CLI tool will soon be added to support compiling and using the tool for platforms without wxWidgets, or if you just don’t like GUI applications (note that the GUI will support the same CLI).

Also, the GUI has been restructed some. Now the input file is queried before the tool, so file type detection can be done on it and significantly shrink the list of tools from (or remove it completely), note that the code for the automatic detection does not work fully yet, and I have not commited it. (btw. GIT would be great, keeping local branches in SVN is a pain).

Another addition to the GUI is a progress bar in the process window, to show the user that something is going on if the tool produces no console output. However, updating progress needs to be done explicitly for each tool, so by default it’s just a ‘pulse bar’ that indicates general progress (usually just bars moving back and forth). Looks much better than previously! One thing that bothers me is that the compression still spawns subprocess for lame etc. to compress the files, which (atleast on Windows) makes a million console windows pop up, freezing the UI. Not very user friendly as you most likely want to let the tool run in the background while you complete other tasks, which is made impossible by the windows constantly popping up.

I expect the tool to be working by the end of the week (well, it already is, but not finished per se.),  and after that I’d very much like some help to test it, since I can’t think of all test cases myself. Also, I don’t own all the games supported by these tools, so I unfortunetly can’t even test if some run without errors (I’ll make a list soon, so people can see what needs to be tried out).

I’ll make another post soon about what will be done in the upcoming weeks, which is mainly GUI refinements and small tool improvements. Especially the last conversion squeezed in alot of hacks into the tools themself and that might be nice to get rid off. ?

For the last week, I’ve been working on getting the GUI together, and now, it’s almost complete in it’s basic appearence. Now all the pages, from starting the guide to completion, are functional (except the input selection, which is working through a local hack). And the “Display output folder” doesn’t work either, since I couldn’t find the wx function to open an explorer window (neither could #wxwidgets), perhaps it will require platform-specific code in the end, I hope not…

From the game selection screen, many games are missing, and help with making a more complete list would be most appreciated (will ask on the mailing list as well, to reach everyone).

The Input/Output page is looking kinda cruddy, in my opinion, especially for tools that accepts multiple inputs, as there is no room for help text. Perhaps it would be better to split that into two pages?

You might be wondering why the output window ain’t displaying any, well, output. The reason is that redirecting the output of a subprocess turned out to be quite tricky, so instead I’ll do it another way:

I (joostp included in me ? ) have decided it would be easier to simple compile in all the different tools into the GUI directly, rather than run them as subprocesses. This way, we don’t have to ship 25 extra executables with the program, but rather a standalone file (if the image files are also compiled into it, and it still works even without the image files, just display orange/light orange stripe instead of the logo). The size of the tool would also be much smaller, since we avoid the overhead of static linking of libraries. The only issue with this would be that some tools don’t do cleanup very well, and all of them use exit(), the exit issue can “easily” be solved by adding a #define if main is exported to replace it with “throw int(retval)” or something akin to it, I’m not sure it will cover all cases, though, the alternative of going through the code to make everything return true/false on success would be a huge task.

The linker errors with stuffing all tools in one project I have already resolved, but I’ve tried commiting three times now and SF SVN servers seems to be down for the count. Those GIT offline commits would really be nice right about now. ?

So far, progress is going well! And after the UI has been tweaked and is functional, I can start work on the heuristic to deduce tool automatically, and the ability to run the tool using command line arguments (using it as the “One tool to rule them all”).

Unfortunetly, I’ve been sick for the last few days (fever etc.), which has slowed down my progress alittle. But I’ll recover it tomorrow! I still managed to do the first commit with some GUI yesterday. Which cheered me up since the GUI is actually the primary goal (no matter how much fun CLI is *cough cough*) this summer.

This is how it looks now, with my background in the background:

I took the banner from the ScummVM site, I hope the creator doesn’t mind. It looks good after all!

As a piece of trivia, I can say that the line between the content area and the buttons were by far the most time consuming, since I couldn’t figure out how to position it correctly (with the text) to not make it grow in ugly ways. wxSizers can be tricky sometimes. ?

There already is a system in place to go back and forth between the pages (there is another page to choose game to compress for, nothing for extraction (commited) so far).  However, I’m not entirely pleased with it since it requires the current page to know which page preceded it, it would be better if the pages were entirely modular and could be moved around. Which could be accomplished through some C++ template + inheritence trickery (basically, make a base class with a virtual method that returns a WizardPage *, and overload the method with a template subclass).

Going to sleep now to shake of the last remants of the fever. Goodnight!

Since my last post, I’ve gained SVN access and been busy working on converting all command line tools, although I had some slowdowns (partly due to last week being graduation week for many of my friends, and partly due to sneaking in some work on the GUI).  But now the task is finally complete, with the possible exception of the mandatory bugs from such a large rewrite.

I had some issues, both with my computer (the harddrive died half way through the week) and clumsiness on my part, as I missed a few files when commiting, and even lost one completely. Forcing an unfortunte hour or two spent doing the same work all over again. But it’s all over now.

Most of the tools also still lack thorough testing, something I will be doing in tandem with developing the GUI, as testing them will be much more durable with a nice front end.

If you’re interested in the changes to the tools, it’s most noticeable through looking at the README.  Just compare the old with the new.  The change in the examples is quite appearant. But still the the one tool to rule them all is still missing (which will be the same as the GUI tool, accepting command line arguments).

While working on the CLI, I’ve also been strating with some work on the GUI, and I’ll soon follow up with a post (and a commit) on how it looks.

Despite my lack of SVN activitity, which is due to the fact that _sev (the administrator) is on vacation and cannot give me access yet. But still I’ve been working hard, although my progress might have been delayed alittle by the excellent weather, but there’s nothing but rain coming up for the weekend so…

Conversion of most of the extraction tools are done now, only some bugs to sort out, and today I started working on the compression tools. I planned to finish the extraction tools rewrites by Monday, and as it looks right now I might overshoot that by a few days, however, after that I’ll start with the GUI, which is much more fun in my opinion, especially when writing original code, and as such naturally develops faster. ?

Once the GUI is working, I’ll make sure to post some screens for you to see, as I guess most won’t feel motivated to compile it themselves.

Cheers!