Month: July 2008

Thought it was a good time for a quick update. I’ve made a lot of progress on the keymapper since my last update, with most of the infrastructure in place now. Most of the interface to the rest of the system is implemented (in the Keymapper class). The KeymapManager (where the loading/saving and automatic mapping will be done) still needs work, and will probably be quite tricky! Anyway, here is a summary of my major progress and any design decisions I have made since my last post:

• KeymapManager now uses Domain classes to separate out global and game-specific keymaps – this was inspired by ScummVM’s ConfigManager. Each domain has a default keymap, which incorporates the idea of a “super-global” keymap. Also, this means that a game can provide a specialised version of a global keymap, because keymaps are only requested by name (not domain) and if a keymap of the given name exists in the game-specific domain it is given priority over a global keymap with that name.
• Keymapper has initGame and cleanupGame methods. The idea here is that these methods need to be called before and after a new game is run. In initGame the keymapper looks at the ConfigManager’s active domain (to obtain the gameid). In cleanupGame the keymapper tells the KeymapManager to free up all the game specific keymaps in has loaded. This should provide support for when we have to return to launcher feature.
• Where it comes to actually adding keymaps to the Keymapper we have addGlobalKeyMap(name, map) and addGameKeyMap(name, map). All these methods do is register the new Keymap with the KeymapManager (which is where the loading / automatic mapping of the keymap will occur).
• So the idea is that whenever part of the system needs a specific keymap they will call one of these methods in their initialisation code. Then when they actually want that keymapping to be used they call switchMapping(name).
• The actual mapping of the keys is done (see Keymapper::mapKey). The key point here is that both KEYUP and KEYDOWN events are mapped, and the mapped event is modified to make sure it correlates with the type of event that prompted it.
• The instantiation of the Keymapper and the calling of mapKey has been integrated into the DefaultEventManager.

So I am well on track with my schedule at the moment. For the next few days I will go back to working on the Virtual Keyboard, as there are a couple of features that are still needed.

One last thing…thought I’d better let you know that I’m going to stay with my gran up in Scotland from tomorrow till Monday, which means I probably won’t be online much. But, I’ll be taking my laptop and should be able to work without it.

So its been a while since my last update, and for most of the time I’ve been busy planning the rest of my project. I’ll get on to that later, but first let me sum up exactly where I got to with my coding.

Virtual Keyboard

• Dragging feature
  • made it possible to drag the keyboard around the screen when part of it that was not a key was clicked
  • also made it snap to the sides of the screen
• Fixed bug in event delivery
  • found that when many key presses were delivered after keyboard was shut the SCUMM game that I was testing with would ignore the first few keypresses and only register the final one
  • fixed this by implementing a delay in the event manager’s delivery of queued events
• Handling of GUI and VK displaying at same time
  • before I would unconditionally clear the overlay before displaying the keyboard, but if the GUI had been displaying then bad things would happen once the keyboard was closed
  • to get round this, when the GUI was active I would save the overlays contents when the keyboard was opened
  • then this would be the background to blit the keyboard into, every time it had to be redrawn
  • and when the keyboard was hidden, the original overlay contents would be written back to the overlay
• Documentation
  • added comments to public interface of classes
  • added explanation of XML file format to VirtualKeyboardParser.h, which will later be added to wiki

After doing this, I started researching the keymapper task, so that I could get a better idea of all the stuff I have left to do. After reading up and asking questions on the mailing list I was ready to come up with a breakdown of my design.

Keymapper breakdown:

I chose to break down the keymapper task into four main classes that I would need to implement.

• Mapping class:
  • will store details of a particular key mapping and offer simple methods to carry out the mapping
  • will also provide methods to edit the mapping (which will be called by the Edit Mapping GUI dialog)
• Keymapper class
  • will provide interface to rest of system, and will be accessible via EventManager
  • will store a pointer to the active Mapping class
  • and other state such as the active gameid, etc…
  • will provide method to switch to a different Mapping (which will be called by engines or other parts of system, e.g. main menu dialog)
  • engine will call methods on game start that will allow it to specify game-specific Mappings (and the UserActions they require)
  • backend will call methods (or possibly subclass?) that will allow it to specify which HardwareKey’s are available
• MappingManager class:
  • will be responsible for managing all the different mappings that are available
  • involves the loading and saving of mappings to the standard ini file
  • will load global mappings on creation, and will load game-specific mappings when prompted by engine calling methods on Keymapper class
  • if no valid game-specific mappings can be loaded from the ini file the automatic mapping algorithm will be run
• Edit Mapping GUI dialog:
  • will be implemented using the existing GUI toolkit
  • using the descriptions from the UserAction and HardwareKey structs will make the dialog easily understandable
  • the will be a number of label / button pairs
  • each label will contain the UserAction description text
• • each button will contain the mapped HardwareKey description text
  • clicking the button will start an event loop, and the first event that is recognised to have come from a particular HardwareKey will re-assign that HardwareKey to the UserAction in question

Now that I had a good understanding of the second part of my project I came up with a rough TODO list.

TODO list:

Virtual Keyboard

• prepare a complete virtual keyboard pack
  • waiting on original images from rob (IRC: sanguineh)
• add textbox to show what user has entered – 2 days
  • maybe a special part of the keyboard image could be marked with a rectangular imagemap area
  • then this area will be filled with text as user clicks keys
  • will then need left/right arrows and backspace keys to alter this text
• shift key functionality – 2-3 days
  • currently the keyboard only offers functionality of a caps lock key because mode can be changed to caps but then must be switched back to normal
  • should be easy to offer a key that switches the mode but only until another key is pressed, at which point the mode is switched back to the original mode
  • this leads to a separate point – do we require the virtual keyboard to offer functionality of control / alt keys?
  • if so, then shift functionality should probably be done without mode switching but by implementing modifier keys in the keyboard – I think this is the most flexible approach

Keymapper

• Mapping class – 1-2 days
• Keymapper class – 3-4 days
• MappingManager class – 4-5 days
• Edit Mapping GUI dialog – 2-3 days
• Integration into system for testing purposes – 2-3 days

The last part of my planning extravaganza was to plan how I would spend my time over the remaining weeks.

Plan of Action

I have decided to get going on the Keymapper class in the next few days, before hopefully coming back to finish the Virtual Keyboard once I have some good images available. 18th July – 23rd July

• Mapping class – complete
• Keymapper class – complete
• MappingManager class – basic outline done

24th July – 28rd July

• Finish remaining features of Virtual Keyboard

29th July – 8th August

• Bulk of the MappingManager class
• Integration with rest of system

8th August – 11th August

• Edit Mapping GUI Dialog

12 August onwards

• Cleanup and finishing off code
• Writing documentation
• Preparing for final evaluations

Done! Sorry for boring you guys with all my planning, but I guess this benefits us all! Time to get back to the code…

Time for a quick update on the huge progress I have made in the past few days.

Virtual Keyboard Parser

Thanks to Vicent’s great work implementing the XMLParser base class this was a relatively simple task. There was a lot of error handling code to write, and I added a couple of new features to the base class. Apart from that it didn’t take me long at all to have a parser that was parsing the file format that I had earlier settled upon.

Keyboard’s main loop

This involved writing the event loop that would be run while the virtual keyboard was visible. It involved catching all the mouse click events, and passing them to the processClick() method. I also had to ensure the screen was being updated (so that the mouse cursor continued to move) and that EVENT_QUIT events were handled properly (so that closing the window had an effect).

Displaying the keyboard

The main challenge here was implementing keycolor transparency for the keyboard bitmap. To do this I created a subclass of Surface, SurfaceKeyColored, that provided a blit() method. This method would be passed a source Surface and a transparency color, and any pixels in the source Surface that was not the same color as the transparency color, would be copied into the Surface. Once I had this class, I could grab the overlay into an instance of SurfaceKeyColored, then blit the keyboard bitmap’s Surface into it, before copying the SurfaceKeyColored’s pixels back into the overlay. Below are some images demonstrating it in action:

Imagemap testing

After I had the bitmap’s being displayed and the event loop implemented, I could test my image map code that I had written last week. The processClick() method of my VirtualKeyboard class simply had to pass the mouse co-ordinates to the corresponding ImageMap class and it would return the ID string (ie. the target attribute) of the matching area. I created a few image map areas for some keys of my test image in GIMP (making sure to use rectangle and polygon areas), then used the exported HTML in my keyboard pack XML file. After a couple of bugfixes I got the ImageMap working perfectly, proving that my Polygon::contains() method was correct.

Delivery of virtual events

I had written the VirtualKeyboard class to save all the key presses it created into a queue. But, once it was no longer visible it had to insert these key press events into the main event queue. I decided that the VirtualKeyboard should have a pollEvent() method, that would return an event if the keyboard was closed and there was a key press in the queue. Then all I had to do was modify the DefaultEventManager to first poll the VirtualKeyboard, and then poll the backend, but only if the VirtualKeyboard did not return an event. The DefaultEventManager class was also made responsible for showing the virtual keyboard when a special button press was made. While writing this code, it was decided that the VirtualKeyboard class should be instantiated from within the DefaultEventManager class, because they were very closely related.

Next…

Tomorrow I will try to create a complete keyboard pack, with multiple modes and resolutions, together with complete image maps, and event specifications. This will allow me to fully test my current code, as well as test the little bits of code that I am still to write.

Before I start work on writing the parser for the virtual keyboard’s XML file, I need to tie down the format of the file. Yesterday I came up with the following as my first idea:

<keyboard modes="normal,caps,symbols1,symbols2">
  <mode name="normal" resolutions="320x200, 640x480">
    <layout resolution="320x200" bitmap="normal_320x200.bmp">
      <map>
        <area shape="rect" coords="84,64,246,146">
          <event type="key" code="97" ascii="A" modifiers="shift" />
        </area>
        <area shape="poly" coords="2,12,25,9,34,8,19,22">
          <event type="mode_switch" name="caps" />
        </area>
        ...
      </map>
    </layout>
    <layout resolution="640x480" bitmap="normal_320x200.bmp">
      ...
    </layout>
  </mode>
  <mode name="caps" resolutions="320x200, 640x480">
    ...
  </mode>
</keyboard>

Here each mode tag refers to a different keyboard display, and each mode has a number of layouts – which provide different resolutions for each mode. Doing it this way – rather than having layout as the parent of mode – was a pretty arbitrary choice, with a slight advantage of being more flexible.

The main flaw I noticed here was that the event tags would have to be repeated for each different layout resolution. If we can assume that all different layout resolutions will contain the same events then we can do this:

<keyboard modes="normal,caps,symbols1,symbols2">
  <mode name="normal" resolutions="320x200, 640x480">
    <event name="A" type="key" code="97" ascii="A" modifiers="shift" />
    <event name="shift" type="mode_switch" name="caps" />
    ...
    <layout resolution="320x200" bitmap="normal_320x200.bmp">
      <map>
        <area shape="rect" coords="84,64,246,146" target="A" />
        <area shape="poly" coords="2,12,25,9,34,8,19,22" target="shift" />
        ...
      </map>
    </layout>
    <layout resolution="640x480" bitmap="normal_320x200.bmp">
      ...
    </layout>
  </mode>
  <mode name="caps" resolutions="320x200, 640x480">
    ...
  </mode>
</keyboard>

Where the target of each area refers to an event tag by its name property. This is nice because it means the map sections now use pure HTML IMGMAP syntax. This means the target property can be specified when using an image map tool, and then the output just pasted into the file without any additions.

Therefore I think this format will be the basis for my first attempt at the parser. However, any comments on it will be much appreciated!

We are now in the 6th week of the Summer of Code and through a number of circumstances, both in and out of my control, I am pretty behind schedule. My coding in the last few days has shown great promise that I can quickly get back on top of things, but I think it is time for me to lay down a revised schedule for the coming months. The first stage of my schedule is pretty tight – but I have ample time at the moment and am willing to work all the hours in the day to make up for lost time.

Now – July 12th

• Write all the code for virtual keyboard
• ALREADY DONE
  • implementation of image map functionality (ie. polygon/rectangle intersection code)
  • Skeletal version of main virtual keyboard class
  • XML file format has been defined
• TO DO
  • XML parser – a case of extending Vicent’s XML parser to parse my file format = 2 days
  • Keyboard main loop – to handle mouse clicks when keyboard visible and draw the keyboard to the overlay layer = 1-2 days
  • modification of event manager to receive events from keyboard and handle mode switching of keyboard = 2 days
  • integration of virtual keyboard into rest of system = 1-2 days
  • Other stuff – extracting from zip file & creation of example keyboard pack = 1-2 days

July 13th – July 14th

• Prepare mid-term evaluation material

July 15th – July 20th

• Clean up of existing code
• Make sure code is commented clearly
• Extensive testing of virtual keyboard

July 20th – July 25th

• Detailed research into keymapping task

The following is slightly vague, due to my lack of knowledge

July 25th – August 2nd

• Writing of general keymapper interface
• involves providing the link between engines and backends
• working out how each is going to tell the keymapper what it needs to know

August 2nd – August 9th

• Writing of algorithm to automatically map actions to buttons
• gui to allow user to change these mappings
• saving of these mappings to ini file

August 10th – August 20th

• Additional key mapping features

August 20th – Sept 1st

• prepare for final deadline
• Lots of testing, code cleanup and documentation

PS – progress report coming tomorrow (including details of XML file format)