Category: Uncategorized

Hi All,

First of all my apologies for showing up late this week, as something urgent came up this weekend.
Next, some updates on my work till today :
Last week I started with implementing the exit Dialog using ListWidget, which is shown to the user before exiting. The advantage of using a ListWidget is making the exit dialog scrollable.

Next I moved on to implement AudioCD tests.
Backend provides an API to play audio tracks from a CD Drive or a hard drive if no tracks are found in the cd drive.
To play tracks from hard drive, the search is made in predefined directories using SaerchMan.
To test if the implementation is fine or not, I have created 4 mp3 files and play them in order i.e
track1-track2-track3->track-4.
Then I verify the ordering from the user and decide if the implementation is fine or not.

The audioCD data tracks are MP3 files, converted from text using text-to-MP3 converter.

Next I moved on with producing audio outputs with variable sample rates.

Finally, to test MIDI, i brought changes from trunk to my branch, through a SVN merge.

Next, I figured out a bug in the Message Dialogs. The problem was, the hotkeys were not working as they should be.
I explored this and found out that the Initializer list for the button instances was incomplete. Finally  committed the patch to the trunk.

Next I did worked upon improving tests/UI as pointed out by Eugene.

I now plan to finally get done with MIDI tests, and send my code to review.

__

Neeraj

Hi everyone,
Here are some updates on my work last week.
I started up with fixing the boundary palette problem in the palette rotation test. The problem was I was taking two colors for background and foreground, these were now included in the palette to be rotated. This also involved implementing colored progress bar so that they look nicer.
Next I moved on with Sound Subsystem tests. The intent was to test if the backend was able to identify sounds or not using beeps. For generating the sound data I used the PC speaker emulator, the stream could also have been a file but that would couple it with FS tests.

Next I found out that the progress bar displays the testsuite counts to the sizes even if none of them are enabled. The next commit intended at displaying correct information, like if X out of Y tests are enabled, the bar should be like XX of X tests instead of XX of Y tests.
Similarly,, as required, a finish test zone was added to mouse event tests. This would be useful for keyboardless backends.

Next was GUI, of sound subsystem tests. As earlier uses of GUI in the engine code were using some stuff, so this work involved a bit refactoring of GUI code in the engine with introduction of Generic class and a few generic methods.
This was used in generating Multichannel sound to test if Mixer works fine. The user can pause and play a particular channel at his choice using the GUI.

Next, I tried an entire run under valgrind, this identified new-delete mismatches at two places, these were fixed as well.
Next I moved to fix a bug in configFile handling, with testsuite rerunning, the size of the config file key-value pairs was increasing, for each rerun. The fix to this was simply an one-liner :).

Finally, we have a GUI in the end, which displays a summary of the current execution of testbed. One can rerun the testsuite from that dialog.

For next week, I have:
Monday/Tuesday : AudioCD/ Sample rates
Wednesday onwards : Midi Tests, testing and documentation later on once Midi tests are done.

Thanks

Hi everyone!,

Here is a brief summary of my work last week.

I started with implementing progress bars, now the progress within a testsuite and the overall progress are reflected by two text lines of form (X of XX tests), In addition the progress within a testsuite is also shown graphically as progress bar, which gives a cool look ;).

Next I moved on to some more improvements in GUI as suggested by Eugene. There existed a button to select all tests at once but no such button to do the reverse. This was balanced by making the same button do both the jobs, i.e if number of selected tests is more than deselected ones, it shows “Deselect All”, “Select All” otherwise. Other improvements included, modifying the GUI texts, and some formatting.

Next, their were tests which when ran in interactive mode showed no activity, code was added to make them display the information about what they are working with.

Finally, I started on enabling the testsuite and test selection using a config file. Initially, I rolled out my custom implementation to read and parse config files, later i figured out that this could also be done using the ConfigFile class. So I switched my implementation to make use of that class.
Now we have a layout-like this:

[Global]
# Global-parameter-name value pairs here
isSessionInteractive=true
[Testsuite]
# Decides if testsuite itself is enabled or not
this=true
# Tests within the testsuite go here
dummyTest=true

This does two useful things:

• It saves your configuration each time you customize and remembers it, so that you don’t have to customize every time if you needed a particular customization.
• One can also manually edit t his file to affect exactly which tests of a particular testsuite be executed.

Next, I also worked on writing a testcase for cursor-trails in GUI bug.
The problem was if the shake offset is set, and then GUI is shown, cursor leaves trail in GUI and its very difficult to click any button. I first reproduced this problem which quite ended up as a test for it.
I have an offset of 25 pixels set and I then prompt GUI dialog to verify if the user sees a trail or not, If there are no trails, user will probably click the “No” button on right, if there be trails, clicking any button would be difficult and user may end up pressing “return”, which is interpreted as “Yes” i.e trails were there.

Finally towards the end, I enabled the logging feature.
Now testbed outputs are logged with more details, and are also produced on stdout with lest explicit details.
One can provide his own logDir/logFile, by default log files are “testbed.log” produced in game-data directory.

Next week I plan to take on sound subsystems. This is probably the last subsystem to be tested.
I want to do all this quick so that I could get some time to modify things as mentors want them to ;).

—
Neeraj

Hi everyone,
Once again its time for some update on my work. Here is a brief summary :

• Being done with the Save games test, I moved on to testing miscellaneous features, which include Mutexes,timers, Date time and events.
• Testing datetime was straightforward, get the date calculated from the backend, verify that it increases fine by putting delays, and if the session is interactive, display the date and verify directly with user.
• Testing mutexes was a bit complicated, It is not always possible to generate a situation in which an incorrect mutex implementation fails, However, I came up something like this:
  Took to shared variables: first and second.
  <Critical Section>
  first ++;
  delay(2)
  second *= first
  </Critical Section>Here, the output depends upon whether the mutex behaves correctly or not, additionally based upon my observations I had also added couple of checks which minimize the situations in which an incorrect implementation would get through.
• Testing timers was a bit easy too, I passed a static var to the timed process, and verified whether the callback function works fine by checking the value of that variable.
• In the meantime, I did some work on improving the code as suggested by eugene and jordi, there are no printfs anymore in the code. I have also added the code so that data gets written to the log whenever we want, If there is no logfile, outputs are put on stdout.
• One more thing I added was, that the engine putting up a list of testsuites to execute, this is now handled in a better manner than previous.
• Finally I moved on to events, Mouse events were easy to implement, but I was almost stuck with the RTL event (ctrl+f5) for almost a day or two, what happened was whenever I press (ctrl+f5), the code crashed with message “could not load widget GlobalMenu.help”.I thought I found a bug :), but after hours of debugging and examination of the code in GUI and themes, I finally figured out that the newer scummvm binary requires an updated theme_layout.stx, while it was using the previous one from “/usr/share/scummvm” and hence the problem, When I changed my theme path to “scummvm/gui/themes”, all worked fine :-). Finally I did “make install” to update the copy in “/usr/share/scummvm/”.
• For keyboard events, what I tried to do was, make user type some word input, we would display the word on screen, and make the user verify them. I have committed some code on it yesterday but needs some more rework.
• Fianlly the events RTL and QUIT would be handled, they will enable the testbed to exit even if all tests are not complete.

So that was all from the week, This week I have planned to :

1. Done with all the event stuff (Sunday).
2. Refactoring as suggested by Eugene (Monday-Saturday)
   — Rainbow palettes (Monday)
   — Possibility to go forth and back in tests/ skip some tests. (Tuesday-Wednesday)
   — Show test number and progress bar (Thursday – Friday)
   — Improve the scaling test and detections (Saturday)

The only tests that remain now are of audio/audio cd/ Midi subsystems. Once the refactoring stuff is done, I shall be ready to step into those. Additionally If the mentors have any input or feature (they might want to see), please drop me a mail and I would be glad to include that in.

Cheers,

Hi everyone,

Its sunday today and time for some update regarding how am I moving on.

Last week I worked upon some loopholes in the GFX tests in the first half of the week, implemented some FS stuff in the middle
and finally ended up with completing the savegame tests.

Crash while scaling with 3x scalers and odd dimensioned cursors.

while working on the scaled cursors, I noticed a weird behavior, My code happened to crash always with 3x scalers.
I tried some more permutations of cursor dimensions, and with hours of efforts with valgrind and gdb, I was finally able to establish  that the behavior is due to somewhere inside the scaling code.(and not from my code). Thanks Jordi for verifying it for me.
Finally, we found that there is some assumption in scaling code which relies on cursor dimensions being even. I am tabulating here the results of other permutations i tried with.

For square cursors:

• Odd  x Odd crashes
• Even x Even Fine
• Even x Odd Crashes
  
• Odd x Even Works fine

same behavior for non-square cursors.

The overlay rectangle test was corrected as well, thanks to jordi again, which points to some update on documenting how showOverlay() should be called.
If called after writing some data to overlay, copyRectToOverlay() in my case, there is nothing visible on the overlay, however, moving the showOverlay() ahead does all the charm :D.

Ideally, I expected it to be like this, “write something to overlay and then make it visible”, unless it is documented to behave otherwise.

Next, two new tests were added, Palette rotation and Testing pixel formats. As pointed out in my previous post, in palette rotation, we try to demonstrate how user can change the data displayed on screen without having to paint anything. This should be possible in palette graphics only.

In pixel tests, what I did was iterate over all PixelFormats, displayed some colored rectangles for each, and verify if they are rendered correctly.

Next returning to FS tests, a cache insensitive file read/write/navigation tests were implemented as suggested by Eugene.
However, the system is still not able to open files with names “file.” (with a trailing dot) as “file”.

In savegames, each and every function defined in common/savefile.h was tested.
However, I noticed that the error handling functions are not working as expected.

What I tried:

• Cleared the error state in savefile,
• Opened a non-existent file,
• re-examined the error state, which happened too be Common::kNoError, clearly not expected (as it appears to me).

As I am done with all these, I have moved upon the Misc tests datetime/mutexes/timers, the details of which are topic of my next post.

Furthermore, today I got some feedback from Eugene too on my efforts so far, which was encouraging :), He had some inputs as well, which point to even more work ahead! :D.

—

Neeraj

Hi everyone,

Last week I was working on GFX subsytem tests. Currently we have following tests in working condition with some details about each of them:

1. Blitting Bitmap on screen: This is the most basic GFX test, here we display a colored rectangle on the screen using game palettes. If this works, most GFX drawing will.
2. GFX Transactions:
   • Full Screen Mode : Here we toggle between the fullscreen and windowed states, ask user which state is it and based upon the response the engine decide if it works or not.
   • Aspect Ratio: As described in the last post
   • Iconfiying Window: As described in the last test
3. Cursor Tests:
   • Palettized Cursors: Use game palette and cursor palette to render the cursor on screen.
   • Scaled Cursors: For testing scaled cursors, what I do now is set different graphic mode scale factors and vary the cursorTargetScale parameter, the cursor should behave accordingly. (However, this currently needs some rework)
4. Effects
   • Shaking feature : displays a word on screen and makes it shake.
   • Setting and hiding focus: I tried with two rectangles on the corner and toggle focus between the two. However, this optional feature might not be implemented on every backend, or in other words they simply do nothing.
5. Overlay Graphics
   Here, we use overlay graphics to display some graphics on screen. However, this test needs some fix as in currently I am not able to notice any change despite of my few attempts. :(.
6. Palette Rotation
   Palette graphics has one side effect, use can change the display of screen by just changing the palette and without have to draw any pixel. In this test, what I want to do is take some colors, draw some rectangles on the screen, Now rotate the colors in the palette and call updateScreen(), a rotation in colors of the rectangle should be noticed. This test has not been implemented yet, but i am thinking of merging it with the blitting bitmap test where we display rectangle and palettes. Any suggestions are most welcome.
7. Testing pixel formats.
   We were working with 8bpp (8 bits per pixel) format so far, clearly the 16 + bit pixel  formats should support more colors, the intent here should be to display some “nice gradients”, as Eugene pointed about in his email to me. I assume displaying some colored graphics is what he meant ;-), once again some suggestions about testing it are most welcome.

So that completes the GFX test description, but the work is not complete yet. Some more work is needed with scaled cursors, overlay graphics. Testing features in points 6 and 7 are yet to be implemented as well.

As i was a little bit stuck with the overlay stuff not working and all, I tried my hands on File System tests, just for a change ;-). I hope I would finish up the graphics thing once i return back to them this weekend.

Meanwhile, In the FS tests, I have done/working on the following:

FS tests in addition require some additional infrastructure. Now you would need game-data for the “TestBed” game engine.
Unlike other games TestBed game data is not that fancy ;-), its just a collection of 3 files: two directories and one file. the hierarchy is something like:

testbed –> TESTBED (text file, used for game detection)
|———> testbed  (directory 1)
|                       |—–> testbed.conf
|———> TeStBeD (directory 2)
|—–> testbed.conf

The testbed/ and TeStBeD/ directories contain the same file “testbed.conf” with different data written on them.  We use this case difference to open the appropriate file.

1. Test Reading File :
   Here, we use the API defined in common/fs.h to access the game root node and open the file “TeStBeD/testbed.conf”, read the information in it and compare it with what it expects. If it founds them same, it marks the test as correct.
   However, this test is not flexible as in if someone tampers with the file “testbed.conf”, the test would fail.
   Any workaround suggestion around this would be much appreciated :).
2. Test Writing File
   Here I try opening a file using the FS API, in the game root directory, call that “testbed.out”, write some information on it, open that file again for reading, compare the data read to the data written, if they are same, the test works good. In case of any error, the test would fail. So one would need to make sure if the directory is writable or not , unless the test would display wrong results. Any suggestion on this would be much appreciated as well :).
3. Test Saving game states
   Here what I am trying to do, is preserve the state of testbed engine, if this test is executed write that state to file and when the test is restarted, it should resume from the state it was left to.
   To keep things simple, I am thinking of something like this, We use an array to store which testcase of a testuite are executed and which are running, what i would do is, store the data from the array “_testsToExecute” for a dummy testsuite in the savefile and restart that dummy testsuite, the engine should be able to regenerate the data structure “_testsToExecute” from the savefile state.
   Any suggestions on these will be, as well much appreciated :).

So, thats all for this post. I have asked some questions here as well, assuming that the intended viewers would mostly be  the ScummVM team ;).

Cheers,
—
Neeraj

Basically, our objective is to make sure that the interface to the graphics provided by the backend works as expected. However as most of these can only be verified by a manual interaction, these will mostly be interactive.

The following list summarizes the way the testcases are going to be implemented:

• Testing a specific feature : for example, fullScreen, iconifying window will just require enabling that feature and updating the screen and collect if it works or not from the user. There is not much to it.
• Testing Aspect ratio correction: Like the previous case they can be just invoked and noticed if it works but additionally we need to make sure that it does correctly the job we intended. So for this case, we expect the screen to resize to 320×240 as the game data is meant for that resolution, for other resolutions the data should appear compressed or stretched. To simulate this I have drawn an ellipse with minor/major arc ratio as 200/240. So that when the backend does aspect ratio correction, it necessarily stretches the minor axis to 240 and the ellipse should appear as a circle.
• Testing palettized cursors: For testing palettized cursors, I have drawn my own cursor which is basically a square with a black dot that signifies the hotspot of the cursor. The user should be able to see it, move it and when clicked the test terminates. In the background I would also make a check if the palette are the same as specified.
• Testing blitting a bitmap on screen: This is the most basic test of the GFX testsuite which makes sure if the backend is able to illuminate the screen. If this works probably the other code which draw things on screen should also work. This is implemented by setting the buffer to a color and then putting it up on the screen.
• Testing scaled cursors : Scaled cursors basically act as a hint to backend that when or not to scale the cursor.
  Normally, the data presented on the screen is scaled by some factor (1x, 2x or 3x). So if the cursorTargetScale parameter is set to 1, the cursor should scale as the normal data would, however if it is set to 2 and the screen is scaled y 2x, the cursor should not be scaled.
  To test this feature we would iterate over all the graphics modes making combinations of cursorTargetScale and scaling factor, the cursor movements should ideally work as normal, if they don’t there might be a bug in the backend.

Thats all for now, i would update more once i finalize all this ;).

Hi everyone!

The 2nd week of the program is about to end as well. so far I have made some progress with the code. I am currently working on the GFX Subsystem.
Basically, before starting to add tests to the framework we needed the following infrastructure:

1. A basic Testsuite class encapsulating all the generic non-subsystem specific methods like generating reports, executing tasks etc.
2. A method to transparently add tests to the testsuite
3. As almost all the tests in GFX would be interactive, a means to interact with the tester was required.

My work last week was focused on accomplishing these goals which at a greater extent are achieved.

To interact with the tester, I am currently using GUI::MsgDialogs. Another option I thought of was to use the graphics screen for interaction but later i figured out that writing code for that would consume some time and may delay the primary goal.
Currently these work fine for interaction, could later be changed to a different implementation.

As much of the infrastructure required is ready, I would now start adding tests for the GFX subsystem and  I expect it to be done by next week.

The project implementation design and architecture can be found on this document.

—
Neeraj