Week 11

Welcome to this week’s blog. This week, I worked on testing the workflow for scan data as well as the user file integrity service.

For the scan data, we tested with the WAGE game archives, which provided a good opportunity to test both the scan utility and the scan matching for the Mac files. Some fixes were indeed needed for the matching process. Initially, I was using both size (data fork size) and size-rd (resource fork’s data section size) simultaneously while filtering filesets. However, this was incorrect, since detection filesets only contain one of these at a time. Additionally, I fixed how matched entries were being processed. Previously, entries matched with detection were placed for manual merge to add specific files while avoiding unnecessary ones like license or readme files from commercial games. However, it made more sense to merge them automatically and later remove such files if necessary—especially since, for the archives like WAGE, the issue of extra files from commercial games would not occur.

I also carried out testing for the user integrity service, focusing on different response cases:

1. All files are okay when a full fileset matches.

2. Extra files are present.

3. Some files are missing.

Another missing piece was reporting files due to checksum mismatches, which previously was being classified under extra files. This is now fixed. I also reviewed the manual merge process for user filesets. Unlike set filesets, the source fileset (user fileset here) should not be deleted after a manual merge, since it could be a possible new variant which would need additional metadata information. To support this, I implemented a feature to update fileset metadata—though it still requires some refinement. An additional thing that I need to add is to create an endpoint in the web server that can be triggered by the mail server. This endpoint will provide the mail information, particularly the user fileset ID, for which the user has provided some additional information via the pre-drafted email that is promted when user uses the ‘check integrity’ feature in the ScummVM application.

A few other fixes this week included:

• Deleting multiple files from a fileset through dashboard: Previously, the query was being generated incorrectly. Instead of ‘DELETE FROM file WHERE id IN (‘1’, ‘2’, ‘3’)’ it was generating ‘DELETE FROM file WHERE id IN (‘1, 2, 3′)’ which, of course, did not work. This issue is now fixed.

• Search filter issue: A bug occurred when a single quote (‘) was used as a value in search filters, breaking the query due to missing escaping for the quote. This has also been fixed.