Month: August 2023

Google Summer of Code is coming to an end. Thanks to my mentors, it has been a pleasant experience.

Here are the things I have worked on during this Summer:

Universal Packaging Script

Code: https://github.com/scummvm/scummvm/pull/5300

Tasks Done

1. Identify metadata requirement for DLC download support for various distribution stores. It has been compiled here.
2. Create schema for metadata and JSON files like dlc-games.json and export-platforms.json to maintain all required metadatas for games and how to process those metadatas for different distribution stores.
3. Write a python script that takes game location, platform type and the ScummVM binary location as parameter. It will determine the output based on the export-platforms.json and dlc-games.json. E.g. For Android Play Store, we will get a new AAB file bundled with games.
4. Add entry for Android Play Store in export-platforms.json and all required dependencies (bundletool.jar, python scripts, etc.).

Possible Improvements

1. It takes a lot of time to package games one by one. Create an option to mass package games from a folder.
2. There are some dependencies that are required to be installed on the host computer. E.g. specific version of Android SDK, Java, Python, etc. Utilize container based approach for quick setup.

In-app DLC Downloader

Code: https://github.com/scummvm/scummvm/pull/5134

Tasks Done

1. Create GUI dialogs and widgets to interact with the DLC Downloader and to show download progress, in-progress DLCs, etc.
2. Create DLC Manager to manage downloading DLCs.
3. Create an abstract class called Store. Different distribution store will need to implement this class using their store-specific API/SDK.
4. Create a new path called dlcspath for storing the downloaded DLCs.
5. Create a new compile time flag, USE_DLC for including DLC downloading support. This is disabled by default. To enable it, use --enable-dlc with ./configure. It will only be included on backends where it is supported (kFeatureDLC is true).

Possible Improvements

1. GUI dialog for DLC Downloader could see some visual overhaul like list with game icons.
2. Generalize DLC to allow other types of data like themes, icons, shaders, etc. Currently, only games is supported.

ScummVM Cloud Integration for DLC Downloader

Code: https://github.com/scummvm/scummvm/pull/5134

Tasks Done

1. Implement ScummVM Cloud to download games from a URL (ScummVM’s server) by utilizing existing networking methods.
2. Handle zip extraction and deletion of zip after extraction.
3. Improve existing method Archive::dumpArchive().

Android Play Store Integration for DLC Downloader

Code: https://github.com/ankushdutt/scummvm/commit/c313a43cf1415598144c816474eb38ddbe56a8b1

Tasks Done

1. Implement Play Store DLC support by using Play Core SDK for Java.
2. Create JNI methods in C++ to interact with Java code for Play Store or vice-versa. Java code utilizes the SDK to implement functions necessary for handling download.

Possible Improvements

1. Create script to generate the list of assets with required details like – id, name and detection keys for ScummVM configuration like engineid, gameid, extra, guioptions, language, platform and description directly by utilizing the dlc-games.json.

Documentation

Link: https://wiki.scummvm.org/index.php?title=Game_Packaging

The implementation of Play Store is mostly done. The downloading works along with the download progress and cancel functions. These functions need to be implemented on the java side since we use the Play Asset SDK for Java. I also needed to call these functions from C++ and vice versa. So, for this, JNI comes into play. The current code for Android also uses JNI extensively for some of its functions. So, the base work was already done. I had to check the current implementation of one of the JNI functions and follow the steps used. The other thing I needed to do was create simple functions that handle Playstore SDK on the Java side.

In Playstore SDK, there is an interface called AssetPackManager that contains the functions used in managing the downloads for the DLCs from Playstore. For starting a download, we can use the fetch() method. It only needs a packname, which is a unique string to identify a game package. I also had to create a listener that keeps track of the download progress, completion, and cancel events to call required functions.

To support the download of the DLCs, Play Store requires us to bundle the base APK (ScummVM) together with all the on-demand assets in an AAB format. I had previously created a Python script to bundle the AAB (containing only base APK) with games. The workflow for creating the final ready-to-upload AAB should be: extract games somewhere, fill some file (JSON or something else), and run gradle to generate the full AAB. The current workflow is more involved. I will also simplify this process next week.

Now only one week is remaining. I will focus on completing the Play Store support and simplifying the process of the AAB build. And, of course, writing proper documentation.

This week I had my work reviewed by my mentors, and I mostly worked on resolving these.

This involved forward-declaration of several classes in the header files, which only refer to those classes as pointers. This will help reduce the compilation time since the compiler will not have to process additional headers.

Also, I have now introduced a new configure feature called USE_DLC along with USE_SCUMMVMDLC. This USE_DLC can be activated by --enable-dlc configure flag. However, if developers want to specifically use the ScummVM cloud (hosted on ScummVM’s server), they need to provide --enable-scummvmdlc with configure. The USE_DLC is a general flag, which can be used with any dlc-supported backend if needed. For example, for android play store release, developers will use --enable-dlc, and for android apk distribution on ScummVM website, they can omit it. Both --enable-dlc and --enable-scummvmdlc is disabled by default.

To determine if the DLC feature is supported in the current architecture (backend), we check if kFeatureDLC is set. We set it in the backend of supported platforms.

Another thing I worked on was error handling during the extraction process. I had to handle errors like archive is broken/incomplete and storage is full. For this, I have to make changes to the existing dumpArchive() function, which extracts the archive in the provided path. I had to make the return type of this function Common::Error and return the error. Once dumpArchive()‘s change is merged, I added the checks in ScummVM Cloud’s code so if there is any error, we show the error to the user, and also we set the downloading DLC’s state to kDownloadingError and not add the DLC to the scummvm configuration file.

I have also removed the incomplete Play Store code from my PR. So, it will add support for the ScummVM cloud only. I have started working on adding the Play Store support on a separate branch. The Android building (using Docker) on Mac with silicon chip was very slow. So, I had to shift to my old laptop running Ubuntu. There has been a bit of progress in adding Play Store, but I think it’s doable in a few days.

Now we are very close to completion of Google Summer of Code program. I plan to make a PR for the Playstore support and add documentation and HOW-TOs to the wiki next week.

This week I have finished working on the GUI for showing the progress of the download of a DLC. Other than the progress, all the queued downloads are also listed. There’s also a text widget to show an error if there is an error in extracting the zip (i.e., the archive is broken). Users can select a DLC and cancel the download by clicking on the Cancel button. Here’s a video for the GUI:

The cancel button works as follows – If the DLC is already in download progress, it will interrupt the download by closing the request (and remove any incomplete .zip). If the DLC is scheduled in the download queue, it will change the state of the DLC to kCancelled to get it skipped. I am checking if the DLC state is kInProgress before starting an async download request.

I have also worked on several bug fixes and improvements like disabling the download/cancel button when no DLC (list item) is selected, fixing the launcher not refreshing when the DLC browsing modal is closed, restricting adding downloads to the queue if it is already in the queue, etc.

Finally, as discussed with my mentors, I have wrapped this ScummVM cloud in configure enabled feature, which will be disabled by default. It can be enabled using the --enable-scummvmdlc flag.

What’s next? After cleaning up the code and preparing the PR for review, I will work on adding support for Play Store DLCs on a separate branch in the coming weeks.