Month: July 2023

Categories
Coding Experience Fun Nerdy Talk Technical

Creating sandwiches in Total Distortion!

Greetings everybody, in this blog I will be going through my work for this week! So first of all like everytime let’s start with title!

Creating Sandwiches in TD!

Well, amazed? Just let me explain what I meant, so in this game of total distortion-win, there is this minigame/feature to create your own food for exploration. Why?

Because the story follows a guitarist who teleported himself to a faraway dimension called “Distortion Dimension”, where he must collect unique sounds and ideas to create his next blockbuster! (And thus make a return trip to earth)! In all of these ventures, He needs to have ample health (including mental and physical prowess), and that is where food comes in! The food in total distortion is created using something called “Food Goo”, and sandwiches are just one of the many things you can craft!

That’s it for background, now when I was playing and testing game, there was this bug where I could not change ingredients while creating food/drinks, and that left me quite unsatisfied (and hungry too, getting to eat only one type!) It has been documented here – https://trello.com/c/CexIGcq8/629-food-maker-up-down-item-change-keys-not-working

So with a sense of responsibility, I took it upon myself to fix this and with some tracing, debugging.. It has been found that the problem lied to this statement:

do("set sProtein2 = the number of items in sProteinDB")

Here, the up/down keys were not working because of the anonymous script that is being executed inside (ie set sProtein…), Why?

Because in traces it has been found that

varFetch: variable sProteinDB not found

and so, the whole logic short-circuited and the change in items were not being registered! Well after debugging it has been found to be a problem with how Lingo anonymous function execution works inside handler of factory!

The part do(…), is anonymous calling of script, because it dynamically compiles and execute whatever script or command you feed to it at runtime (just like Python’s eval function), and just like that when the statement was being executed, It didn’t have any idea about sProteinDB which is instance variable (or object variable) for factory! Now since the command is executed in a different Context (or background) therefore It didn’t have any knowledge of object variables and thus that error!

The Fix? Well it was easy, I added a simple check that allowed me to share underlying factory context when the call is made to an anonymous function belonging to factory/script! This allowed the function to view/modify factory variables and thus fixed Sandwiches in total distortion!

Now that’s what you will call a YUMMY change!

(Please ignore the sound when sandwiches are created.. kinda gross)

PR Link – #5191

Making Buildbot Happy

This week, I also dedicated some time in looking at director build bot leaks. If you don’t have much idea, I strongly advise you to read the section “Leaks are annoying (and dangerous)” from my previous blog, At that time, I broke things due to not checking for leaks which made Buildbot quite unhappy and complaining however this time, I tried looking into some common existing memory leaks and fixing them! While doing that I came across one where an “Archive” (ie a structure to manage resources used by game) was leaking when changing Shared Cast. Well don’t sweat up technical terms but after like 4-5 hrs of debugging through code and frustrating nights, I finally found the problem and fixed it!

Results? A good spam and buildbot happy!

buildbot reports on discord, green means what has been fixed since last build
me and sev agreeing on satisfaction of the green spams

PR Link – #5190

More work done this week

Rebuild frames when jumping to the current frame

As on-demand frames loading has been finished and merged, new bugs and regressions keep popping up, just like how in Majestic-win there was the freezing when opening game, I investigated this and fixed it up with simple change.

Link to PR – #5176

Fixing black patches in Mediaband

and there was this regression, where mediaband had black patches all around the screen, again after tracing and debugging where could be the problem, It was fixed with PR #5178

Improve screenshots comparison for buildbot

This one is an upcoming feature for our buildbot, as you know currently we don’t have any system to compare screenshots of game, we only run them and see debug logs for any problems, However in some time we will be implementing frame-by-frame screenshot comparison, so buildbot can flag changes that potentially can break graphics, palettes, rendering etc! (Just how cool is that!)

As part to support that transition, I improved the algorithm for screenshot comparison, initially MD5 of image bytes were compared but since this method wasn’t foolproof, I implemented logic to simple compare each pixels of two screenshots and check if they are within threshold or not! PR with change – #5196

…and that’s it for this week! Before ending up

I have something to tell you all, my university will be opening tomorrow, and I really hope that it will not affect my pace of working, or fixing bugs. But as much as hopeful I want to be, there’s no telling about delays, or slowing down in the future! So I hope everyone keeps up with it and keep looking forward to next blog post!

The End

Thank you for reading up, If you are still here with me, I’d like to express my sincere gratitude for reading this up, the support of fellow readers is one of the thing that keep me up and inspire me write creative/fun blogs. I hope it was a good read and like always do forgive me for any bad writing/grammatical mistakes, etc. With that being said, Adios!

Categories
Coding Experience Fun Technical

It’s not you, It’s me!

Greetings, everybody. Welcome to this week’s blog post! I hope you are all doing fine, so first of all let’s start by this blog’s name, why “It’s not you, It’s me”, Well there’s the reason behind it as one of the bugs that I encountered has to do something related to this term ‘me‘ and thus when referring to this problem, rvanlaar (discord) used this as a joke… more ahead..

Let’s start with good news

Frame Loading Revamped

As presented in so many previous blogs, we are finally reaching the completion of demand frame loading and successfully working on total distortion’s bed dream games, this single big change took so many weeks, and after much effort frame loading has been completely revamped and the master branch is now updated to use on-demand frame load. If you do no know what I am talking about, read this and this to get a basic idea about the change!

PR #5124 by super mentor Sev laid the groundwork for frame loading by merging some 40 commits into master and thus the era of demand loading came! However, some more tweaking had to be done to get everything working properly and most importantly, getting total distortion bed dream games working, for which all this was started!

So after I made more changes and fine-tuned things after which bed dream games started to function normally.

Preview:

PR Link – #5163

It’s not you, It’s ME

Finally, to the fun part, bed dream games were fixed, so what next? To go next part and play the minigames (the bulb one you see in the above video, at the end).. but while testing it, I encountered another bug documented here, that is:

################ LINGO: syntax error, unexpected tINT at line 1 col 13 in NoneScript id: 0!

What does this mean?

To explain it better, I would have to give you a brief idea of what lingo is, and how is it used! But first, you need to know that the Director engine(for which I fix bugs) is a multimedia authoring platform, oh btw

Multimedia authoring is a process of assembling different types of media contents like text, audio, image, animations and video as a single stream of information with the help of various software tools available in the market.

ie such a platform needs different tools and things to control/create movies, and games.. Similarly in director engine, there is Lingo, a programming language that can be used to control various elements of games, like sound, images, etc!

Quoting from this Wikipedia page about lingo

Lingo is a verbose object-oriented (OO) scripting language developed by John H. Thompson for use in Adobe Director (formerly Macromedia Director). Lingo is used to develop desktop application software, interactive kiosks, CD-ROMs and Adobe Shockwave content.[1][2]

So it is a language that was originally used with director engines, now to replicate that engine in scummvm, scummvm must support reading/parsing this custom language right?

How is a programming language understood by the computer? or more specifically, how is lingo being read by scummvm? Well to understand so, we have to deep-dive into compiler designs and things even I am not comfortable with, but in a nutshell there are these components:

  1. A compiler, which takes for example Lingo and converts it to C++ comfortable instructions.
  2. AST Trees for operands, operations, etc.

Oh, Lingo is a context-free language, I would really love to go deep inside automata theory but that’s an article for some other time!

But what happens, in a nutshell, is a simple instruction like put “Hello” when read by our compiler/parser, jumps into state, ie, converts everything to the token and then token-by-token creates logic/relations etc about different components of instructions, for example here let’s say our parser was initially in state X where it can accept a,b,c,d…. tokens, “put” being one of them, once it received “put” token, it simultaneously jumps into state Y where now it can only accept token like " , etc.. well, all these are then converted to C++ objects which will later be used to execute lingo!

What did go wrong with this bug?

Well, as seen by the error, called syntax error, what happened was that the game was trying to execute an instruction that is little different than what scummvm expects, here in this case, the game tried to execute lingo instruction

me(bubdie2 5, 75)

however, we expected an instruction of this type to be written something like this:

me(bubdie2, 5, 75)

(Please note the comma there), until now, there were no cases observed when a game used the wrong syntax (well correct, to be precise, just non-documented syntax), which was working fine in the original engine, and once again, we didn’t implement this peculiarity!

To be extra sure that it’s “us” not “them” who did this wrong, I verified this by creating testing movies for the director engine and trying to work this off and as expected our Lingo parsing was incomplete as it didn’t account for this edge case! Well time to fix it (but with a twist)

As this was too difficult for me, again super mentor Sev came to the rescue, he told me the problem and after discussions, djsrv implemented this, Link to PR: #5170

Here’s a working screenshot:

screenshot showing working dreamlights minigame

Oh, by the way, this title was inspired by this message of rvanlaar

You can always join our discord using this link and watch progress, and even better contribute and help improve ScummVM!

More Work this Week:

Fixing leaks in lingo-tests: #5164

This fixes leaks, and errors in lingo-tests which makes the build bot green yet again!

The End

Thanks for reading so far, I hope you like it, as usual apologies for any grammar mistakes, etc! With that being said see you in next week!

Categories
Coding Experience Fun Nerdy Talk Technical

Regressions: I make code, then I break code

Greetings! First of all, I would like to apologize for the long break. I was on leave and thus could not find time to be active here! So it’s time to be twice as hardworking to finish everything that lagged behind! (And create quality blogs as well XD)!

So this post is about regressions, what are they, and how do they work? Well…

A software regression is a type of software bug where a feature that has worked before stops working. This may happen after changes are applied to the software’s source code, including the addition of new features and bug fixes.
Again, google to the rescue! But anyway, so you get the general idea! It is about something that used to work (but not now)! So what happens is that in a big project with many moving parts, It is almost impossible to keep track of what new code changes can break other features. Even a seasoned programmer can make mistakes of accidentally breaking some other parts of code while implementing something new. (or perhaps fixing bugs) 
Although checks are in place, which help avoid frequent regressions like Buildbot tests ScummVM uses where each new addition is checked against several test cases already prepared, and their output is compared to expected ones! This is all automated, so much testing is done without much effort (except for setting it up XD).
So it’s been so many weeks already, and as we speak, I am on my last day for mid-evaluation. One question you might ask is:
“Hey HSK, working for so long, didn’t you break things? ie Regressed something?? I see all the good codes and how everything is magically working but where’s the dark part??”
The answer is

Regressions Introduced:

Popup Menu leaks

Linked to #5005 where I first implemented the popup menu, there are some occasional memory leaks(if you have no idea what this is, read the section “Leaks are dangerous” from my previous blog), which I have yet to find concrete ways to reproduce, however with some time I am sure it will be fixed.

Symbol and String Comparison

Related to #5107 where I fixed a comparison between symbol and string and evidently a lot of bugs in total distortion, Well.. I was wrong, Symbol and String are never equal on basic level, just sometimes when used to retrieve items from list using those as keys, do they behave like same! Fixed in #5158 where I moved the code to appropriate location and get everything working.

BDFScaler Scaling

Related to #5125 where I improved the text font scaling algorithm which was initially implemented by another student, however even though I managed to fix many bugs but there were still some more left which were very tricky and we had to disable that feature again in-favour of better GUI.

WinFonts Scaling Memory Leaks

Related to #5126. where I initially implemented Windows font scaling, well some leaks were not being accounted for (ie, scaling fonts), fixed in #5136

Well, even as we speak, and for the sake of open pull requests, We are never sure when another regression is introduced, but what I can do is make sure to fix the regressions and thus help in improving the stability of the director engine! Let’s look at some of the ways to fix/identify regressions:

Git bisect to save us here!

Read more about the git bisect command from here, what it does is to use Binary searching to systematically checkout commits and ask you if the commit is “good” or “bad”, depending upon your answers it will point out where things broke!

It is such a simple and powerful feature that you can traverse thousands of commits in 10 iterations! (for nerds that will be log2(1000) or 9.965 iterations, don’t ask me how)

It is so good that I once wrote a tweet about it:

that is it for the regressions and the dark part.. let’s see what I fixed for these weeks and my work so far!

This week

Use “me” to set “the perFrameHook”

Implemented in #5152, this allows for expressions like

set the perFrameHook to me

total-distortion was using this to assign hooks to call itself perframe using this expression, this is a special case where “me” is used to point current object instance to set “the perFrameHook” property!

Fix the working of m_perform properly

m_perform is a lingo handler to execute an object’s method, much similar to “do(command)”, there were errors when this was used inside mAtFrame, but after proper debugging and discussions it has been fixed by a change in #5153, however the change was little unorthodox so a test is also created in-case it breaks in future.

Fix the execution sequence of lingo scripts in case of frame jumps!

As fixed in #5156 where lingo executions inside the handler were being called immediately after frame jumps, ie (go commands) without waiting for the frame to load. Fixed this by freezing the execution of lingo and waiting before the frame is properly loaded before executing any remaining window!

 

The End

Thank you for reading so far, I would love to hear your thoughts and any feedback/suggestions, as usual apology for any grammatical mistakes and writing errors and especially the long delay for this blog, anyway with that being said, see you next week!

Categories
Coding Experience Fun

The Art of Scaling Fonts

Hey! Welcome to yet another blog article showing my progress this week, before starting let me remind that for this long week I was on vacation and therefore the progress is very slow, but regardless here’s what’s been done since last blog!

The Art of Scaling Fonts!

If you remember, my last blog where Fonts text height were fixed and text was aligned properly, this week I tried implementing scaling of Fonts (ie character scale) for windows font and fixing bugs in already scaling code for BDFonts!

Let’s start with terminologies and keep it aside!

What is BDF?

Wikipedia says:

The Glyph Bitmap Distribution Format (BDF) by Adobe is a file format for storing bitmap fonts. The content takes the form of a text file intended to be human- and computer-readable. BDF is typically used in Unix X Window environments.

https://en.m.wikipedia.org/wiki/Glyph_Bitmap_Distribution_Format

What is Glyph?

A glyph is a single representation of a character.

In these bitmap fonts, what happens is that a single character (or glyph) is represented by sequence of bytes, now let’s take an example of character A

It is already understood that an image is combination of dots (ie black and whites), what we see as a 2D figure is actually a matrix with filled 0s and 1s, similarly the pictorial version of these characters (ie how A looks on scteen) is given by a matrix that will have 1 in places where a black spot appears.. (when seen as image this matrix will be seen as how we understand character A should look like :)!

How this matrix information is packed into text?

Let’s assume we have this character A which is 12 bits in width and 16 bits in height,

Now if you understand basic data types or logic gates, you can already deduce that it takes 1 byte to pack 8 bits, or here in this case 2 bytes to pack 16 bits

Also since height of this glyph is 16 bits, therefore it takes 16 x 2 or 32 bytes to represent full character matrix!

 

That is all in the basics of Fonts, if you understood it all, unfortunately I still cannot award you the prestigious certificate of completion in basic font terminologies as the real deal has lot more technical jargon like offsets, ascent, descent etc etc, but for our case only understand these bits will do!

So what is font scaling?

It is, as name sounds scaling each character to some size and thus using it, a good scaling is one in which the texts using font is still readable (and crisp), however in lot of cases it is difficult to moderate how good we scale.. Here at scummvm currently we implement a primitive scaler that works like:

1. Take each of the size parameters ie width, height etc and multiply it with scale (ie newsize/oldsize) round it off and that will be our new size!

2. For glphy scaling (ie the actual image part), what we do is to draw the character on empty surface and scale the surface upwards, finally we trace the scaled screen and convert it into bytes form!

3. This new object (ie scaled font) is returned to be used as per demand!

Fixing bugs in BDF Font Scaling!

Now bdf font scaler was implemented by a student themselves, and it was kind of not finished or let’s say not furnished and was giving crashes and overflows! So as usual I started looking into it and after some time found the source of error! Finally fixing it I was able to run the program with scaledFont..

But it doesn’t end here:

While debugging the already written code, I found lot of issues with gui, like different text preview with different size, artifacts in certain glphys and many other..

After spending some time finding and fixing, bdf scaler was fixed upto certain extent (however would like to mention that not still some minor issues persisted, which are still being looked through)

Link to accepted PR: #5125

Implementing Windows Font Scaling

Related to scaling of Fonts, I was asked to implement windows font scaler, these are in formats of FOND and FNT, etc with their own specifications etc, as usual I read up some old documentations and archives information about fonts and went to implement it just like how BDF Scaler was implemented! A day later it was merged and yay, there we have our windows font scaling implemented, now windows games using fonts of different size will be rendered!

Link to merged PR: #5126

The End

Thank you for reading so far, this week blog, I apologize for the delay for this weeks progress and as mentioned I will be fully active and working from next week so look forward for the next release!