Progress Report

Progress Report August 2022

MutantAura

• 11 min read
Progress Report August 2022

Another month rolls by the wayside and that means another progress report from your favorite Nintendo Switch emulator, nay, favorite piece of software ever developed. It’s hard work being this humble let us tell you.

To dispense with the pleasantries, what can you expect to read below? We’ve got the usual rolling improvements to our GPU, CPU, Kernel and Services emulation, a heck tonne of code-cleanup and finally a new LDN release.

Before all that let us talk to you about our sponsor. You guys!

Patreon Goals:

ARB Shaders - Goal reached in April 2021.
Work is ongoing, please wait a little while longer until we are able to deliver this update into a state we are happy with.
ARB shaders will further reduce stuttering on the first run by improving the shader compilation speed on NVIDIA GPUs using the OpenGL API.

$2000/month - Texture Packs / Replacement Capabilities - almost there!
This will facilitate the replacement of in-game graphics textures which enables custom texture enhancements, alternate controller button graphics, and more.

ETA once the goal is sustained: ~3-4 weeks

$2500/month - One full-time developer - Not yet met.
This amount of monthly donations will allow the project's founder, gdkchan, to work full-time on developing Ryujinx. All our contributors currently only work on the project in their spare time!

$5000/month - Additional full-time developer - Not yet met.
This amount of monthly donations will allow an additional Ryujinx team developer to work full-time on the project.

3….2….1…. GO!

GPU:

Switch games using the guest OpenGL driver are no strangers to appearing in progress reports, and this month is no exception. Titles such as Digimon Cyber Sleuth, River City Girls Zero and Layton’s Mystery Journey would exhibit small texture corruptions, usually in the form of lines or colored vertices where they shouldn’t be. This was ultimately caused due to the method Ryujinx used to flush texture data to the CPU being fairly inefficient and prone to bugs of its own. While fixing the underlying issue in the flush mechanism is important individually, the solution proposed to fix these OpenGL titles was to implement a new fast path for these texture data transfers, which bypasses the flush altogether.

After the merge of Vulkan, owners of AMD and Intel GPUs soon realized that the Mii Editor applet was somewhat busted when the API was selected. While it worked fine in OpenGL, attempting to open the Mii Editor in Vulkan would either cause an outright crash or exhibit graphical bugs such as some visual elements not being shown at all:

This particular problem was tracked down to the assignment of an invalid bool constant (1 byte), which was then assigned directly to an integer (4 bytes). This was causing a SPIR-V parse failure on certain GPU vendors, so the fix here was to remove these invalid assignments.

On the topic of Vulkan: the delivery didn’t come without teething pain and a couple of minor regressions. One of these regressions manifested as flickering in Animal Crossing: New Horizons and Atelier Ryza on both Vulkan and OpenGL graphics APIs.

Jump-scare alert

Both issues were traced down to an oversight during the final Vulkan rebase where, after the newly added sampler pool cache, texture bindings would use the wrong sampler pool. A single line fix was all that was needed here to pass the correct sampler pool as an argument.

Segueing off the word ‘textures’, there were a couple of more general housekeeping changes made this month with regard to defined formats and their associated format tables. ETC2 texture formats were added to Vulkan although, in an odd flip, it’s actually NVIDIA this time that doesn’t support their use. AMD and Intel owners should see Vegas Party, Radiation Island and any others that used said formats become playable.
The entire format table was also refactored to give meaningful names to the formats instead of using so-called ‘magic numbers’. A magic number in programming is a value that is used raw and has no meaningful context as to its use.

The numbers Mason!

Best practice would be to assign a constant to these magic numbers so that, in the future, contributors could understand what was meant. This isn’t always easy, especially in reverse engineering and emulation where you may not know exactly what something is until much later. Thankfully for us, our knowledge of the Switch has had years to mature; NVIDIA recently released a load of documentation too! Even for the many non-coders who read these reports, the difference in readability is night and day:

After

Some color formats such as RGBX have no alpha (transparency) and, as such, when certain operations that expect alpha are applied, these should ideally not touch color formats that do not have the required component. This unfortunately is not possible to do easily, but we can instead make them behave as if the alpha was always a static 1 (fully opaque). This resolves an issue in LA MULANA where the ground is rendered as a black rectangle instead of… well, not a black rectangle.

Xenoblade Chronicles 3 was an interesting release, to put it mildly. On the one hand, the game graphically seems to be holding up really well against hardware; on the other hand, stability has been hit or miss with not one, but three distinct crashes, all centered around the menus.
The first of these to be investigated and resolved was unique to OpenGL and expressed itself in the form of a TDR (Timeout Detection and Recovery).

OpenGL does not allow a vertex buffer’s size to be specified, which can indicate that a function may call an address from outside the bounds of the buffer on occasion. To fix this, a second temporary buffer is created to accommodate these out of bounds access scenarios. Ironically enough this issue is well known; the most obvious secondary fix is actually the waterfall in Super Mario Odyssey.

Users have noted for going on 4 years now that triangular artifacts would be visible in the water smoke effects, also visible in Captain Toad: Treasure Tracker. This is caused by the exact same issue that results in the TDR in XC3, and showcases why you should always put off one thing today that may fix two things tomorrow!

Vulkan can still exhibit a DeviceLoss crash during gameplay and in the menu of XC3, although we aren’t yet sure why, and the issue fixed here for OpenGL doesn’t impact NVIDIA Vulkan, as one would expect. There is also a final memory-related crash, independent of selected graphics API. Why they all chose to happen on the exact same menu, doing the exact same thing, and seemingly for completely different reasons, is beyond our understanding for the time being; we hope to have some more progress for you in the near future.

We aren’t done with XC3 yet though, as a second fix was pushed through this month, this time specifically for AMD users. Not content to let NVIDIA be the sole offender for format support shortcomings this month, AMD does not support the RGB16 vertex format on Vulkan and, as such, a fallback format was needed. Using the RGBA16 vertex format if RGB16 is not supported resolved the instant crashes these GPUs faced upon trying to boot the game. An HLE macro for render target clears was also added, as the AMD driver protests vehemently against clearing individual slices.

If you hoped we were escaping Xenoblade territory, think again. A recent change brought about a regression with resolution scaling in XC: Definitive Edition, where things looked a bit wrong. Some scale values were not being correctly updated across textures and images, resulting in the above issue. The game now scales properly again.

A couple of more minor changes to close out this month's GPU section include:

CPU:

The CPU recompiler continues to improve, and this month witnessed the addition of a bunch of new instructions with convoluted-sounding names that are becoming a bit of a blur.

More 32-bit Thumb instructions:

  • LDM/STM
  • LDAEX/STLEX
  • LDR/STR
    LDRD/STRD

Even more 32-bit instructions:

  • SHA256
  • MRS Rd

The second batch were most interesting as they were now, seemingly out of nowhere, required by the latest Mario Kart 8 Deluxe update: 2.1.0. As a direct Wii U port, Mario Kart 8 Deluxe is one of the few notable Switch games to run on a 32-bit instruction set, and so has tested us from the very start. With those new instructions in place, the game returns to its usual splendor.


The SHA256 instructions specifically received hardware-acceleration treatment and a check was added to ensure an instruction supports vex encoding; without this there was a possibility of an invalid allocation.

KERNEL/SERVICES:

While Ryujinx currently blocks connections to Nintendo online services, that doesn’t mean we don’t have to battle with a slew of network-oriented issues. Some applications react differently on boot if they “think” they’re connected to the internet: attempting to connect to servers, calling different services or just generally being annoying.

With this is mind, let’s glance at a couple of service implementations that were finalized this month:

  • ‘LoadNetworkServiceLicenseKindAsync’ was needed by Pokémon Legends Arceus when guest internet access was enabled. The game still gets a bit stuck loading, but one roadblock at a time, right?
  • ‘EndFreeCommunication’ prevents some titles such as Splatoon 3: Splatfest World Premiere from crashing just after the first splashscreen.

Not one but two oversights in the network sockets implementation were resolved, with the end result of Minecraft being bootable while guest internet is disabled. If this setting is enabled the game will still crash; if Minecraft is a title that for some reason interests you on Switch, keep this in mind.

gdkchan also leveraged August to extensively optimize how the kernel looks up blocks of memory, migrating from a linked list to a red-black tree. If you haven’t taken a computer science course then these won’t mean much to you, but rest assured that the new method is faster by a fair margin. For those of you that have taken a CS course: a linked list has O(n) complexity, whereas a red-black tree is O(log n), meaning that in the worst-case scenario the new method is dramatically more efficient at finding the desired memory block.

GUI/MISC:

Contributor CloneDeath this month took a mop to our codebase and decided it needed a spring cleaning in quite… a few…. different… areas, with other contributors joining the mix removing all sorts of unused strings, renaming functions and subtracting redundant code. Not the most glamorous of jobs, but with a project this big the basics are just as important.

So clean. So Tidy.

We’re getting more translations and locale updates for the WIP Avalonia GUI than we can shake a stick at, but we’re grateful for such wide community support in this regard! Polish and Japanese locales have been added with Japanese (yep again), Chinese, German and Turkish all receiving updates to include the latest strings. The list of translations itself is now sorted more intuitively, and each appears in their own native language.

We have a longstanding falsehood that it’s time to finally own up to. It will come as an enormous shock to most people that the old “purge PPTC cache” button… did not actually purge the PPTC cache. We know, unacceptable. Only on the 18th of August 2022 (a date which will live in infamy) was someone finally brave enough to tackle such a flagrant abuse of trust. Instead of purging PPTC, the function actually queued a rebuild on the next run. This grave misprint has since been updated in the UI to reflect the real behavior. An apology video will follow shortly which will include tears and a commitment to grow from this experience.

Onto some quality-of-life changes: your game directory list will now only refresh if settings relating to the list are altered. Think adding or deleting a directory, or updating your game to the latest version. Previously, even changing the graphics backend or increasing the resolution scale would cause a full reload: a death sentence for those with large libraries or making use of network storage.

So you have chosen... Death.

Cheating is a fundamental part of video gaming, so when some random cheats don’t work, it can feel like the universe itself is trying to set you on the right path. Luckily, we can just ignore the universe and resolve a small bug that was causing a few cheats not to work. Never forget your null terminators kids.

Closing us out we have a couple of cleanups from the July Vulkan merge. The voices of millions of Flatpak users all cried out at once and were swiftly silenced with a fix making sure some required packages were available on this build. Without it, even selecting Vulkan could cause a rather nasty crash.
For those unaware: Avalonia is actually a fully rendered framework and as such does make use of OpenGL or Vulkan to draw even itself. With the merge of Vulkan came the daunting task of not only making sure games rendered well, but also making sure it didn’t destroy all the work put into our GUI transition so far. Luckily, the fixes needed were minor and sorted out in a flash.

Closing words:

If you’re one of those people who frantically Ctrl-Fs every progress report for the words: LDN, Mac or Vulkan 2: Electric Boogaloo (just me?) then we have good news on the LDN front and just a canned laugh track on the other ones. If you haven’t already heard, we released LDN version 2.5 in August, which brings the LDN build up to date with master version 1.1.224. This means a lot more games are playable and, more importantly, every game is more playable for those poor souls stuck on AMD and Intel GPUs. As a result, we’ve seen many more folks enjoying Splatoon 2 and Mario Kart 8 Deluxe on LDN (both shader-heavy games). For the download and a more comprehensive list of what’s changed since 2.4, check out the blog post here: https://www.patreon.com/posts/ldn-2-5-vulkan-70757628

As per, well, always, we’d like to extend a thanks to everyone who supports the project with their time, knowledge, money or enthusiasm. We really wouldn’t still be here without them. As an open-source project we thrive off of community involvement and external contributions; if you’ve got some knowledge in computer graphics, low-level systems or anything up to UI and web design, there is a place for you. Emulators can seem unapproachable beasts but there are truly endless ways to dive in.

Hope to see you all next month!