Progress Report February 2022
February has gone and left us all too soon once again. Who thought that 28 days was enough to make it a worthwhile month?
But rest assured what the month lacked in days our development team more than paid back in the avalanche of improvements, fixes, additions and ongoing project work!
Patreon Goals:
Amiibo Emulation
Merged into the main build in March 2021.
While compatibility is close to being perfect, there are still some improvements to come for Amiibo which can be tracked on the associated Github issue here: https://github.com/Ryujinx/Ryujinx/issues/2122
Custom User Profiles
Merged into the main build in April 2021.
Vulkan GPU Backend - still in progress
A public test build is delivered and is available here.
ARB Shaders - Goal reached in April 2021.
Work is ongoing alongside Vulkan, please wait a little while longer until we are able to deliver this update into a state we are happy with.
ARB shaders will further reduce stuttering on the first run by improving the shader compilation speed on NVIDIA GPUs using the OpenGL API.
$2000/month - Texture Packs / Replacement Capabilities - almost there!
This will facilitate the replacement of in-game graphics textures which enables custom texture enhancements, alternate controller button graphics, and more.
ETA once the goal is sustained: ~3-4 weeks
$2500/month - One full-time developer - Almost there!
This amount of monthly donations will allow the project's founder, gdkchan, to work full-time on developing Ryujinx.
$5000/month - Additional full-time developer - Not yet met
This amount of monthly donations will allow an additional Ryujinx team developer to work full-time on the project.
Without further rambling from me, to quote a clichéd line, let’s just jump into it.
VULKAN PROGRESS:
So we’re back for another round and this month is a good one, if I may say so myself. Last month the SPIR-V backend had a bunch of new shader instructions added which started to get it into great shape for more general testing. However, while the shading language is a superb replacement for the painfully slow GLSL, it can get faster, and one way of doing that is multithreaded (parallel) shader compilation. This is a touch more complicated than on OpenGL, as with all things Vulkan, but riperiperi has taken the challenge upon himself and is already delivering some truly impressive results.
For titles that use many shaders simultaneously, parallel compile will have the largest impact especially compared to OpenGL using GLSL. This feature isn’t quite ready to be added directly to the main Vulkan branch due to some other bugs (thanks AMD) and it needing a general code clean-up, but we hope that user testing can start relatively soon!
Alright here’s another one:
GPU:
Improvements to the emulation of the Switch’s GPU are always the flashiest and most visual of changes and this bout of updates is no different.
Game builder garage had a recent regression and was showing all kinds of graphical issues that ranged from minor texture bugs:
To some quite major scene-changing problems:
Luckily our very own gdkchan was on the ball with a mere 2 lines of code to resolve these issues and make this a much more faithful experience to any budding game developer… any budding game developer playing via emulation anyway.
Moving onto some big hitters; Pokémon is a constant thorn in both my wallet and, until recently, my eyes. While the game was fully playable at launch it did not take long for users to start to notice some peculiar rendering in one of the games many caves:
This was an interesting problem as it was exclusive to OpenGL and so this eventually led to a whole rabbit hole of fixes that the Vulkan backend once boasted exclusivity over. gdkchan soon came to OpenGL’s rescue and fixed a whole host of issues; some of which have been long-standing indeed!
Fixes cave rendering in Pokémon Legends: Arceus.
Fixes outlines showing through geometry in Pokémon Sword/Shield.
Fixes black water in Paper Mario: The Origami King.
Fixes blue emblems on ships in Monster Hunter Rise.
Fixes over-bright jellyfish in NEO: The World Ends with You.
Unreal Engine games also got some much-need love this month with riperiperi implementing a new fast path for 2d engine copies (blit) which drastically reduces stuttering related to texture-streaming in some UE games such as A Hat in Time and Yoshi’s Crafted World while fixing the water in Fatal Frame: Maiden of Black water to boot!
Fixes the water in Fatal Frame: Maiden of Black Water
Since Ac_K implemented the Mii editor applet all the way back in June, Miitopia has been the game that lots of people wanted to take advantage of their own custom Mii in. Unfortunately while the majority of the game is functionally sound, there was a consistent crash that has been impeding progress since the game launched.
The fix by gdkchan has been in review stage purgatory since September but it is finally possible to get through this door! Who knew 3DS ports were so annoying.
Changing gear slightly, as many of you are aware, shaders are small programs that run on the GPU and are used to make ‘effects’ happen on-screen. Puffs of smoke, flashes, you get the picture. There are a couple of cases where Ryujinx can fail to compile a shader and one such case was addressed this month by gdkchan where either the Texture is sampled with a depth compare later in the pipeline or the Texture pool type doesn’t match the sampled type. While we aren’t aware of any games that were affected so far by this particular shader mishap, now we have peace of mind that we will never know of any!
Our lead developers aren’t just targeting those overpriced AAA 3D games that bring both your switch and PC alike to max fan speed; if you were an avid player of ‘River City Girls Zero’ then you were in luck this month as a fix for a crash after a cutscene was opened… and then swiftly closed. Luckily this was due to gdk doing some hardware tests and discovering that, while the fix above worked, it wasn’t how the hardware actually behaved. The cause of the crash was due to the game not writing to the X/Y region registers and so the first fix simply zeroed them out. It turned out the switch doesn’t do this but instead employs the rather disappointing workaround of simply ignoring those region registers! A second more accurate fix from gdkchan was therefore put forth and accepted.
AUDIO:
Since the merge of “Amadeus” Thog’s complete audio service rewrite a very long time ago in the fever dream that was the summer of 2020, Ryujinx has been relatively free of major audio bugs. But with 2022 comes new games, new firmware services and new problems to torment the developers.
The first problem child was with the hotly anticipated Nintendo Switch Sports Online Playtest (what a mouthful). Some of you may be wondering “what’s the point in emulating an ONLINE playtest on an emulator?”, to which we reply… Making sure it works day 1! Can’t lose our record that easily. Thog served an ace by swiftly allowing the game to output audio and then followed it up with a typo fix.
Skyward Sword HD on occasion can sound rather terrible especially on audio with high frequency effects. An adjustment to audren’s upsampler from nearest to cubic interpolation was swapped in by riperiperi and should improve the situation and act as a temporary stopgap until the sampling algorithm that the switch actually uses is reverse-engineered.
CPU:
Get a snack, turn up the mood-lighting and strap in for this because CPU improvements were hot in the street this February. Both gdkchan and this month’s MVP merryhime, the brain behind “dynarmic” (a popular dynamic recompiler for ARM written in C++), graced us with a whole host load of additions, fixes and optimizations which I will do my best to simplify.
Let’s first look at some of the new instructions ARMeilleure (Ryujinx’s dynamic recompiler) now supports:
- SHADD8
- SHSUB8 and UHSUB8
- CMN (RsReg)
- Thumb support (All T16 instructions)
- ALU instructions (T32 shifted register)
Even more are still in the review stage and while this probably doesn’t sound too exciting the more instructions the CPU recompiler can understand the less likely your favorite upcoming game will crash on you when you boot it up day 1!
Something pretty cool to note of the thumb support specifically is that they are crucial in helping run and develop the new PS Vita -> Switch compatibility layer, aptly named vita2hos by xerpi (check it out if you haven’t already heard about it!).
As with any sweeping changes, there's always a chance you break something else. Luckily merry squashed a bug that was preventing games booting, while also fixing a potential issue in the Thumb instructions AND implementing single-stepping of instructions, which will help both the core development team and also any switch homebrew developers enormously.
To seal off her month in style, a final couple of BLX and BXWritePC instructions were fixed, and you can expect to hear more exploits in the next report!
Not to be completely outshone, gdkchan took up the CPU torch and added a limit on the number of uses a constant may have which was preventing “Deathsmiles 2” from getting in-game.
A second interesting addition by gdk was the implementation of CPU JIT invalidation (along with a quick PPTC version bump) which extends the existing region that can be invalided by the JitCache to actually remove functions that overlap a given range. This change is of particular interest because it begins to lay the groundwork for applications that dynamically load code in NRO’s or for code that is dynamically self-modified to function.
That last part is required for modding frameworks like Skyline and ARCropolis as they take advantage of this self-modifying code to hook the game at run-time.
MISC:
While we’re on the topic of Super Smash Bros: Ultimate many of you will be aware that the game can stutter in a few places. There are three causes for this: the first and most common on first boots is shader compilation stutter, the second is NRO stutter at the beginning of each match and the last component was menu and character select screen stutter. This final problem was tackled by riperiperi this month with the addition of a dedicated thread (ServerBase) for FileSystem services.
The root of the problem was that some file system services were blocking other services that did not have a dedicated thread and thus giving FileSystem its own prevents it from bottlenecking other services and causing stutters while they wait for the filesystem service to clear. This change also improves other games that suffered from filesystem related stuttering such as some Xenoblade: DE cutscenes, Fire Emblem: Three houses cutscenes and also for users who store their games on network drives or other external solutions.
Controller drift is a phenomenon that has come into the spotlight in recent years and one of the few ways we have of tackling this globally is via an increase to the emulated deadzone. However some users quickly noticed that the math we used to calculate how to apply the deadzone was quite flawed. See below a visualization of our old implementation:
This means that if you set a deadzone of say 30% it would be applied in both X and Y equally meaning that fine motions at the outer edges would also be considered “dead” due to X and Y being treated as separate components of the stick vector. At extreme deadzone values of above 50% this could result in the analog stick feeling almost like an 8-axis d-pad with only cardinal directions functioning properly.
Quick to jump on this problem, skrekhere implemented a new deadzone algorithm which now smooths the deadzone at the outer edges and makes fine control possible again even at massive deadzone values like 80%. Check out the new visualization below:
Backend infrastructure is something most people who have never worked on large projects never even think about but the work never stops there either. For some smaller tweaks this month turbedi was knocking it out of the park with: unused EnumExtensions being removed, optimizations to static data in C# compilation, swapping BitUtils with .NET BitOperation methods and finally collapsing AsSpan() operations to use a more modern approach with less, faster code paths. Small changes add up and we always encourage any novice-expert C# developers to look around the codebase because optimization is always possible!
Thog continued her reign of terror over infrastructure changes with a data type adjustment to PID and, thanks to merryhime fixing some GTK bugs in our dependencies, an update to the GtkSharp (Ryujinx’s GUI framework) which should speed up Windows build times and fix a bug where the menu icons would flicker when hovered over.
Game icons in the GUI used to simply use their base icon for the preview but some games actually include an “updated” icon in their update files that would display on the Switch if you were to update a game. Ac_K decided they couldn’t stand this inconsistency any longer and now Ryujinx will pull the game icon from the update file if one exists.
Our filesystem wizard Thealexbarney (or as you may know him by his discord name ‘Moosehunter’) merged three changes this month. The first being a fix for a file system “permission denied” error that was plaguing users seemingly at random. This change runs some extra data fixes on any saves without a valid owner ID and should resolve most of these issues. Next was the removal of a lot of log clutter in the “ServiceNv map” creation spam that would take place in every game. This provides no information to the general user and no additional information when troubleshooting generic problems so it has now been shifted to the debug logs category where we wish it a very happy retirement.
Last but not least LibHac was given a bump to 0.16.0 which added support for reading NCA’s with compressed sections, increased resilience against invalid extra save data (may help recover from external “messing” from other programs/OS etc.) and finally fixed a FileSystem access control check. The first of these changes is perhaps the most exciting as it allows both Iridium and Gunvolt Chronicles to get in-game and it seems both are fully playable!
To no one's surprise our linux users are also very active in the development process and regularly submit PR’s with various fixes for 3-letter acronyms and words with lots of underscores in them. Continuing this trend, edisionnano pushed a fix for Ryujinx’s Backend Multithreading on MESA drivers where we were providing a string raw and MESA was expecting it to be lower-case. This simple problem was wreaking havoc when setting environment variables and so passing a lowercase argument is now enforced.
They also resolved a minor build bug where the platform specific binaries for the SoundIO audio would be shipped wholesale to all OS’s it was built on. This meant Windows and Linux would build binaries for each other that couldn’t be used and was ultimately just bloat.
A new contributor to the project, mlgatto, added a new trace-level log which gdkchan immediately made use of by moving all kernel syscall logs into and continuing in the vein of simplicity wss445566 fixed a long-standing typo for us further proving that just about anyone can help us out even if it seems small!
Ac_K is going to finish off this report in style by stubbing some new service calls: mnpp:app, which seems to be a telemetry module for chinese consoles which was causing NES/SNES NSO to crash, and some HID services required for the aforementioned Switch Sports to launch.
CLOSING WORDS:
We know it’s tough times for everyone right now so we’d like to express our immense gratitude to everyone who has been contributing to the development of Ryujinx, whether it be through patreon, testing, or just being part of the community. Never could we have imagined the amount of support we’d get for this project and for that you all have our gratitude. Until next time, stay safe everyone!