Patreon Goals Update & Progress Report February 2021
Progress Report February 2021
We are well into Q1 and have once again been burning the midnight oil to crank out bug fixes, optimizations, and quality of life improvements for end users. February also saw the release of Haydn: Part 1, a complete re-implementation of the audout & audin functionality resulting in, among other benefits, a significant reduction in CPU usage while using the emulator.
Before we jump into last month’s nitty-gritty, it’s time for a Patreon goals update:
Amiibo - PR is publicly available for testing. There are some intended changes & improvements to functionality that will be forthcoming soon prior to the PR being merged into the main build.
Custom User Profiles - in progress
Vulkan GPU Backend - in progress, ETA April 2021. See below for a more in-depth report on the progress of this feature.
$1500/month - ARB Shaders - Not yet met
ARB shaders will further reduce stuttering on first-run by improving the shader compilation speed on NVIDIA GPUs using the OpenGL API.
We have added the following goals for developers to be able to work full-time on the Ryujinx project:
$2500/month - One full-time developer - Not yet met
This amount of monthly donations will allow the project's founder, gdkchan, to work full-time on developing Ryujinx.
$5000/month - Additional full-time developer - Not yet met
This amount of monthly donations will allow an additional Ryujinx team developer to work full-time on the project.
We realize there is a large gap between the ARB Shaders goal and the first full-time developer goal, so we will be looking to add at least one more goal soon to bridge the gap.
On the topic of Patreon goal-spiking:
With the ultimate goal of our Patreon campaign being the ability of one or more developers to work on the project full-time, we are looking for steadily growing support and thus a steadily growing number of Patrons. This is why we set some easily attainable monthly income goals.
Our $1250/month Vulkan goal was recently reached largely by a single donor “subscribing” with a sizable sum who then immediately withdrew their support. Technically this pushed the Patreon goal into “reached” territory briefly and gdkchan, the Ryujinx project founder, initiated work on Vulkan as promised. However, you can see currently on our Patreon page that the goal no longer shows as reached. Unfortunately this puts the team in a difficult position as the bar for delivering the feature hinges on meeting a particular income amount per month, and the goal of receiving $1250/month is clearly not met.
All the same we have decided to continue prioritizing Vulkan until the implementation is completed. However, to ensure any future Patreon goals are being legitimately met, we will require at least two billing cycles of a Patreon goal being reached before dropping everything to work on the feature in question. We hope this is seen as common sense, and want to emphasize our gratitude and appreciation for all Patrons.
Vulkan Progress
As mentioned above, last month we reached the Vulkan goal and work is already underway to support this API on the emulator. It is still in the early stages of development, but the results are already looking promising.
We plan to deliver this feature in the middle of April, and if possible, make a preview available by the end of March, so stay tuned for any updates! Below you can see some screenshots that show how far along the Vulkan backend currently is.
Vulkan FAQ
Will Vulkan include a shader cache?
The main goal right now is getting Vulkan working before the estimated release date. If this is done before the estimated date, and there's still enough time left for a shader cache implementation, then shader cache could be added. Otherwise, it will be left as a follow-up task and will be implemented after the Vulkan implementation is merged into the master branch of the emulator. To clarify, our ETAs of these Patreon goal features is the estimated time to develop the feature and submit the PR (pull request) for public testing & review. Actual merge dates into the main build may vary and do not have ETAs.
Will Vulkan include async shader compilation?
No, for the same reason it is not implemented in OpenGL. The async shader compilation feature implemented on other emulators come with compromises like visual glitches while shaders compile, some of them being persistent until emulation is stopped. It also does not fully eliminate stutter as some shaders cannot be compiled asynchronously. For those reasons, the current Ryujinx GPU emulation developers are not interested in implementing it. We are not opposed to an external contributor implementing this feature, however.
Will Vulkan support resolution scaling?
Yes. The goal is having feature parity with OpenGL upon release.
What kind of improvement can I expect?
AMD and Intel GPU users should expect a significant improvement in compatibility, performance and, especially for Intel GPU users, a drastic reduction in graphical glitches. NVIDIA users can expect a small performance improvement in a few titles (we will defer saying which ones until after the Vulkan implementation is properly optimized and tested). All GPU vendors should benefit from faster shader compilation and less stuttering on Vulkan.
What's currently missing?
The main features missing for parity with OpenGL at the time of writing are resolution scaling, transform feedback, multisampling, buffer textures, alpha test and conditional rendering. There are still a few bugs that need to be fixed, and after this we'll need to look into vendor specific issues and commence general testing. The SPIR-V (Vulkan’s native shader language) implementation is about 65% complete.
Progress Report
GPU Improvements
Vegas, Baby! - Implemented missing ETC2 (RGB) texture format
Missing texture formats can present in a variety of ways, usually isolated to a particular object or area in a game. In Vegas Party’s case it wasn’t just a particular object affected, but nearly all graphics being rendered in the game.
Here we can see Vegas Party looking as if the user’s GPU fan has stopped spinning and is nearing meltdown:
With the ETC2 (RGB) texture format added, we are ready to start placing some bets.
Implemented on (#2000) by gdkchan
Kernel & CPU Improvements
Disable partial JIT invalidation on unmap
Back in December, some JIT cache changes were implemented to fix a memory leak and resolve a bug with homebrew. Part of those changes involved the invalidation of functions after unmapping memory. Unfortunately, a side effect of this incurred longer boot times in games that make heavy use of Map/UnmapPhysicalMemory, which includes some games utilizing Unreal Engine 4.
Completely removing the invalidation on unmap function resolved the boot time regression on affected games; fortunately, the fixes provided by December’s JIT cache changes were also able to be retained.
Implemented on (#1991) by gdkchan
Fix inter-process data copy on non-contiguous physical regions
After the latest IPC refactor changes in December, a few games that use non-contiguous physical memory regions began crashing on boot due to memory corruption. Some of those affected included the Bravely Default II Demo and Balan Wonderworld Demo. Changing the inter-process data copy approach to take physical regions into account fixed the issue, allowing the games to boot again.
Implemented on (#1988) by gdkchan
Share the Load - Enable multithreaded VP9 decoding
Many Switch-exclusive games use a particular codec for their pre-rendered videos called VP9. This codec was developed by Google and first used extensively by YouTube. Up until now, Ryujinx has been only using a single CPU thread to decode these NVDEC videos. While sometimes this was sufficient, in many cases it just wasn’t enough horsepower to get the job done. This led to stuttering video, choppy audio and, in some cases, lengthy pauses while the emulator struggled to keep up. This change allows up to 4 cores to be used for decoding.
Hover over the videos below to reveal volume & playback controls if desired.
The Legend of Zelda: Link’s Awakening (tested on a Ryzen 7 5800X)
Before:
After:
Tokyo Mirage Sessions #FE Encore (tested on a Ryzen 7 5800X)
Before:
After:
Super Smash Bros. Ultimate (tested on a Ryzen 5 3600)
Before:
After:
Similar improvements, dependent on your hardware, can be seen in other videos included in Animal Crossing: New Horizons, Fire Emblem: Three Houses, Pokemon Let’s Go Eevee/Pikachu, and Mr. Driller DrillLand.
Implemented on (#2009) by gdkchan
PPTC & Pool Enhancements
As PPTC (Profiled Persistent Translation Cache) has matured, so have its advantages and associated benefits. Though LDj3SNuD’s introduction of a simple Pools limiter in January resulted in a significant reduction of emulator memory usage and further shortened boot times, it came with a unique issue seen in many games (including Monster Hunter Generations Ultimate and Tokyo Mirage Sessions #FE Encore) involving what amounted to memory spikes at timed intervals. These spikes happened at the same time a console entry about PTC profiling info was logged, and seemed to cause a noticeable micro-stutter in video rendering and audio playback, if the user was paying close attention.
This update not only fully resolves the aforementioned memory spikes but also overhauls PPTC .cache file handling, dramatically reduces emulator memory usage, accelerates PTC load and save functions by a factor of 10, and reduces the duration of reJIT operations by roughly half, among other optimizations and fixes.
Below we can see the effects of this update on Monster Hunter Generations Ultimate memory usage while in-game (note the clear memory usage peaks in the “Before” screenshot).
Before:
After:
Implemented on (#1968) by LDj3SNuD
We don’t serve your kind here - PPTC: Fix unwanted propagation of a relocatable constant in a specific case
In December the “Clear JIT cache on exit” PR was merged, resulting in a simplification of the Add instruction and subsequently the unwanted propagation of a relocatable flag of a constant. This in turn could cause a “failed to encode constant” crash if PPTC was enabled in a small subset of games. With this update, constants can no longer be contained in the instruction’s immediate and the issue is resolved.
Implemented on (#1990) by LDJ3SNuD
HLE Improvements
Haydn: Part 1 - Reimplementation of audout/audin
Ryujinx’s original audio api was added in 2018, without adapting it to audout & audin (the audio output & audio input services on the Switch OS, which allows interfacing with the Switch hardware), and had largely not been touched since. After Ryujinx’s release of Amadeus (complete audio renderer) last August, it seemed time to revisit the overall audio API and give audout & audin the attention they deserved. Wanting to have the audio IPC side fully done and accurate, Thog set about reverse engineering audio 11.0.0 and developed a complete reimplementation of audout/audin.
The result of this achievement is that audout is finally accurate, and end users can enjoy lower CPU utilization while playing their favorite games (50% reduction in CPU utilization on average while using the SoundIO backend!). Additionally, in preparation for the next steps to finish out the audio implementation (audio backend rework), the Ryujinx audio project was reorganized.
Implemented on (#2007) by Thog
Allow modding AddOnContent RomFS
Modding the romFS of a game can add all sorts of fun to a game—replacing in-game objects like textures, character skins, or even entire karts in Mario Kart—but up until now that has only been possible on a game’s original files. With this update, DLC itself can also be modified.
Implemented on (#2024) by mageven
Improve heuristic for showing the keyboard
The software keyboard implementation has received several recent updates, improving functionality and compatibility last month in Monster Hunter Generations Ultimate, Fate/EXTELLA LINK, and Gnosia. However, there was one field in God Eater 3 that was impossible to edit. While not being a game-breaking issue, it still presented a jarring experience to an end user wishing to change their CodeName.
As you can see below, attempting to change the CodeName resulted in the word “Ryujinx” being automatically substituted in even across multiple attempts.
After this update, the software keyboard applet pops up correctly and allows the user to enter in their desired text.
Implemented on (#2066) by Caian
Implement prepo: GetSystemSessionID and add perms
February’s debut of the wildly popular Super Mario 3D World + Bowser’s Fury game bundle was fully playable immediately on day 1 in Ryujinx, that is, as long as the “Ignore Missing Services” option was enabled! Without the option enabled, the game would crash after the title screen indicating a missing prepo service. Within a few hours of the game’s release the missing service was reverse engineered and implemented, allowing the game to boot normally with the emulator’s default settings.
Implemented on (#2014) by AcK77
GUI Improvements
Now you see me, now you don’t - Hide Cursor On Idle
There’s nothing like a mouse cursor hanging out in plain view to ruin the experience, whether you’re watching a movie or playing your favorite video game in an emulator. Ryujinx now allows you to hide the mouse cursor after idling for 8 seconds, if you so choose!
Implemented on (#1993) by pineappleEA
We’ll do it live! - Allow reconfiguring “Ignore Missing Services” during emulation
Many users are already aware of the “Ignore Missing Services” hack that allows games with missing services to proceed with emulation. In many cases this facilitates normal gameplay whereas, in those few games that have missing services, without the option being enabled the emulator would crash. But a sticking point about this option is that it had to be enabled before booting a game in order for a missing service to be “successfully” ignored. With this change, the option can be toggled on the fly! So if you have already launched a game and wish to change the setting, it will take effect immediately.
Implemented on (#2034) by mageven
Make windows DPI aware to display properly on high-resolution screens
With the advent of 4K monitors and other high-resolution screens, many users end up enlarging font size in their operating system to compensate. Unfortunately, this caused some unwanted behaviors in the controller mapping UI when fonts were scaled, causing fields to go out of alignment, and images to be cut off. This update informs Windows that Ryujinx is DPI aware, and takes care of the scaling in other operating systems as well.
Before:
After:
Implemented on (#1983) by saldabain
Unsolicited Input - Controller Input handling refactoring
A new code contributor made a splash with their first PR getting merged, refactoring controller input and making several quality-of-life changes benefitting users with DirectInput controller devices & joysticks. One of the most notable issues plaguing DirectInput controller users was that while a user was trying to map their buttons, a single errant button would be constantly spammed instead. This made it frustrating if not impossible to map button configurations using the GUI. With this update, that problem is finally resolved. Other improvements include the conversion of button events to signals in range [-1,1], detecting a pressed button by measuring a change in signal while taking the min, max and avg into account, and in cases where a trigger is detected as both a button and an axis, the axis wins.
Implemented on (#1751) by dce31t
Miscellaneous Improvements
Revise SystemInfo
A small but impactful change was made to how Ryujinx initializes, gleaning system info like processor & RAM information from CPUID and parsing of procfs files in Windows & Linux, respectively. The net result is that the Ryujinx application time has been cut roughly in half.
Watch the before/after comparison video below to see the improvement in startup time:
Implemented on (#2047) by mageven
Pacify updater
Portable Mode options were added in August of 2020, and many users rejoiced at the new flexibility. When an automatic updater was implemented last Fall the Ryujinx community at large celebrated the saved time, no longer needing to manually fetch the latest build from the Ryujinx website each and every time the emulator was updated.
But a specific use case made this a potentially deadly formula regarding one’s Ryujinx system data folder.
Until now, the auto-updater process purged the entire Ryujinx program folder during an update process. For those particular users who manually assigned their data folder to be a subfolder within the Ryujinx program folder, this meant that all of their save data, shader caches, PPTC caches, etc. would be nuked simply by updating to the latest version of the emulator. Despite included warnings in portable mode guide documentation, some users understandably fell prey to this insidious combination.
With this update only Ryujinx program files are purged during an auto-update process, preserving any other folder structures underneath the Ryujinx program folder. This allows relative paths and other standard subfolders to be used without any risk of deletion from the auto-updater.
Implemented on (#1899) by mageven
New code contributors February 2021:
D3ce1t
Mou-Ikkai
Thanks to everyone that has supported us so far, be it via Patreon donations, code contributions, testing games in the emulator, or simply being an active member of our community. You’ve helped make this emulator what it is today!
We now have an active Patreon campaign with specific goals and restructured subscriber benefits/tiers , so if you are interested, head on over to help push Ryujinx forward!