Progress Report November 2021
November was a brilliant month for Ryujinx and Nintendo game fans alike. We saw some shining GPU improvements and HLE updates and an update to our .NET version. Nintendo game fans saw the long-awaited release of Shin Megami Tensei V and also remakes of the classic DS-era Pokémon games re-released as Brilliant Diamond and Shining Pearl.
Patreon Goals
Amiibo Emulation- merged into the main build in March 2021. Some new amiibo were added this month! Check below for more details. While compatibility is now almost perfect, there are still some improvements to come for Amiibo which can be tracked on the associated Github issue here: https://github.com/Ryujinx/Ryujinx/issues/2122
Custom User Profiles - merged into the main build in April 2021.
Vulkan GPU Backend - still in progress, a public test build is delivered. A lot is being worked on.
ARB Shaders - Goal reached in April 2021. As seen from August's progress report, preliminary work on ARB shaders has begun. More information at the end of this post!
ARB shaders will further reduce stuttering on the first run by improving the shader compilation speed on NVIDIA GPUs using the OpenGL API.
$2000/month - Texture Packs / Replacement Capabilities - Almost there!
This will facilitate the replacement of in-game graphics textures which enables custom texture enhancements, alternate controller button graphics, and more.
ETA once the goal is reached: ~3-4 weeks
$2500/month - One full-time developer - Not yet met
This amount of monthly donations will allow the project's founder, gdkchan, to work full-time on developing Ryujinx.
$5000/month - Additional full-time developer - Not yet met
This amount of monthly donations will allow an additional Ryujinx team developer to work full-time on the project.
Amiibo updates
Added Animal Crossing Series 5 Amiibo:
- Tom Nook
- Timmy & Tommy
- Isabelle
- Orville
- Wilbur
- Blathers
- Celeste
- Mabel
- Sable
- Label
- K.K. Slider
- C.J.
- Flick
- Daisy Mae
- Kicks
- Saharah
- Harvey
- Gulliver
- Wisp
- Lottie (Island)
- Niko
- Wardell
- Tom Nook (Coat)
- Isabelle (Sweater)
- Sherb
- Megan
- Dom
- Audie
- Cyd
- Judy
- Raymond
- Reneigh
- Sasha
- Ione
- Tiansheng
- Shino
- Marlo
- Petri
- Cephalobot
- Quinn
- Chabwick
- Zoe
- Ace
- Rio
- Frett
- Azalea
- Roswell
- Faith
GPU
Clamp number of mipmap levels to avoid API errors due to invalid textures
Some mods for games use an invalid mipmap level count and that causes the textures to be solid black as the texture initialization will fail. The approach to fix this is fairly simple, Ryujinx now clamps the amount of levels to avoid OpenGL errors on textures with an invalid number of mipmap levels. While this technically means the mod is incorrect, doing so more closely matches the hardware behaviour, as the GPU does not perform validation of those parameter, and would not fail to sample from such texture.
Implemented by gdkchan in #2808
Implement DrawTexture functionality
A few games on the Nintendo Switch that use OpenGL or NVN (the Switch's proprietary graphics API) uses the DrawTexture command. It can be used to draw a texture on the current render target without needing to use shaders, or having any geometry.
This fixes Steel Assault not rendering:
Also fixes Final Fantasy VII:
Implemented by gdkchan in #2747
Fix InvocationInfo on geometry shader and bindless default integer const
The generated geometry shaders were trying to access the gl_PatchPrimitivesIn built-in variable, which is invalid according to the specification. This is because the invocation info system register is accessed on those shaders, however this built-in can only be accessed on tessellation evaluation shaders. This also contains a fix for another issue that was found while testing this, bindless images with unknown handle source uses a 0 constant right now. However, it was using a float constant for images with integer formats, which is incorrect. Change it to use the proper integer type, this fixes some shader compilation errors. The first issue apparently doesn't affect NVIDIA, as apparently their compiler doesn't complain on OpenGL and it "just works". On AMD and Intel, this fixes a regression that broke geometry shaders; they caused some UE4 games to only show a black screen there because those games wrote a 3D LUT texture from a draw that uses geometry shaders (to specify the slice).
Implemented by gdkchan in #2822
Fix bindless/global memory elimination with inverted predicates
This fixes bindless texture elimination with inverted predicates (like @!P0). Before, it was working with the assumption that the branch would always enter the predicated block if the condition is true, but this is not exactly the case. What the branch does is skip the predicated instruction if the condition is false. So for non-inverted predicates, it will skip it if the condition is false, and for inverted ones, it will skip it if the condition is true (as if it is false, then the instruction is supposed to execute). In both cases, the branch not taken is where it enters the block, so in both cases, the Next block is the one that should match the Phi incoming block.
This fixes lighting issues on Disaster Report 4, and maybe other games. Should also allow global memory SSBO replacement to work in more cases.
Before:
After:
Implemented by gdkchan in #2826
Support shader gl_Color, gl_SecondaryColor and gl_TexCoord built-ins
A few games on Nintendo Switch that use the OpenGL API use shader gl_Color, gl_SecondaryColor and gl_TexCoord built-ins. Nowadays it's pretty much useless since you can just write a fragment shader and do whatever you want from there. Those built-ins are now implemented; it was mostly used for fixed-function functionality before fragment shaders were a thing. One could set gl_Color and gl_SecondaryColor from a vertex shader to set the parameters used for the fixed-function lighting. This is sort of similar to when we added Legacy Attributes gl_Color and gl_TexCoord but instead we don’t make use of any of the compatibility profiles. The key difference here is It passes the values on regular user attributes that are otherwise not used on the next stage. That means it should work on OpenGL without needing a compatibility profile, and it should also work on Vulkan.
This fixes black screen on a few OpenGL games, such as:
rRootage Reloaded:
This one only uses gl_Color as the game doesn't use textures at all.
Dragon Quest III (and possibly the older ones too?):
This one uses both gl_Color and gl_TexCoord.
Implemented by gdkchan in #2817
Limit Custom Anisotropic Filtering to mipmapped textures with many levels
Anisotropic filtering is where it improves texture quality at oblique angles, you can see this very clearly in games like Fire Emblem Three Houses or the Pokémon Sword and Shield games.
While for many games this worked and improved the games low quality textures greatly, sometimes it would cause some severe rendering issues. This was because anisotropic filtering is enabled on something that it shouldn't be, such as a post process filter or some data texture. Instead of relying on a system like this, now anisotropic filtering on Ryujinx maintains two host samplers when custom AF is enabled, and only uses the forced AF one when the texture is 2d and fully mipmapped (goes down to 1x1). This is because game textures are the ideal target for this filtering, and they are typically fully mipmapped, unlike things like screen render targets which usually have 1 or just a few levels. This also only enables AF on mipmapped samplers where the filtering is bilinear or trilinear.
Astral Chain
Before (16x Handheld):
1x Handheld:
16x Handheld:
Along with this great improvement, it’s now possible to change Anisotropic Filtering at runtime, and you can immediately see the changes. All samplers are flushed from the cache if the setting changes, causing them to be recreated with the new custom AF value. This now brings it in line with our resolution scale. Test it to your heart's content!
Implemented by riperiperi in #2832
Fix shader integer from/to double conversion
On the Nintendo Switch, some games use the I2F or F2I shader instructions to convert between double and integer types. If the game tried to do this on Ryujinx, it would report back this error Invalid reinterpret_cast from "F32" to "F64". This was happening because those float conversion instructions have a 32-bit float type, while it was trying to store them as a double, which made it try to reinterpret the value. It was fixed by adding new instructions for explicit conversion from and to double precision.
Implemented by gdkchan in #2831
Better depth range detection
This is a follow-up on a previous update to our depth range detection. It now reads the depth range from the register when it determines it can't guess it from the depth near/far + translate values. On top of that, it adds another case where it uses the register, which is when both near and far values are 0, as in this case too the assumption that TranslateZ = (Near + Far) / 2 for -1 to 1 and TranslateZ = Near for 0 to 1 no longer holds, as 0 divided by 2 is still 0, so there's no way to tell the depth range in this case using this method. This improves rendering of Bastion on OpenGL.
Implemented by gdkchan in #2754
HLE/GUI/Kernel
infra: Migrate to .NET 6
The .NET6 update was quite a nice update as it brought a lot of improvements internally and users should notice a small performance increase with this update.
Implemented by Thog in #2829
kernel: Add support for CFI
Nintendo Switch firmware 11.0.0 introduced basic support for the CFI value being passed in X18. Ryujinx does not implement any random generator in the kernel at the moment as it is unnecessary. As such the KSystemControl.GenerateRandom function is stubbed.
Implemented by Thog in #2839
kernel: Fix sleep timing accuracy
This corrects some mistakes in our previous implementation. The inaccuracies were caught because of Thog’s reversing of kernel 13.x and comparing it with Ryujinx’s, WaitAndCheckScheduledObjects timing accuracy was also improved.
- Greatly improves loading speeds in BOTW 1.0.0. Before they were very long and inconsistent, now they are short (3-5x faster) and consistent.
- Hatsune Miku: Project DIVA Megamix seems to no longer crash for the first time on boot
- Fixes several issues with Hyrule Warriors Definitive Edition where timings would exhibit odd behaviour.
- Users should also note a small performance improvement in quite a few games.
Implemented by Thog in #2828
account/ns: Implement 13.0.0+ service calls
The new Animal Crossing New Horizons 2.0 update was something fans of the series have been craving for. Alongside the enormous content that was added, the new update introduced some new services that were implemented on the switch in firmware version 13.0.0, acc:u0 InitializeApplicationInfoV2 and aoc:u NotifyMountAddOnContent/NotifyUnmountAddOnContent/CheckAddOnContentMountStatus are both needed for the update so they are fully implemented. While work was being done on these services another one was spotted that was not necessary for Ryujinx so IPurchaseEventManager PopPurchasedProductInfo is stubbed as it is used for E-shop purchases and we don’t support purchases from the Nintendo E-shop. It is needed by Dying Light. Which can now boot further.
Implemented by Ack77 in #2820
Nickname! - Init Amiibos with Profile's name!
Not too long ago we hit our patreon goal of adding amiibo support and a lot of work went into it. There are some things left unfinished within the implementation, one of which is amiibo nicknames. Before they were automatically set as no name. With this new change it now defaults to the name of the users Nintendo Switch profile that they have set within Ryujinx
Implemented by Mou-Ikkai in #2804
When waiting on CPU, do not return a timeout error from EventWait
When the nvservices EventWait function was updated to not error out when the function returned Success some new timeouts were introduced by this change where they did not exist before. This likely happened because before this change, it could return without waiting in some cases and after the update that fixed this inaccurate behaviour things such as shader compilation stutter or PPTC stutter started causing timeouts which made some games assert. CPU waits are supposed to prevent this, and it was assumed that they could never timeout since it passes an infinite timespan but some certain aspects of the code forced the timeout to 1s when it was infinite, so it could timeout. So to fix this, the nvservice is now changed to return "false" even when there is a timeout, to avoid guest asserts. This fixes some crashes users noticed in Tokyo Mirage Sessions #FE Encore.
Implemented by gdkchan in #2780
hle: Make Ryujinx.HLE project entirely safe
This follows up on a previous update making the entirety of Ryujinx.HLE project safe.
Implemented by Thog in #2789
Ensure sync points are released and event handles closed on channel close
Some games on the Nintendo switch create a lot of channels and when the game attempted to close these channels, the sync point was not being released resulting in this error (Cannot allocate a new syncpoint!)
Although it did not cause any sort of crashing (at least to our knowledge) it was best to fix the error rather than it being left in the emulator and forgotten for a while. Another issue that was solved by this change was event handles not being closed when returned by QueryEvent. This caused Legends of Mana and other games to crash with the "Out of handles!" exception. This should be fixed as well now, as events are closed when the channel is closed, or when the event is unregistered.
Implemented by gdkchan in #2812
Fix direct mouse access checkbox label
The "help hint" of Settings > Input > Direct mouse access had the wrong text, which was referring to the keyboard instead of the mouse. This corrects the typo.
Implemented by adryzz in #2827
Don't blow up everything if a DLC file is moved or renamed.
When our DLC support was first implemented, although it worked very well, it had a few big issues. Often a very common issue many users ran into is if they moved their DLC file from the path it was set in the DLC.json file that Ryujinx made once you mapped your DLC, Ryujinx wouldn’t know what to do as there was not a fallback put into place if the DLC path was modified in any way. This change makes it so, if the DLC path was modified, it just ignores the file instead of crashing the emulator or bugging out the DLC Manager window.
Implemented by Iostromb in #2867
The ARBitious work continues!
As you can see, progression on ARB is going well! Our project lead gdkchan and developer Thog are both working very hard on this making sure it works well for everyone that can use it for when it's ready to become public. There have been some massive improvements across the board from where we left off in August. The most notable one is that games are rendering correctly now. Our new shader tester greatly helped on the development of this as it helped us ensure ARB shaders were working correctly. In addition to helping us a great deal on the development of ARB it even helped us find bugs that affected GLSL shaders as well. There is still much work left to do on this but don’t worry, we will try to update everyone as much as we can on the progress of this.
ARB FAQ
Q: What is ARB?
A: ARB is a low-level shading language, created by the OpenGL Architecture Review Board, which can be characterized as an assembly language. ARB shaders are significantly faster to compile on NVIDIA drivers than the OpenGL shader language (GLSL) that we currently use.
Q: What can I expect from ARB?
A: Users should expect a significant reduction in shader stutter and overall loading times. This will improve almost every game in terms of shader stutter.
Q: Can I use ARB?
If you have an Nvidia GPU then yes! Sadly Nvidia is the only GPU vendor to make use of this shader language as AMD and Intel drivers lack support for most of the required functions to make it work for them. While this is very unfortunate for our intel and AMD GPU users, work is ongoing to improve our Vulkan API backend and improve overall user experience.
Q: When will ARB release?
A: Soon.
New code contributors November 2021
adryzz
Closing words
We are all incredibly thankful for everyone’s support towards this project so far whether it was through Patreon, reporting bugs, or code contributions. Because of all of you, we’re now able to boot so many games on their release day and have them be playable. We are truly in awe of how far this project has come, so once again thank you!
We have an active Patreon campaign with specific goals and restructured subscriber benefits/tiers, so please consider becoming a patron to help push Ryujinx forward!