We apologize for the late Progress Report, but at this point it's partially by design. There's been an ongoing issue with Dolphin's updater being recognized as a trojan by Window's Defender Cloud AI scanning. The good news is that Microsoft has acknowledged that Dolphin's updater isn't a trojan, however for now they have to manually whitelist our executables. In order to ensure that the monthly builds distributed through our update track aren't deleted by Window's antivirus, we've been verifying that the build we've chosen is whitelisted. If you're interested in learning more about how something like this happens, MayImilae researched the issue and wrote up a detailed report below on what is happening and where we stand on the problem for now.

Until further notice, please keep reporting these erroneous detections so our builds can be whitelisted by Microsoft until they get their AI sorted. Thank you. Without further ado, let's jump into a smattering of significant changes that hit this month, including a way motion features in some of your favorite controllers.

Notable Changes¶

5.0-10950 - Remove XAudio2 by ligfx¶

XAudio2 was Dolphin's default audio backend on Windows for a very long time. It was pretty much the only way to get decent low latency audio on Windows, plus it was a stable, consistent, and reliable; an obvious choice for default audio backend on the platform. But times have changed.

Dolphin's new default audio backend Cubeb is an upgrade in every way. Developed by Firefox developer Mozilla, Cubeb is an open source cross-platform audio library that interfaces with local audio APIs to bring low latency audio everywhere. It is extremely easy for us to support thanks to its open design, and it's multiplatform so any improvements and new features will benefit all operating systems that we support. Most importantly to our users, Cubeb utilizes with the same internal audio API that XAudio2 uses (WASAPI) on Windows, while providing slightly lower latency! The only thing that our Cubeb audio backend lacks is a feature that XAudio2 also doesn't support: WASAPI exclusive mode. This mode is a direct way to access the core API and provides extremely low latency at the cost of only allowing Dolphin to emit sound. But we now have a dedicated WASAPI (Exclusive Mode) audio backend specifically to take advantage of that feature.

XAudio2 still worked fine, but with superior options available that offer more versatility, easier maintainability, and better performance, it was no longer worth maintaining.

Now that all of growing pains with these new audio backends are sorted, we decided it was finally time for Xaudio2 to ride off into the sunset. For users coming from an older build with Xaudio2 selected the Audio Backend dropdown will appear blank since Xaudio2 is now an invalid selection, but Dolphin will automatically use Cubeb in that scenario. We'd recommend users switch the audio backend to Cubeb on their own just so it isn't blank, but there will be no issues if you leave it be.

5.0-10996 - Move CPU Emulation Engine Options to the Advanced tab by ethteck¶

There was a time when Dolphin had multiple CPU emulation options that users would want to switch between. Between JIT bugs, JITIL weird quirks, and the constant need to verify issues with the interpreter, it made sense to have these options front and center. Nowadays, CPU emulation options have been shored up and there's no real reason the average user would want to swap out of the JIT. Considering that these options now only appeal to advanced users looking to debug issues, it was determined that they should be moved to the Advanced tab.

5.0-11035 - Expression Parser Improvements by Billiard¶

The GameCube and especially the Wii are home to some very unique controllers that can be difficult to map to conventional input devices. A lot of work has been done to make sure that Dolphin's controller configuration interface is up to the task, providing an easy to use way to map all kinds of advance functions. But there is a limit to what the main controller configuration can do, so for those times when it just isn't enough, Dolphin has an Advanced Input Configuration window (available by right clicking input boxes) to help users handle even the most complicated control setups. That said, it wasn't exactly easy to use. Users were entirely on their own to figure out this powerful tool, and even those familiar with it could stumble from time to time. Billiard, continuing his crusade of improving our input systems, made our Advanced Input Configuration UI a bit more accessible so he could get away with making it even more powerful.

On top of adding a ton of new expressions, Billiard also added dropdowns with various functions to help users see what is possible, and verification so users can see mistakes in their expressions. Advanced Input Configuration is more powerful than ever, and a little easier to use, too.

5.0-11054 - Add Redump.org integration and 5.0-11059 + 5.0-11071 - Improve Verifier Edge Case Handling by JosJuice¶

Game preservation is a very important topic among our community. Our developers have dedicated years toward properly reproducing the behaviors of the GameCube and Wii, all because we care about these systems and their games. But no matter how accurate emulation becomes, an improperly dumped disc can cause all manner of bugs, from simple graphical glitches to game breaking crashes. The recently added verification tab in the Game Properties Menu was our way of helping users more easily check their games without the need of separate programs. It not only can check games, but it can even notify users of different potential problems in their dumps.

There is a limitation to game verification heuristics, though. While the tab could tell you if the dump should work or not, but it had no way of telling you if it was a verified dump. This is especially problematic with GameCube games, as there aren't many ways to tell if the dumped data is correct or not. In fact, the best way we have is to simply compare the results to what other people got when dumping. To do this, Dolphin now leverages a database maintained by Redump.org. Their database is home to near complete libraries of verified dumps for many consoles, including the GameCube. By leveraging their database in the verification tab, Dolphin can now identify bad dumps that would previously go unnoticed.

As an aside, special mention to the Datel Freeboot Wii disc for not having a data partition even when correctly dumped and forcing us to add an exception with our verifier. Even with all the rules to proper Wii dumps, they still somehow broke things.

5.0-11083 - Support for motion controllers like the DualShock 4 by rlnilsen¶

Earlier this year, Emulated MotionPlus hit the scene making games like The Legend of Zelda: Skyward Sword playable without needing a Wii Remote. For the first time, you could map both the gyroscope and accelerometers inputs to other input devices. However, many of these games are difficult to play without motion controllers, so another step had to be taken. If we are going to preserve these experiences beyond the lifespan of Wii Remotes, we needed to be able to harness the power of modern motion controllers.

Unfortunately, figuring out a way to do this cleanly has been nearly impossible. On Linux, motion sensor axis are just exposed and available, because its Linux and people would probably riot if they didn't have full control over everything. On Windows, the situation has been quite the mess. Pretty much every controller API on Windows was designed for contentional controllers and will completely ignore motion sensors. A few options have existed on the fringes, and we have even implemented a few of them such as our old UDPWiimote, but we found they had either huge limitations or weren't workable for our needs. Fortunately, we are no longer alone in dealing with this problem. Cemu and Citra both take advantage of the DSUClient standard. Using a fork of DS4Windows' custom drivers and a UDP server/client system, they can access motion data directly without any standard controller API! And now Dolphin can take advantage of this as well, thanks to rlnilsen's work.

Even though Linux users don't need the DSUClient, the work put into our UI to expose motion input will benefit Linux users as well! The process is more or less the same on Linux, minus the custom driver and DSU client bits that make it work on Windows. Linux users can just map the motion sensors directly.

What Can This Do:¶

If you have a controller such as a DualShock 3, DualShock 4, Nintendo Switch Controllers, or a Steam Controller, it's possible to pass motion sensor data directly to Dolphin. As an added bonus, Dolphin can also create infrared data based off of Gyroscope and Accelerometer input thanks to the simulated Wii Remote added to make Emulated MotionPlus work properly. Essentially, this means that you don't need to worry about infrared data at all, simply moving the controller will move the infrared pointer as if you were using a Wii Remote and Sensor Bar. In games like Resident Evil 4 and Super Mario Galaxy, the simulated pointer can actually make a game easier to play, as you're no longer tied to many of the limitations that can hamper infrared.

Motion passthrough also has some other benefits that go beyond using real Wii Remotes. In Super Mario Galaxy you can still take advantage of other emulated inputs, such as mapping shaking to a button. This means that you can point with real motion while being able to press a button to do Mario's spin. Undeniably, there are limitations. Some games are designed around the very particulars of the Wii Remote's shape, and there is that little issue that its attachments can have a second accelerometer.

For a game like No More Heroes that forces players to move the Wii Remote and Nunchuck in all kinds of different directions at the same time, things get a bit trickier. We can actually use our good friend 5.0-11035 - Expression Parser Improvements to make some adjustments. Because the game expects you to hold the Wii Remote with one hand, left and right movement can be more difficult to do at the force the game expects. But using the expression parser, we can add a multiplier to the accelerometer movement to make motions easier to use. With only one set of motion inputs, we can use the second stick of a DualShock-like controller to map the Nunchuck swings. With all of this put together, the game is playable and enjoyable.

By wielding the power of modern controller sensors, using emulated Wii Remotes is now almost on par with real Wii Remotes. That isn't to say that it will always be perfect, and some configuration hassles may be required, but we can finally say that we're at a point where we're not dependant on original hardware to give a good experience. Considering how awkward Dolphin's motion simulation was just a year ago, we have to acknowledge the work several developers have put in to get us to this point. Whether it was removing unmaintained features, fixing bugs, or reverse engineering behaviors, they're the ones that made getting to this point possible.

In addition to this, there are several more changes to emulated Wii Remotes in the pipeline. We expect Nunchuck Motion Input to be finished in the next couple of weeks, allowing you to map Nunchuck motions to the same controller or another. On the API side of things, we're keenly looking forward to being able to leverage "wand" type controllers as they should give a nearly 1:1 user experience to that of the Wii Remote.

How To Setup:¶

Because motion controls are such a mess on Windows, setting this up the first time can be a bit daunting. Depending on what controller you want to use, your instructions and what program you need to use to passthrough motion data will vary. You can access Dolphin's wiki and check out the DSU Client Page for detailed instructions on how to set it all up.

On Linux, setup is far simpler. Just correctly map the motion sensor axis into the Motion Input tab and you are good to go!

5.0-11098 - Android - Mali EFB2RAM Optimizations by Stenzek¶

We finally have some positive news for our Android users on Mali devices. While Adreno devices are still faster, Stenzek has finally managed to eliminate one of the huge bottlenecks causing abnormally low performance on Mali. Store EFB Copies to RAM is a demanding feature, and some performance drop should be expected, however Mali devices were dropping far beyond our expectations in test cases.

Stenzek investigated the framedrop and realized that Mali was spending a ton of CPU time in the "__pi___inval_cache_range" kernel function. While that may not mean a lot to most of us, it essentially told Stenzek that a lot of time was being spent invalidating cached memory. By switching to uncached memory, he improved framerates by roughly 30% on afflicted devices in games that require Store EFB copies to RAM.

As a note, this only affects games that use Store EFB Copies to RAM only, so don't expect performance boosts on every single title. This also does not affect Adreno devices whatsoever, as no change in performance was recorded in vigorous testing.

5.0-11102 - Support Vulkan Exclusive Fullscreen through VK_EXT_full_screen_exclusive by Stenzek¶

This change is a bit of a formality, as most graphics drivers already attain Exclusive Fullscreen with Vulkan on their own. By supporting the Vulkan Exclusive Fullscreen extension, we're just making it so that there is no confusion and hopefully it makes things more consistent. As mentioned in other Progress Reports, exclusive fullscreen is essentially a more direct way to output to your monitor, reducing latency by bypassing desktop compositors and making sure all frames are delivered properly. While this won't affect most users, this may eliminate some edge cases where certain users were unable to attain exclusive fullscreen.

As a special note, this extension is fairly new, so you will likely need to update your graphics driver for support.

Windows Defender: Yelling at a Cloud AI¶

As we previously talked about on our twitter, Windows Defender (soon to be renamed to Microsoft Defender) has been falsely identifying our updater.exe as a trojan. We went over the struggle of identifying and reproducing the issue in the tweet, so if you want a high level discussion of this issue as well as details on our struggle to indentify it, please give it a read. For the Progress Report though, we thought it would be best if we go full on and properly explain all of the details of the situation as we understand it. There was no way to explain this while holding back, so this is going to get technical!

After a lot of back and forth with users and testers, difficulty reliably reproducing the issue, and a few weeks of stumbling about trying to reproduce a moving target, we realized from the detection name that the source of Windows Defender's false positive is from Microsoft's machine-learning based cloud services. Microsoft is aware of the issue and responded very promptly, but because it is machine learning based, they can't so much "fix" the false positive as much as "encourage it toward the correct answer.", and it appears their AI is very convinced that updater applications are trojans. All Microsoft could do was whitelist our updater as a workaround. We assumed that this would solve the problem, but as the user reports continued to roll in, we realized this issue was going to be a lot messier than we hoped.

Due to how compilers work, even if you change absolutely nothing within a file, the act of rebuilding is going to change a few bits here and there. This is normal. We don't know much about Microsoft's side of the equation, but it appears that Windows Defender's whitelisting is using a very basic hashing algorithm where even a single bit change in a file will completely alter the hash and Windows Defender will consider them to be distinct, unique files. The combination of these two factors is why the whitelisting isn't helping much. Every new dev build has a freshly built version of updater.exe, but since that updater.exe is a "new file" to Windows Defender, each build's updater.exe must be specifically whitelisted by Microsoft. So nearly all builds with the updater, old or new, are being falsely detected as trojans and deleted by Windows Defender until someone reports that specific build's false positive to Microsoft and they whitelist it. This isn't great, to say the least.

We can't do anything about Microsoft's side, and unfortunately there isn't much we can do on our end. After all, we're not doing anything unusual. Small changes during rebuilds is normal for compilers; deterministic (bit accurate) building is a fairly recent concept and isn't all that well supported yet. For example, MSVC, Microsoft's own compiler and what we use to build Dolphin Windows builds, has an undocumented and hidden compiler flag called BREPRO which is intended to improve bit accuracy between builds. Unfortunately it didn't work for us, and since it is totally undocumented we have no way to know why.

One possible way to avoid these minor build variations is to store all of our previously built files, and if a file doesn't have any change in a new build, we could reuse that existing file unmodified in the new build. That's known as compilation caching, and could, in theory, allow our updater.exe to stay exactly the same for longer so each whitelisting by Microsoft could last longer too. But, as always, reality is not so simple. After all, Dolphin's buildbot has used compilation caching for years, yet we still encountered this problem! Well, compilation caching can carry over files unaltered between builds, but only if the file and its dependencies are unchanged. Our updater shares a lot of code with Dolphin proper and has dependencies in Dolphin's common code, so any time common is changed the updater is relinked. So even with compilation caching, updater.exe has those extremely small changes nearly every build.

Due to the permissiveness of cmake, any changes in common will lead to a relinking of the updater, whether they are updater dependencies or not. We could reduce the relinks somewhat by moving to a less permissive build system (e.g. bazel), but that would require us to exchange our entire build system, which would be a massive amount of work. This would only reduce the relinking a bit, and it wouldn't give us any performance advantages since relinking is free. It would be huge waste of our time and energy, all just to make it a little easier for Microsoft to dance around their cranky AI.

The options just get worse from here. We could separate the updater entirely from the emulator and duplicated the shared code, but, no. No. Just no. That's a horrible idea.

Honestly, these are all things we shouldn't need to even be worrying about. Small bit variations between builds is completely normal, and the amount of work required to maybe reduce these variations is immense. While we'd love to find some way to fix this for our users, it doesn't seem like we'll be able to. It's up to Microsoft to improve their misbehaving AI and stop falsely flagging applications for normal behavior and variations. Until Microsoft is able to solve things on their end, please continue to report Windows Defender false positives of our updater to Microsoft. Not only will this help them whitelist new versions so other users won't encounter the false positive, but it may eventually convince their AI that we are not a threat.

Last Month's Contributors...¶

Special thanks to all of the contributors that incremented Dolphin from 5.0-10947 through to 5.0-11102!