Dolphin Progress Report: August 2016


Dolphin started solely as a GameCube emulator, focused only on the one console. But, when the Wii was released and it was discovered to have hardware almost identical to its older sibling, Dolphin naturally evolved into a GameCube and Wii emulator. All of our readers are probably familiar with this. However, many people don't know that there is yet another console based on the GameCube, one which Dolphin has emulated - the Triforce. An arcade system board developed in a joint partnership between the three powerhouses Namco, Sega, and Nintendo, the Triforce used the GameCube hardware as the heart of many arcade games. Mario Kart GP and GP2, F-Zero AX, along with many other titles headline the Triforce's release library.

This month, Dolphin developers have removed Triforce emulation as one of Dolphin's notable features by removing the ability to use the AM-Baseboard, which was the key to activating Dolphin's Triforce features. After months of discussion, it was determined that while Dolphin should be able to emulate Triforce titles, there simply isn't anyone around to maintain and update the Triforce code. It was implemented in a different time and more or less bruteforces the Triforce games into working in Dolphin without much care into how it fits in and interfaces with the rest of the emulator. A branch still exists that is capable of booting many Triforce games for those interested in playing them.

Developers decided to disable the current triforce emulation with the intent of spurring interest of having efforts toward emulating it revived. Working from a crippled base isn't going to help anyone. The other reason for disabling it is that it has little to no relevance for users: no one is even sure if it could boot any of the triforce games in the condition that it was left in for master.

While Triforce emulation has been disabled, there have been a lot of changes improving the emulation of GameCube and Wii games this month. It's that time again, for the month's notable changes!


Notable Changes


5.0-389 - Pixel Engine Token Cleanup and Optimization by degasus

Silent Hill: Shattered Memories comes up as one of the slowest games in Dolphin fairly often. Most slow games either pushed the console incredibly hard, or use some feature that is very difficult to emulate. With Silent Hill, nothing it does is actually all that demanding. While it's a good looking game with high quality shadows and models for the Wii, nothing it does justified just how slow it ran in Dolphin.

While there were quite a few theories as to why it was so slow, it turns out that this game was simply hitting a strange bottleneck in the GPU thread. Occasional Dolphin savior hk.konpie discovered that the game was creating tons of Pixel Engine Tokens (PETokens), which can be queried by the CPU thread at any time. Dolphin was querying them one at a time, resulting in the game hitting the bottleneck. He surmised that if this were optimized that Silent Hill: Shattered Memories would run a little faster.

degasus not only optimized it, but he also cleaned up the ancient PEToken code resulting in a massive speedup for the game. Instead of sending PETokens one at a time, they're batched together to prevent the severe CPU to GPU stalling that greatly affected this game.


petokenperf.svg

That's certainly more than "a little faster"!


There is zero behavior change based upon these optimizations, so no one has to worry about other games that rely on PETokens breaking.


5.0-440 - Vertex Depth Range Emulation by Armada

Everyone hates depth bugs, especially Armada, who has made it his goal to eradicate them all while adding depth whenever he can! This time, he's taken the proverbial hammer to the remaining depth issues.

Previously, Dolphin relied on the API (OpenGL, D3D, etc.) for depth range handling and depth clipping, but there was a fatal flaw in doing this. The GameCube/Wii allowed developers to choose depth ranges much larger than the depth buffer allows, while OpenGL/D3D do not. This was a trick used to forcibly render certain objects even further away at the expense of precision.

Dolphin tried to use a series of clamps to try to mimic this GameCube/Wii behavior and got fairly close. The overabundance of zfighting and depth issues have been culled down to about a dozen games by the time 5.0 released. But when it failed, especially in D3D, it was quite a mess.


Dragon Ball Z

What calamity has befallen us! A giant chasm has opened, and our heroes are missing!

Spider-Man 2

It's the Attack of the 50 Foot Spider-Man!



OpenGL's failures were a little less dramatic, usually resulting in missing shadows and other flat objects sometimes flickering into and out of existence. Armada tried a few more things to get the clamps working but it seemed as though there was no happy balance to be had.

degasus had his own idea to handle this and suggested moving the depth range emulation back into the vertex shader. Dolphin used to do this before, but developers weren't able to wrangle it to work properly. This time some new ideas were applied, Dolphin no longer relies on depth clipping and instead uses its own defined clipping planes. The trade-off to this is that turning off depth clipping requires GL_ARB_depth_clamp, but, any GPU that supports Dolphin has the capability to support it. Even OSX's OpenGL drivers support it!


depthequation.svg

The new Vertex Depth Range shader equation.


Dragon Ball Z

Someone must have called on Shenron.

Spider-Man 2

Oh, Spider-Man was just in the foreground. He must have been pretending to hold the towers up.



Now things still aren't perfect due to rounding errors. We have a few more minor issues remaining, but, with complete control over the clipping planes, it's only a matter of time until a solution is found that fixes all games. Currently, Metroid Prime 1's Visor has zfighting, but fixing that makes the UI for Sonic games disappear. A fix is already in the works that will hopefully not break anything else.


5.0-464 and 5.0-466 - Further Depth Fixes by Armada

You didn't think it'd be that easy, did you? There's always at least one case that mucks up even the best of plans.

The way Dolphin achieves the depth precision it needs is to use reversed depth. It brings the far plane to near for higher precision where it's needed, while computing the near plane in far where the precision is usually not necessary. For Dolphin to emulate the GameCube's depth in current graphics APIs without zfighting, this is a necessity.

The problem is that certain games, such as Dragon Ball Budokai Tenkaichi 2 and 3, had the same idea. They used reversed depth to achieve greater depth precision on console! This means that Dolphin was reversing reverse depth, losing precision and causing issues.

On OpenGL, Dolphin can handle this situation by simply not reversing depth in these cases, but D3D11/12 do not support that. With a Vulkan backend in development, it's worth it to note that while the API allows it, most available drivers currently do not. Hopefully that will change and allow for Vulkan to match OpenGL in depth accuracy in the future.


Dragon Ball Z

The zfighting is minor enough that we had to really zoom in.

Dragon Ball Z

How it should always look.



This isn't the end of the world, though. Unlike the previous case of everything being broken there will instead just be some minor zfighting on D3D11, 12 and eventually Vulkan. Thankfully, this should be limited to titles using their own form of reversed depth.


5.0-461 and 5.0-471 - Debugger Improvements by Aldelaro5

Sometimes debugging needs a little bit of attention too. After all, Dolphin's debugger sees a lot of use for modifying games and making cheatcodes. So when Aldelaro5 ran into various bugs and user interface issues with the debugger, they made it their mission to fix things up. The most noteworthy of the fixes is that modifying memory addresses will now clear the instruction cache, meaning that modified memory addresses will immediately take effect instead of requiring a manual invalidation.

These changes do not effect Action Replay/Gekko Codes, only memory address manipulation through the debugger.


5.0-476 - Fix Load String Word Indexed by phire and Extrems

Credit for debugging this goes to Extrems. Not only did he write not64, the Wii port of Mupen64 that ran into this issue, but, he also helped debug which instruction was failing and then helped figure out why. Dolphin's Load String Word Indexed (lswx) was getting a bit too antsy and clobbering the last register. This is a rare enough instruction that no one has found any games that have been helped by this fix quite yet. Considering the bug is in Dolphin's interpreter, and the JIT/JITIL don't have their own implementation of the instruction, a game that didn't work with any settings may be fixed now!

phire's fixes to the instruction fix the bug and gets not64 to finally boot!



5.0-480 - Allow Texture Samples from 0x0 by Armada651

Hang on tight everyone, this is one of those weird issues where we have to wonder just WHAT the game developers were thinking.

Link's house in Twilight Princess holds a mirror that serves no other purpose other than for developers to show off and say "Hey, look at this fancy mirror I made on the GameCube!" Fancy things usually spell trouble for Dolphin, and this was no exception. The reflection would only show if Link's hand was behind the mirror. If you're anything like us, you'd probably assume that some kind of depth test was inverted causing this strange issue.

And just like us, you'd be wrong.



So, what exactly was causing the mirror to malfunction in Dolphin? Well, firstly, consider that the Software Renderer was already working for some reason, which squarely put the focus on the hardware backends. Armada651 put his debugging skills to the test and delved into the draw calls to figure out what's going on.


Twilight Princess

Is this conclusive proof that Link is a vampire? No, that's stupid, he's not sparkling. Nevermind.

Twilight Princess

The reflection is actually there. Unfortunately most players don't play in wireframe mode to notice this.



The game is using a technique that was common on the GameCube: the "reflection" isn't a reflection at all, but a duplicate of Link's model in an empty room, viewed through the frame of the mirror. This technique is used in many other games and is emulated fine, so why is this mirror failing in Dolphin?

In the case of Twilight Princess, it draws a small rectangle in the depth buffer over the "mirror world" to confine the reflection to the mirror frame's dimensions. This is very normal for this kind of effect. However, the game's developers accidentally turned on alpha testing for that rectangle, specifically alpha testing with values from one pixel of a texture. For the rectangle to pass the alpha test, it needs to read back values from the pixel that are not transparent.

In Dolphin, the game never set a texture to read from, so it was reading from whatever was used as a texture last - which unfortunately was a transparent pixel from the glow texture on Link's lantern. So the entire rectangle was discarded and everything in the mirror was cut away. Thus, Link's reflection would vanish whenever you pulled out the lantern required to see the mirror. The only exception to this was when the lantern goes behind the mirror. Then, it isn't drawn and another texture is used that doesn't have any transparency, explaining the flickering.

So Armada started looking into why a texture was not set. It turns out the game did load a texture to use, but it loaded it from address 0x0.

This is something that would crash a normal computer program, so Dolphin just assumed that 0x0 was invalid. After all, there was never going to be a texture at address 0x0 anyway, because the GameID is stored there. But degasus pointed out that on the GameCube, 0x0 was considered valid memory. And it turns out, the GameID doesn't have any transparent pixels. As insane as it sounds, reading back the GameID as a texture for the mirror's alpha test actually works!



With that, the bug was fixed! But, it felt like an empty victory for everyone involved. Even though Dolphin now displayed the mirror correctly, how could it be confirmed that this was indeed the correct behavior? delroth suggested modifying the GameID of an ISO and then loading it onto a Wii to see if the alpha test failed there. The only issue with this is that using a corrupt GameID caused other complications and difficulties.

Once everything was sorted out, everyone waited with heavy anticipation. Is it possible to make the mirror alpha test fail on console?



Yes, yes it is.

This hardware verification proves that Dolphin is handling the case correctly. Thank you, Nintendo, for one of the weirdest game behaviors we've seen in a very long time.


5.0-492 - Handle Case of Texture Coordinate Q equal 0 by Ornox

Dolphin makes a lot of assumptions that games will act sane and not rely on undefined behavior. Repeatedly, the games betray Dolphin and do insane things. After what happened with Twilight Princess, we shouldn't be surprised that more software is relying on the strangeness of the GameCube and Wii. This one though, traces not to a game but to a DevkitPPC demo; one of the simplest possible homebrew cases was failing in Dolphin: Lesson08.


Lesson08

Dolphin's rendering of Lesson08

Lesson08

Console rendering of Lesson08



Ornox decided to research this a bit further, by which he created hundreds of hardware tests to try to narrow down exactly what was going on. In almost every single case, Dolphin was absolutely fine, until he stumbled upon certain values that would cause the textures to glitch out on Dolphin. A bit more research revealed that Texture Coordinate Q was the source of all of these problems.

So, for context, STQ are texture coordinates. Q is the perspective divide factor, similar to W in XYZW. Since the Wii is right-handed, Q>=0 and W>=0 are undefined cases (the point is behind the camera, or is on the camera plane itself). However, "undefined" in this case means "whatever the hardware does in this undefined edge case" and Dolphin has to handle that accordingly regardless of how insane it is.

Let's just say there's been a lot of thought into how to handle this case; and there are a lot of nuances to it, but now it works to the best of anyone's knowledge. In the most basic case such as the one presenting this visual glitch, when Q = 0 the coordinates are divided by two. It isn't that simple though; as there can be stacked matrices, missing coordinates, and all kinds of scary things to complicate stuff. Some of it is pretty trippy!


trippy-dolphin

This is a super corrupted version of Lesson08 to stress test Dolphin's handling of the Q = 0 case.

trippy-console

And this is the Wii verification of it.



For the longest time, everyone was convinced that this wouldn't actually affect any commercial games. Searching far and wide, a few obscure WiiWare games actually did run into this case!


HB Arcade Disc Golf

The Lesson08 bug has made everything shrunken and offset.

HB Arcade Disc Golf

But now you can torture yourself with this boring WiiWare game as much as you please!



HB Arcade Cards

Even the characters are disinterested!

HB Arcade Cards

See? They are all bored out of their minds.



But, it was a foolish endeavor to look so far and wide for games when just booting up a single Factor 5 game has more test cases than anything else. And verily it came to pass: Star Wars: Rogue Squadron 3 manages to hit the Q = 0 case on a layer of clouds on Battlefield Hoth.


Rogue Squadron 3

Can you spot the difference? Click for full size.


There appear to be a few other games that are affected as well, proving once again that Dolphin should never, ever trust the software it is running to be sane.


Last Month's Contributors...

Special thanks to all of the contributors that incremented Dolphin from 5.0-351 through to 5.0-500!


You can continue the discussion in the forum thread of this article.

Next entry

Previous entry

Similar entries