Dolphin Progress Report: February 2017


As most of you know, Dolphin was born a GameCube emulator. A lot of its core design and concepts are based around assumptions made that it would only be a GameCube emulator. And, as a GameCube emulator, Dolphin performs admirably, with the ability to boot every single title and a large portion of the library having no major issues.

But, Dolphin isn't just a GameCube emulator. One of the more incredible things about its history is that it was modified into a Wii emulator around the time it went open source in 2008. While the core of the Wii is a supercharged GameCube, and things like CPU and GPU emulation were fairly easy to modify into working with only some minor details changing, there are a lot of quirks around it that have been problematic. Not only are there emulation challenges associated with the Wii that Dolphin side-stepped with some dirty hacks, it also struggled to add on all of the new features of the Wii. For many years, the Wii Remote, GameCube controllers configuration, and GameCube controller settings were completely split apart because Dolphin's UI was not designed with more than one primary input method in mind!


dolphincontrollers-old.png

UI Design 101: More menus = better.

dolphincontrollers-fixed.png

Putting all of the options in one place looks better and is easier to use.



In terms of actual emulation, the problems mostly come from the Wii's Starlet ARM coprocessor and everything it brings to the table. To give you an idea of how important Starlet and IOS (Internal Operating System) are on Dolphin, it controls features such as disc access, savegames, networking, USB, ES_Launch (aka, booting games,) and other features necessary for the Wii to function.

Last month, we saw a lot of IOS-HLE improvements, resulting in a big uptick in compatibility. But with these accuracy improvements have come some hiccups and regressions as well. When we fixed The Legend of Zelda: Majora's Mask by using proper, reverse-engineered values for some important IOS-related things, it brought out some regressions too.


Our old IOS-HLE code was more based on guesses than on what the Wii actually did, resulting in a lot of random hard-coded values. When there are hacks on hacks, you can make most things work and this is fine. But, remove any one hack from the chain of hacks, and everything falls apart, which is why the Disc Channel was broken most of the month. On an ES_Launch from the Disc Channel, Dolphin was overwriting critical constants in low mem1. On a real Wii, only certain constants were written on IOS reload, not all of them. Hence, becoming more accurate, we accidentally went too far and were overwriting values when we shouldn't be writing anything at all in those spots!

This is all part of becoming a better Wii emulator. There is going to be some pain, regressions, and breaks in the upcoming months as IOS-HLE is improved, but, the end result will be a better emulator with fewer issues in many problematic titles. We ask users for patience, bug reports, and understanding going forward as we strive to make Wii emulation better from both accuracy and useability standpoints.

As a little taste into some of the fun IOS problems we've run into, here's a game that actually uses IOS quirks as anti-piracy.


Defeating Anti-Piracy with The Smurfs: Dance Party


After that long intro, we hope you can all understand why we're doing what we're doing and hopefully justify some of the recent troubles. With that, let's get into our list of notable changes for February!


Notable Changes


5.0-2302 - Support Setting CPU Overclock via GameINIs by Kurausukun

When making an emulator, we are often faced with "what is best for the user" questions, and they are never really easy to answer... This oft-requested feature was one of those debates. When CPU Clock Override was created, GameINI controls were deliberately not included, out of fear that users would turn on the feature for a game and leave it on forever! Many games malfunction when set at too high of a CPU clock or too low, so, it was merely to try and protect users from themselves.

Newcomer Kurausukun wanted this feature and did all the work, making the decision to merge easier. That, and that times have changed a bit since the original feature was merged. A ton of emulators have Emulated CPU clock override features now, and more users are aware of the dangers involved with using this feature. So, those of you wishing to set an emulated CPU clock per-game can finally do so! This should make life easier for users needing extra performance in super-strenuous titles or wanting to set a higher CPU clock for framerate benefits or even a 60 FPS hack.


5.0-2354 - USB Passthrough Support for Microphones, Wii Speak, and Nintendo Camera by leoetlino

This is the big one: over 100 games made use of USB devices Dolphin didn't support prior to this change. Over 60 of them used a USB device that we didn't support as their primary control method. This USB Passthrough Support update opens up the ability to pass-through devices that use the oh0 (USB1.1) and VEN (USB2.0) interfaces, including the Logitech Microphone, Wii Speak, and Nintendo Camera!

One particular blogger made a mistake of putting out a bounty for this feature many, many years ago:


Whomever implements Wii Microphone support gets to select a song for me to sing. The video will be posted to the blog.
JMC47


Unfortunately, it appears the day has finally come to pay up on this bounty.


JMC47 fails at singing.


But don't fall into the trap of thinking this change just added support for various devices; this was a massive undertaking that cleaned up quite a bit of IOS-HLE as well. It's really hard to state how massive this effort was over the past few months, so let us look at the basics.

At the very fundamental level, this change adds passthrough support for the Wii Speak, Logitech Microphone and Nintendo Camera. It also adds a built-in whitelist to prevent games/Dolphin from grabbing control of devices you don't want them to touch. Games like Guitar Hero 5 would see certain USB devices, try to see if it was an instrument and then promptly crash themselves. The whitelist only applies to what devices games can see. You don't need to use the whitelist for Bluetooth nor GameCube Adapter passthrough since Dolphin itself is interfacing with those devices.

Because this is passthrough support and not support for emulated devices, you need the original device (or another device compatible with the original game on Wii) for the game to be able to use the device. If the Microphone did not work for the game on Wii, it won't work for the game on Dolphin. This change does make it much easier to add emulated devices in the future, so, don't be too surprised if emulated USB devices start showing up in the progress report over the next year or two!


Other Important Changes Within This Commit

To use these USB devices on Windows, we use the UsbDk driver. This is a newer, more updated backend for LibUSB, but, unfortunately it can be a bit buggy at times too. When installed, it completely replaces the zadig guide on the wiki. UsbDk is preferrable when it works, but, because a multitude of users complained of issues (including potential blue-screens on Windows 7!) we've made it optional. If you're using the old WinUSB/LibUSBk drivers from zadig, you will not be able to use these devices. This is not a fault of Dolphin, but, rather lack of isochronous transfer support in those drivers. When the UsbDk driver is installed it will just work; there are no additional steps beyond that aside from the whitelist in Dolphin.

If you're on Linux/Mac, your situation has not changed at all. Linux is also the only operating system to be able to use the Nintendo Camera, as a bug on Windows prevents us from interfacing with it properly.

In terms of actual emulation, the amount of research on USB and how it evolved throughout Dolphin led to a lot of other changes to Dolphin's IOS emulation. Games like The Smurfs: Dance Party, which rely on certain behaviors of certain IOSes, were not emulated in Dolphin before this effort was undertaken. So, while this may just seem like a change to add support for USB devices, it has provided dozens of great changes toward more accurately emulating the Wii's IOS.

Then again... maybe that anti-piracy wasn't meant to keep us out, but to keep Gargamel in...


What lies beyond The Smurfs: Dance Party Anti-Piracy


5.0-2536 - Fix the Initial Content File Descriptor Value by leoetlino

This actual fix is very boring. It's the common "emulator developer in 2008 hardcoded something to a value they shouldn't have," issue we've seen several times by now while working on IOS-HLE. The more interesting part is how this was solved, and that Yummy Yummy Cooking Jam's developers likely hardcoded the same value Dolphin did when making their game. Unfortunately for us, the games are never wrong.

Simple fix: set file descriptor value to something that makes sense. The challenge, though, was finding out where Dolphin was going wrong with this near impossible to debug issue.


Debugging an issue with no leads

Last month, you may remember us going into detail about how we fixed Majora's Mask in Dolphin along with a few other WiiWare titles by debugging on the Wii and finding differences in the boot process. With Yummy Yummy Cooking Jam doing a similar thing to Zelda, we tried the same approach figuring the answer would make itself apparent like last time. Unfortunately, issues with either Gecko.net, or the nature of the problem made it impossible to trace in a similar manner. Booto also joined in to help, but, even with his expertise, no one was able to extract much useful information from the game on hardware.

Instead of giving up, leoetlino did something that many people have said is completely unreasonable: he implemented a hacky IOS-LLE based off of Marcan's old branch that could run Bootmii. This insane branch hooked Dolphin up to an ARM interpreter (skyeye) modified to act as the Starlet Coprocessor.

This did not really work well; Skyeye as Starlet has a ton of issues that make it more or less unusable as Starlet on its own. But, with a few nifty hacks here and there, leoetlino was able to get it working enough to actually boot Yummy Yummy Cooking Jam!


yycj.png

Behold, the final WiiWare game to launch in Dolphin in glorious standard definition graphics since it's all 2D.


His IOS-LLE branch was very limited in what it could do, but, it was enough to confirm that this game's crash is 100% caused by IOS-HLE. At this point, all he had to do was compare IOS-LLE and IOS-HLE and see where they returned different results. This led him to the File Descriptor initial value, which was 0x0 on IOS-LLE, and 0x60000000 on IOS-HLE. Changing that allowed Yummy Yummy Cooking Jam to finally boot normally in Dolphin. According to our database, with this fix, every single (tested) WiiWare title can now at least boot in Dolphin!

Quick Note: IOS-LLE in its current incarnation will never be merged to master. It's for testing purposes only. It's too hacky to be useful to users, and too hard to configure to be useful for most testers.


5.0-2567 - Fix Read Handler Logic by leoetlino

Another bug unearthed by the IOS-LLE branch was in Mushroom Men: The Spore War. This little known game was broken way back in time, not in 5.0, not in 4.0, not in 3.5, but in the early 3.0 era of dev builds! Whenever a fix for another game (in this case, PokePark 1 and 2,) breaks other titles, those are particularly tricky to fix, as fixing the regression is likely to break the other titles.

Thankfully, with IOS-LLE already running the game correctly, all leoetlino had to do was figure out where IOS-HLE was going wrong.

The game creates a save file. It then reads 2560 bytes from it, followed by immediately writing 16384 bytes to it. With IOS, the first read does not change the seek position at all, so the save data is written at offset 0, not 2560. With Dolphin, the read erroneously set the seek position to 2560, which caused the data to be written at the wrong location. Because it loaded a savefile with no weapons on the next level, the game was effectively softlocked, and totally uncompleteable.


shroompeople-broken.png

This cutscene seems to be missing something...

shroompeople-working.png

Right, the shroom was missing his warhammer! ...made of a stick, spring, and bubblegum. This game is an odd one.



5.0-2413 - Preliminary Wii Save Support on Netplay by RisingFog

Wii Netplay in Dolphin is one of those features that have bounced around quite a bit. While it has great potential, Dolphin's netplay core was designed for GameCube Netplay and it really shows with just how difficult it is to use Wii games on netplay. But now, users get a little bit a boost in useability with this latest change: you can now use saves and save in Wii Netplay!

One of the most stringest requirements of Wii Netplay in the early 4.0 era was that you needed identical NANDs for Wii Netplay to sometimes work. We decided that wasn't realistic and changed Dolphin to use a blank NAND long before the 5.0 release. This meant that users wouldn't have to do anything; Dolphin would use an identical NAND no matter what!

This change takes it a step further. While we're still using the Blank NAND, Dolphin will check for a savefile for the GameID you're loading on netplay on boot. If it sees one in your NAND, it will copy it to the Blank NAND! That way, even if your NANDs are completely different, as long as both users have the same savefile, they can play together without issues. When the netplay session ends, Dolphin will then save the game back to your Wii NAND. This allows for longform games like Tales of Symphonia: Dawn of the New World and Dokapon Kingdom to be played on Dolphin!

Note that Dolphin currently does not handle any save transferring for Wii games; users are required to make sure each player has the savefile they want within their NAND (aka, whatever NAND Dolphin is using when you start netplay, if you have multiple Wii NANDs.)


Wii Netplay with Saves


We know a large sect of our users are more interested in Super Smash Bros. Brawl, and use cheats instead of a savefile. For those users, the default behavior has not changed. These new features are all optional, and can be turned on or off within the host's netplay window before the game is booted.


Warning

Games that require Wii Remotes are still incredibly difficult to set up. In the intro, we mentioned parts of Dolphin designed for GameCube, and netplay was one of them. As such, the entire ability to use Wii Remotes on netplay is a hack with a bunch of duct-tape and glue holding it together. It's incredibly difficult to set up, even for senior developers on the project. If you absolutely must use Wii Remote Netplay, you may want to use a separate portable install of Dolphin. Unlike GameCube Controllers, Player 1 configures Wii Remote 1 for themselves and Player 2 configures Wii Remote 2 for themselves. You must have the same number of Wii Remotes set to active. You must set values such as Battery to be the same on all Wii Remotes across all PCs. If the Wii Remotes disconnect at any point, you can crash the emulator.

As shown in the video above, it is possible to get emulated Wii Remotes working on netplay. And once you've figured out every single quirk, it does work flawlessly, so it's more or less a matter of making setup easier.


5.0-2449 - MIOS (Wii GameCube Mode) Support in Dolphin by leoetlino

Though the Wii is more or less an overclocked GameCube, it has a lot of differences that need to be accounted for before it can safely run GameCube games. MIOS is a special IOS that contains a modified GameCube IPL; it more or less disables all of the Wii's new features, changes the CPU/GPU clocks back to where they would be on a GameCube and then launches the GameCube game. It also patches certain games, like The Wind Waker, to not crash on Wii.

Now, before you say this is useless, it's actually not! Unlike the GameCube IPL, which is a pain in the butt to get, you can download MIOS from Nintendo legally currently using NUSDownloader or grab it off of almost any Wii disc! Why is MIOS required? MIOS support in Dolphin is actually an HLE/LLE hybrid! While the Starlet part of MIOS is HLE'd, we LLE the PowerPC parts, such as running the GameCube IPL and the patches it applies to various GameCube games that don't run on Wii properly.


Simple MIOS Demonstration


You can retrieve MIOS legally from almost any Wii disc with an update partition or download it directly from NUS via NUSDownloader. If you put the MIOS WAD into the gamelist, you can then right click it and install it to your NAND. From that point on, you'll be able to load GameCube titles from the system menu on that NAND.

For (closer to) full functionality in the System Menu, we recommend you rip your full NAND off of your Wii and follow the guide on our wiki.


Patched GameCube Titles and Other Quirks

Dolphin supports the patched GameCube titles, which include The Wind Waker, Smuggler's Run: Warzones, Tony Hawk's Pro Skater 3, Zelda: Collector's Edition, Pokemon Colosseum, Phantasy Star Online Episode I & II and likely others. These games do not behave any differently than when booted normally, though Dolphin does successfully apply the patches.

Dolphin also cannot currently load the default settings from the games when they are booted like this, so be sure to have Dolphin configured correctly before using this feature.


5.0-2431 - DVD Timing Accuracy Upgrades by mmastrac and JosJuice

This would normally be a headlining change of a progress report. This and another addition in 5.0-2535 brought unprecedented DVD timing accuracy to Dolphin. It's the most accurate model to a disc-drive we've seen in any emulator to date and times almost everything you could imagine to a high degree of accuracy.

A big part of this change is DVD chunking; it's what fixed the last problematic games, including Sonic Riders, Arc Rise Fantasia, and the Scooby Doo GameCube titles. Before, if a game read a large part of the disc, Dolphin would estimate how long the action would take on console before saying it was complete and then copy everything all at once. This wasn't good enough for these titles, and they crashed no matter how close the load-time was to console. Chunking changes it so that we now execute and copy the reads in 32KB "chunks" just like on console, which fixes the issues in those titles.

Another big part of this change is that great care has been put into making the timings more accurate for everything. While this doesn't fix anything in particular, you should notice the load times matching up much closer to console. In some cases, it's faster! In F1-2002, the initial load time drops by over a minute in Dolphin from ~120 seconds before to just over 44 seconds now, which matches up perfectly to a Wii console.

As with any big change like this, there is a potential for regressions. Unfortunately, it seems Tool Assisted Speedruns are broken with this change because of determinism issues with savestates. This is especially problematic for us, as one of the ways we get our timing changes tested is by having the Metroid Prime speedrunners find all the differences in Dolphin's timings.


Last Month's Contributors...

Special thanks to all of the contributors that incremented Dolphin from 5.0-2237 through to 5.0-2684!


Next entry

Previous entry

Similar entries