For years now, the TASBot team has shown time and again that tool-assisted speedruns—which can feature superhuman input speeds powered by frame-by-frame emulator recordings—can actually work on unmodified console hardware. Thus far, though, TASBot's efforts have focused on defunct retro consoles from the Atari 2600 up through the Gamecube and Nintendo DS.

This weekend, TASBot will finally take its talents into the modern gaming era, showing off expert-level Super Mario Maker 2 gameplay on an actual Switch during the livestreamed Awesome Games Done Quick speedrunning marathon. And this time, the TASBot team is taking pains to make sure no one else can copy its method—to hopefully avoid Nintendo's potential legal ire in the process.

Flipping the Switch

The effort to let a Linux computer take external control of a Switch game began a bit inadvertently back in 2018, when the TASBot team attempted to partner with the AbleGamers charity. Their goal was to create an Arduino interface that would allow inputs (and pre-recorded input macros) from any controller to be re-mapped into input signals for any console interface.

While that AbleGamers effort eventually fizzled out, it did lead to a generalized Linux-to-Switch controller interface that was published on GitHub. At the same time, other efforts like CommunityController's "Twitch plays Nintendo Switch" were using similar concepts to let a Twitch chat room take control of live Switch gameplay (a la 2014's "Twitch Plays Pokemon" phenomenon).

While these kinds of efforts were fun for random tinkering, they utterly lacked the frame-perfect precision necessary for a successful replay of a pre-recorded, tool-assisted speedrun. "We saw massive inconsistencies," TASBot maintainer Allan "dwangoAC" Cecil told Ars about TASBot testing on the Switch in 2018. "Replay device precision was impossible... TASBot is a player piano—he's playing back a predefined sequence of button presses—but if he doesn't know when to send those button presses, it'll never work."

By 2019, multiple TASBot team members were working in parallel to try to solve this seemingly intractable Switch timing problem. One branch of effort even tried to insert a "shim layer" onto a hacked Switch console to force the external input timing to line up with the in-game timing, but "we didn't get far because it's against our ethos to modify the console," Cecil said.

At the same time, TASBot team member KNfLrPn was "using the semi-working system to help test [Super Mario Maker 2] tech for other [efforts]," they recently told Ars. "So while doing that I kept trying different things just in case, and eventually found a combination of multiple pieces that worked together [to fix the timing problem]."

Prior to that first successful test in December, there was "about five months on-and-off of trying different approaches, different code, different hardware," KNfLrPn added. "Until it worked, we had no idea if it was possible (and actually suspected that it wasn't)."

Approaching the starting line

Though TASBot has taken the first step to breaking open robotic Switch play, its method still isn't perfect. For one, Cecil says the hardware still isn't precise enough for games that require analog input.

In testing on Breath of the Wild, for instance, the team tried recording a simple input macro of Link jumping off a tower. But Cecil said slight, frame-level differences between the Linux recording and the controller polling rate during playback led to butterfly effect-style chaos, such that "loading the same savestate and playing [the input] back would result in us landing in a different spot, sometimes substantially so." Using digital inputs on a more deterministic game like Super Mario Maker 2 eliminates those problems, Cecil added.

Playing on the Switch also means the TASBot team doesn't have the benefit of recording its inputs on robust, TAS-configured emulators, which allow for easy pausing, editing, and re-recording of frame-perfect input sequences that can create literally superhuman performance. On the Switch, thus far "there aren't any tools to make this fast," Cecil said. "This was done laboriously by hand and isn't easy to replicate."

For this weekend's AGDQ demonstration, KNfLrPn specifically designed a level to take these limitations into account; for each level section, they "include[d] a spot where I could get a consistent starting point (You might see in each part there's some kind of wall I could press against)."

From those safe spots, KNfLrPn said they could "start with a guess on which buttons to press for how long, try it, see what happens, adjust, and iterate over and over" until they reached the next safe spot. By playing a string of successfully recorded sections back from the start, KNfLrPn could then get back to any safe spot to continue the trial-and-error process.

Without the use of emulator tools, recording a few successful minutes of Switch gameplay took "several hours of trial and error, resetting each section and trying something slightly different each time," KNfLrPn said. "It was also 'only' several hours because I specifically designed each section to be easy to reset. Doing it with a 'real' level would be even more tedious."