While the Google I/O 2024 keynote went pretty heavy on AI, we’re still expecting a bunch more announcements from the rest of the conference. And one of the things Google has already announced is “Project Gameface”, an open-source feature that lets you control Android’s “gaming mouse” using your face.
At the moment Project Gameface is an accessibility feature that’s only available for developers. So it’s entirely up to them whether the feature is actually integrated into mobile games on Android. But just because this is accessibility-focused doesn’t mean that everyone couldn't benefit from a feature like this.
I’ve always found smartphone gaming very frustrating, because it doesn’t mix well with a lot of traditional games that require pressing buttons. I’ve been trying to replay classic Sonic the Hedgehog games recently, and while readily available on Android I’m finding them very difficult to control without the use of a controller. If facial expressions can add an extra control option, then I am all for it.
The idea behind Project Gameface is that it utilizes the device’s camera and a set of facial expressions sourced from MediaPipe’s Face Landmarks Detection API. Those expressions are then used to control the “cursor” (or what passes for it on Android touchscreens) by translating your facial and head movements into actual commands.
Developers will also be able to configure and customize how those commands actually play out, noting the ability to customize expressions, gesture size, cursor speed and more. The feature won’t be exclusive for games either, with Google partnering with Incluzza to offer Project Gameface in other places — like work, school and social situations.
Hopefully developers will take this seriously, and offer this kind of gesture control in their apps. The last thing we want is for Project Gameface to end up ignoring and neglecting Google’s Soli gesture controls that were exclusive to the Pixel 4. Though since Gameface doesn’t require any specific hardware, and uses open source software in its implementation, that shouldn’t happen.
