It seemed like a mostly harmless game, whose main merit, as has often been joked, was convincing even the most hardcore nerds to go outside. And yet, Pokémon Go – the video game launched by Niantic in 2016, which, through augmented reality, allows players to find digital versions of Pikachu and other creatures in the physical world by scanning with a smartphone camera – has proven to be a crucial innovation in many respects.

First and foremost, Pokémon Go demonstrated the potential of augmented reality, a technology that overlays digital elements onto the physical world, at a time when few were ready to bet on it. The game offered a preview of how video games might evolve, especially as AR headsets – currently experiencing a new commercial push after years of failure (such as the Google Glass and Magic Leap One) – become mainstream.

The success of Pokémon Go, whose capabilities are obviously much greater when played through AR headsets rather than a smartphone screen, has in many ways inverted the usual technological cycle, creating a killer app for an innovation not yet ready for mass adoption (typically, the opposite happens). More than that, Pokémon Go has also revealed the next steps of surveillance capitalism, bringing extremely significant (and unsettling) innovations to this controversial field. 

When the Pokémon Go craze exploded, millions of people wandered everywhere – including cemeteries and police stations – to capture the game's creatures. Among the various locations flooded by players were also specific stores, such as McDonald's and Starbucks, where Pokémon suddenly appeared. They were not there by chance: Niantic had formed partnerships with several businesses, charging about 50 cents per player directed to their locations through Pokémon Go.

A year later, the company claimed to have sent approximately 500 million visitors to various "sponsored locations." Millions of people worldwide were thus incentivized to eat burgers, drink coffee, or perhaps buy clothes and accessories through a video game that led users precisely where businesses wanted them. Nothing illegal. And yet, such power to guide people's behaviors raises some ethical concerns: to what extent is it morally acceptable that our choices are so heavily influenced?

This ability to use gameplay mechanics to steer the masses has not gone unnoticed by investors. It may also be one of the reasons behind the sale of Niantic's gaming division – which, in addition to Pokémon Go, includes titles like Campfire and Wayfarer – to Scopely, a company controlled by Saudi Arabia's sovereign wealth fund, for $3.5 billion.

This ownership change, also driven by Niantic's financial struggles following the pandemic, raises new questions about the management of Pokémon Go's collected data and the future direction of the game, now under the control of new actors with different economic strategies and accountability to a state that does not guarantee democratic protections.

With the sale of its gaming division, Niantic will instead focus exclusively on developing geospatial artificial intelligence technologies through a new company called Niantic Spatial. Beyond the future role of the Saudi company, it is well known – as revealed in a Niantic press release – that the California-based software house continues to leverage data collected from its millions of players to train an AI model capable of navigating the world.

It is not new that humans, often unknowingly, play a crucial role in training artificial intelligences. Every time we identify objects in a Captcha test before accessing a website, we are helping AI improve its ability to recognize objects, a process known as image recognition. In deep learning, machines are trained with vast amounts of labeled images, and through repeated analysis, algorithms learn to recognize objects independently.

Similarly, hashtags on social networks serve as "labels" for posted images and are then used to train algorithms to understand the contents of associated images (e.g., hashtags like #sunset or #beach help train AI to distinguish sunsets from beaches). The same applies to voice interactions with virtual assistants (such as Alexa or Siri), which are recorded and used to improve speech recognition and natural language understanding models. Even robotic vacuum cleaners like Roomba rely on user interactions to learn how to navigate autonomously. 

Pokémon Go follows this pattern, utilizing the participation of millions of players to build a geospatial artificial intelligence model. Called the Large Geospatial Model (LGM, a clear reference to Large Language Models like ChatGPT), this system aims to do for the physical world what LLMs do for language, enabling computers – as stated in Niantic's press release – "not only to perceive and understand physical space but also to interact with it in new ways, becoming a critical component of AR headsets and other industries, including robotics, content creation, and autonomous systems. As we move from smartphones to wearable technology connected to the real world, spatial intelligence will become the operating system of the future."

In short, by exploiting the images of the world produced by the tens of millions of people who play Pokémon Go every month, the model is able to predict the shape of the surrounding environment in the same way that an LLM predicts which word is most likely to be consistent with the ones that came before it.

So Niantic didn’t just create a database of millions of geolocated images, but also trained an artificial intelligence to predict – through the same images collected by players – what the environment that is still absent in the collected images could be like.

Let’s take an example: having collected millions of images of railway stations all over the world, LGM could be able to predict the shape of a specific station even without having photographs taken from every perspective. This is because, even if they are all different from each other, railway stations always have common characteristics, which can help the software to predict – with a margin of error – what the complete shape of the station in question will be, and consequently help, for example, a robot to navigate it autonomously and without errors.

Obviously, the aim is not to create robots capable of competing with Pokémon Go players, but rather to sell its database of geolocated images to interested companies, together with the artificial intelligence model capable of ‘connecting the dots’ (so to speak) between one image and another, creating a navigable map of the world.

It’s a step up from Google Maps and similar systems (note that the person in charge of this Niantic system has worked in the past on the development of Google Maps, Google Earth and more), because it allows you to have a digital map of the world created from the perspective of pedestrians (instead of that of cars or satellites) and potentially also of the interior of a place.

Who could benefit from a system of this kind? First of all, us humans: if one day we really do move around wearing augmented reality visors, being able to draw on a map of the world created from a pedestrian’s point of view and exploit artificial intelligence capable of predicting the shape of the environment we are heading towards, it could have undeniable commercial potential (allowing us, for example, to find our way around a railway station, obtaining digital road directions superimposed directly on the physical environment).

But our thoughts inevitably turn to the various home delivery robots already in use in some cities in the US, China and elsewhere, which mainly move along pavements and would therefore benefit enormously from the system designed by Niantic

Are there other robots that could benefit from this geospatial AI system? Absolutely. Niantic's LGM could be incredibly useful for robots (and more) that need to navigate unfamiliar environments and quickly understand their surroundings. In other words, Niantic may have developed an extremely functional system for the world of autonomous weapons: drones, robots, and various types of vehicles that need to navigate autonomously in hostile territory.

When questioned about this issue during the Bellingfest investigative journalism festival, Niantic’s engineering head Brian McClendon admitted to "fully understanding" the military potential of the system developed using Pokémon Go data. "I think the real question is whether something is being done with it that users wouldn’t want. And clearly, if this system is used specifically at a military level to further expand war zones, then it is definitely a problem."

Notably, Niantic has not ruled out the use of its software – still far from commercial release – for military applications, stating only that, if it happens, it would be a problem to address. Perhaps, though, it should be addressed before former Pokémon Go players discover they unwittingly helped train software for warfare.

Opening image: Pokémon Go