Technology Flow View: Decrypting AR Game Pokemon Go

Note: The author Jiang Jiayi, Ph.D., Department of Mathematics, Peking University, is currently a SLAM algorithm engineer at Liangfengtai. Obsessed with the game for 20 years, it has all the game consoles from the era of red and white.

After the release of the AR game Pokemon Go, it brought countless serial effects, crazy fans, awkward servers, a crashing police station, and Nintendo's soaring share price. Since then AR games and the AR technology behind it have begun to go to the general public.

The reason why this game can make the world crazy, in addition to Pokemon's own top IP gene, the addition of AR elements is the most important reason, AR makes Pokemon Go reborn.

AR game, a maverick game

What is AR? The global Pokemon Go players know about AR, maybe not even one in a thousand, but what about it, they have already got a wonderful experience in this magical world.

Such AR games and previous mobile games are completely out of a time and space. John Hanke, CEO and founder of Pianmon Go R&D company Niantic Labs, said: "This is a new experience, mixing game play with other things you do."

The link is real, this is the charm of AR. Whoever stipulates the game must be able to quickly operate the mouse and keyboard in front of the computer, or hold the phone to poke it, it can be another attitude. When I got home, I lit a lamp. On the floor, I was a Bikachu. I walked in the park. There was a wave of Bocobi who could sing. Maybe I could go home and go to the duck when I was in the supermarket.

With the addition of AR, the elves can be brought into real life, and Pokemon Go is essentially different from other games. The long wait for patience has finally changed into tears.

Seller show and buyer show, AR does not stop there

I am in the AR line, so I started to pay attention to this game long after Google’s April Fools’ joke video came out. In the video, the player climbed over the mountains and finally found a small fire dragon in a cave. With a "Gocha!" he showed a happy smile. This is a joke, but Nintendo contacted Niantic to make this joke a reality.

From Google’s video to Niantic’s official promotional video, as an AR technology faction, I previously thought that this AR game would at least achieve the following effects:

1. The elf must be integrated into the real world to show that it does exist in the real world. This requires the reconstruction and recognition of the three-dimensional space, so that the elves are generated in a natural plane rather than some strange places.

2. The elves that the player encounters will have some connection with the surrounding environment. For example, the water in the lake will see the elves in the jungle instead of the grass. This not only requires the mapping service based on the LBS class, but also uses the AR technology to identify the surrounding environment to achieve the fusion of the details of the picture.

3. The phone screen has become a window in the world of players and elves. This requires precise positioning of the player's mobile phone, allowing the player to watch the elf at close range, and even interact based on the positioning, such as the proximity will let the elf escape.

However, in terms of the current demo experience, it is inevitable that there will be a difference between the seller show and the buyer show. If it is a perfect AR game, what kind of technical support will be needed? Simply analyze it:

Scene structure cognition

AR is the abbreviation of Augmented Reality, augmented reality. Since we want to augment the reality, we must first understand the reality. The perception of reality is the most important part of AR technology. The algorithms that understand the structure of real scenes are distinguished by offline and online, which are called SFM (3D reconstruction) and SLAM (map construction and real-time positioning) algorithms. The SFM class algorithm has become a standard algorithm in the construction of 3D maps, and the SLAM class algorithm has been used in the field of robots before, and it has only begun to play a role in the AR field in recent years.

Pokemon Go is a real-time combat game, so SLAM technology is the key and first choice. SLAM technology can use a variety of different sensors to recognize the scene of the player in real time, construct a corresponding 3D point cloud according to the real scene, and then analyze the 3D point cloud to understand the geometric structure of the real scene, such as where the ground is. The desktop, and where is the wall. Once you understand this information, you can generate the correct virtual model in the right place to interact with the virtual model in a way that matches the rules of the real scene.

Unfortunately, perhaps it is limited by the development time, or Nintendo wants to test the water AR first. The current version of Pokemon Go can be said to use less SLAM related technology, so you can often see the elf generated in a non-planar place. It will even float in the air and it looks very illegal. If you use scene recognition technology, the game is more realistic. When the player sees the elf jumping from the ground to the table and jumping to his bed, it will be very shocking.

In addition, the game needs to interact with the reality, but the player does not necessarily understand the algorithm generated by the sprite. If the sprite is generated on the edge of some high platforms, the player may risk to follow and cause unnecessary tragedy.

2. Semantic-based environment recognition

Different from the cognition of scenes that emphasize geometric structure, semantic-based environment recognition is more inclined to understand the actual scene in a human way. For example, to identify whether the scene is located in the city or in the wild, whether there is a lawn or a pond in the scene.

With the current hot learning algorithm, the above recognition functions have been gradually realized, and even beyond the cognitive accuracy of human beings.

Through the picture information provided by the player's camera, the image recognition algorithm can identify the scene of the player and the natural landscape around the scene, so that the player can capture the wonderful frog seeds in the grass and find the Jenny turtle in the water.

Semantic-based environment recognition can easily connect the properties of the elf with the surrounding environment of the player meaningfully. Different scenes correspond to different story scripts, instead of relying solely on LBS technology to randomly and disorderly in the map. Occasionally encounter different elves.

3. Display and positioning technology

Players who have experienced Pokemon Go should have discovered that when you want to "close" with your elf and take a close look at these cute things, the elves still maintain a "distance" with the player.

This is because Pokemon Go uses only the gyroscope that comes with the phone to make a very rough positioning of the player. Therefore, when the player moves forward, he can't experience the effect of the "magnification" of the elf. The elf on the table stood in the air.

Elf floating in the air

In order to achieve a stronger sense of substitution, it is necessary to strengthen the display and positioning technology of the real scene. First, we need to understand the structure of the scene. Only by understanding the structure of the real scene, it is possible to superimpose more virtual models in the real scene.

For example, in the challenge of the pavilion, if you add the algorithm of scene structure cognition, the player will see his house become a pavilion through the screen of the mobile phone, that is, the model of the virtual pavilion is superimposed on the real-world housing structure. The feeling is obviously better.

HoloLens turned the room into Minecraft

Secondly, the player wants to move around in such a fascinating scene and rely on precise positioning technology. Why do you have to position it?

This is because the position of the virtual scene is calculated based on the estimated position of the player relative to the real world and then projected to the real world. The ideal effect is that the player feels that the virtual model is fixed in the real world when moving, and becomes a part of the real world. .

So if you can't accurately estimate the player's position, the player will see the virtual model or the scene deviate after moving, resulting in a sense of "floating".

On the contrary, using precise positioning technology, the player can observe the details of all virtual objects at close range. For example, I have always wanted to get close to see the cute Pikachu. The current mainstream positioning technologies include the SLAM class algorithm and the VO class (visual odometer) algorithm.

4. Voice and gesture interaction

The elf capture process I once imagined is like this: throw out a fairy ball and shout "It's you, Pikachu!" (Good second, shame...) Then the captured Pikachu will be mad. Fully appearing on the screen - it's a god-like experience. It's a pity that such a moving picture can't be reproduced in Pokemon Go's game right now. At present, as the most important interaction module in AR technology, voice and gesture interaction have reached a relatively mature stage, and it is not difficult to achieve the above effects. With the addition of voice recognition, players can call their favorite elves and direct them through voice interactions, just like a real trainer. On the other hand, if you use the gesture recognition instead of the finger swipe to operate the sprite ball, the pleasure of releasing and conquering the sprite will also increase.

If you have glasses, will the experience be better?

Of course, the above analysis and imagination are definitely not to show that Pokemon Go is not good enough, but it just proves that when technology and games develop together, the experience of this game will have great optimization space.

In addition, supporting equipment is another way to optimize. When planning the game, Pokemon Go also thought of the limitations of the phone, so it also launched a bracelet. The bracelet will vibrate when the elf appears, eliminating the need for the player to hold the phone.

But the bracelet + phone may not be the best way. As can be seen from the above analysis, the scene that I originally wanted to see with my own eyes must be visible through the screen of the mobile phone. The interactive mode can only be limited to the screen touch. In the mobile era, the mobile phone can become a player and An invisible wall between the elves. But with glasses, the lack of experience can be compensated.

AR glasses give players the possibility to walk into the world of elves. The camera and various sensors can sense the environment and quickly remind players of the presence of target elves. The optical presentation of AR glasses not only widens the screen of the mobile phone by dozens of times, but also further deepens the integration of the virtual elf in the real world. As if they are really around us; AR glasses also liberate their hands to the maximum, running and jumping are not a problem.

Pokemon Go is a dream of a generation. The childhood sustenance has ran out of televisions and game consoles and become companions in life. As an AR industry person, I will have an idea in my heart. I hope that I can have the ability to make this companion better and better. I also hope that in addition to Pokemon Go, there will be such a good game in China.

Coffee Table

Coffee Table Sets,Modern Coffee Table,Cheap Coffee Tables,Garden Coffee Table

Ningbo Autrends International Trade Co., Ltd. , https://www.homefavorities.com