Engineers taught a neural network to play Minecraft on video – she created a diamond pickaxe and beat a cow

In general, she behaved like the most ordinary player.

About his experiment with neural networks and Minecraft told OpenAI specialists, including those known for the tool DALL-E 2. In order to train the algorithms to play the sandbox, the engineers used a pre-training technique (Video
PreTraining).

First, experts collected videos about Minecraft with a total duration of two thousand hours and marked the actions of the players in them – keystrokes and mouse movements. Based on this data, the engineers trained a model that predicts the action performed at each step of the video. This system marked a much larger amount of data – videos about Minecraft with a total duration of 70,000 hours.

Based on the received labeled data array, the specialists trained the behavioral cloning model to perform various tasks in Minecraft. The created neural network was able to cut down trees, create a workbench, craft items and perform other actions, including hunting animals and eating food. In addition, the algorithms have mastered the mechanics of “pole jump” – this is when the player repeatedly jumps and immediately places a block under him.

The specialists set a difficult task for the neural network – to find diamonds and create a pickaxe out of them. In order to achieve this, the engineers additionally trained the algorithms on the required sequence of more than 20,000 actions.

The neural network finds diamonds and creates a pickaxe

As part of the experiment, the neural network used the standard Minecraft interface, simulating mouse and keyboard controls, emphasized in OpenAI. According to the company’s employees, the experience in Minecraft has shown that the pre-video training method is suitable for many other areas in which the neural network can learn to act after “watching” the videos.

The neural network plays Minecraft

See also  Video – Discovery of a giant bacterium visible to the naked eye

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.