OpenAI, Strong AI Training Environments, and Cloud Robotics


The non-profit artificial intelligence (AI) research company, OpenAI, has released a software environment called Universe, which is a toolkit for training AI agents in different environments. The environments largely consist of video games, but can also include browser tasks or software. Within these environments, agents can learn how to tackle problems and understand the task at hand, with the possibility of transferring the learnings to other environments and eventually into the real world. This is done using reinforcement learning (RL), which is a deep learning technique that reinforces positive outcomes and penalizes negative outcomes. For example, if an agent wants to learn how to drive a car, it can access the Grand Theft Auto gaming environment, and by simply understanding the pixels in the game and controlling a virtual keyboard and mouse, it is able to figure out the positive and negative outcomes, learning how to drive a car in the end.

While training AI agents in gaming environments is nothing new, the advantage of Universe is that it simplifies this task by packaging multiple environments as remote runtimes and doing it with minimal latency across the public Internet. DeepMind also announced the opening up of its training environment, but, in my opinion, Universe beats DeepMind fair and square. Universe also allows agents to run on multiple environments simultaneously, which speeds up training. AI developers can focus on developing better AI agents, which can then be trained on multiple environments. OpenAI also plans to provide transfer learning benchmarks so that developers can check how good their AI agents are.

The Race to Develop Artificial General Intelligence

The next big leap for tackling artificial general intelligence (AGI) will require AI algorithms (or agents) to train and operate across multiple dynamic environments, rather than be constrained by proprietary datasets. AGI, also known as strong AI, refers to AI that can operate across different domains, rather than be limited to a single domain. Essentially, AI needs to follow and understand the real world, and different views of the world, rather than being simply trained on proprietary static datasets like images, voice, or consumer data. Google, Amazon, Facebook, Microsoft, and Baidu continue to dominate AI because of their access to very large proprietary static datasets. But this could change with the race toward developing AGI, with dynamic open-source real world or simulated environments leveling the playing field.

Currently, the best understood environment is games, but Universe wants to open this up to include anything that could be ported into software. For example, Universe allows agents to perform browser tasks by simply following pixels and controlling the mouse and keyboard. This could include anything, such as booking a flight on a travel site or completing a job application on a recruitment portal. Even enterprise tasks, such as building or parsing Excel models, writing emails, or learning how to use software packages could be a training environment. There will likely be proprietary enterprise versions of Universe that will start to emerge to handle AI training in sandboxes. is one such startup focused on developing enterprise-grade AI agent training for specific industry verticals.

The possibilities are endless and any task that you perform on a computer screen could become an environment that an AI agent can tackle. There are some major challenges around defining reward outcomes if they are not obvious on the screen itself (like a game score). By releasing Universe, OpenAI hopes the AI developer and AI research community will solve this problem collectively. In the end, OpenAI wants to build a singular agent that can operate across multiple environments (or types of environments), essentially giving rise to AGI.


(Source: OpenAI)

Employing Universe in Cloud Robotics

The release of Universe also has major consequences and could be beneficial for the adjacent area of cloud robotics. One of the goals of OpenAI, apart from developing an AGI for games, is to develop a robot that can perform household chores. Robots today are very expensive to train and develop, because the testing burden is very costly as individual robots are trained and tested on individual datasets. Cloud robotics can bring down this testing burden by allowing robots to train on multiple datasets or environments simultaneously. By integrating or even building gaming environments for household robots and then training AI agents on those tasks, one could end up with a household robot that uses a singular AGI agent for household chores.

But, a much more powerful future extension for Universe could be to integrate video feeds of people performing household chores, with wearable cameras (or Snap Spectacles) for example, and using that to build thousands or millions of training environments. The same principle could be applied to self-driving cars, integrating dashcam video feeds, or first-person video feeds from a construction worker or crane operator. Using cloud robotics, we can then expect to see robots that have strong AI agents built into household robots, construction robots, or self-driving cars, which can then be tested and deployed in the field.

Watching Future Scenarios

Universe is the first open-source scalable tool for training AI in multiple environments (rather than proprietary datasets), with transfer learning as an underlying feature set to accelerate learning. It would be very surprising if Universe were not extended to OpenAI’s household robot goal. This signals the start of the race toward realizing Strong AI and the emergence of smart robots. It will be interesting to see how long Tractica’s forecasts concerning narrow AI versus strong AI will stand, given how some of these scenarios may play out in the end.

Comments are closed.