Reinforcement Learning has been widely used by machine learning researchers to achieve the purpose of artificial general intelligence and is one of the hottest topics in ML. RL has been applied in several ground-breaking advancements – beating StarCraft champions to defeat the world champion in the ancient Chinese game of Go, checkers, and DOTA among others.
Reinforcement Learning technique helps an agent to interact and learn in an environment from its actions and experiences. This technique is different in the learning aspect from the supervised as well as an unsupervised learning technique. It has been applied in various applications, for instance, in online stock trading, text summarization engines, dialogue agents, among others.
In order to solve a specific challenge or a problem, a researcher needs to understand the system better which includes several measures like scalability, the interactive environment and the key issues prevailing in the AGI system. However, many times, due to the limitations of RL algorithms, researchers find it difficult to implement this algorithm in applications such as in autonomous cars and other important applications which are interconnected with the risk of human lives.
Now, researchers from DeepMind introduced the Behaviour Suite for Reinforcement Learning or bsuite which is the collection of experiments designed to highlight key aspects of RL agent scalability. bsuite, which is completely written in Python, aims to provide a bridge between theory and practice of RL algorithms which will benefit both the sides.
Each experiment in bsuite mainly consisted of three parts as mentioned below
- Environments: a fixed set of environments determined by some parameters.
- Interaction: a fixed regime of agent/environment interaction (e.g. 100 episodes).
- Analysis: a fixed procedure that maps agent behaviour to results and plots.
The analysis part of bsuite includes “score” which maps the performance of the RL agent on the task to [0, 1] and allows to visualise how the RL agents are behaving in a specific environment. All the experiments in bsuite have the capability to only measure the behavioural aspects of RL agents which means that they can only measure the properties which can be observed in the environment and are not internal to the agent.
Each experiment in bsuite requires the five key qualities as mentioned below
- Targeted: performance in this task corresponds to a key issue in RL.
- Simple: strips away confounding/confusing factors in research.
- Challenging: pushes agents beyond the normal range.
- Scalable: provides insight on scalability, not performance on one environment.
- Fast: iteration from launch to results in under 30min on standard CPU.
The experiments in bsuite are a set of environments and a number of episodes of interaction. In bsuite, all the plots and analysis are generated through the automated bsuite Jupyter notebook. In order to generate the plots and analysis via the notebook, the user will require the path to the logged data.
Advantages of bsuite
- According to researchers, one of the most valuable uses of bsuite is as a diagnostic ‘unit-test’ for large-scale algorithm development
- Another benefit is that it can disseminate a developer’s results more easily and engage with the research community
Outlook
The Behaviour Suite for Reinforcement Learning is not a permanent replacement for the major challenges being undertaken in the AI domain. Instead, it is basically a collection of diagnostic experiments which are designed to provide meaningful insights into key aspects of RL agent behaviour. The researchers compared this project with the MNIST for reinforcement learning and stated that similar to MNIST dataset, bsuite aims to instantiate targeted experiments for the development of key RL capabilities. The Behaviour Suite for Reinforcement Learning is open-sourced in GitHub which will eventually provide significant value to the RL research community.
Read the paper here.
