Nowadays Artificial intelligence and machine learning offer unique opportunities and challenges for automating complex industrial systems. And a new paradigm is arriving to help us accomplish that, I am talking about “Machine teaching”. MT helps us to build ML systems, by moving the focus away from algorithms and onto successful model generation and deployment.

Machine teaching infuses subject matter expertise into automated AI system training …with deep reinforcement learning (DRL) and simulations. Abstracting away AI complexity to focus on subject matter expertise and real-world conditions creates models that turn automated control systems into autonomous systems.

Reinforcement Learning

When we talk about reinforcement learning we can talk about how we want to train an agent to accomplish some task. Let’s try to understand better how things work…. in this example, an agent can be a car, a drone, a robot, etc…. and that agent is going to interact with the environment by taking actions… the action is going to transition the environment to a new state …..and the agent is going to receive that new state plus some reward that determines how good or bad that action was.

Its objective is to learn to maximize that reward to find the policy that makes that reward score the highest.

In another example

1.An agent, actor, or brain, in this case, a robot, takes an action in an environment, in this case, a smart manufacturing line.

2.The action causes the environment to change state, and return its changed state to the agent.

3.An assessment mechanism applies a policy to determine what consequence to deliver to the agent.

4.The reward mechanism encourages beneficial actions by delivering a positive reward, and may discourage negative actions by delivering a penalty.

5. Rewards cause desired actions to increase, while penalties cause undesired actions to decrease.

What is Microsoft’s Project Bonsai and how can it help us build autonomous systems?

Let me tell you a short story…. More than a decade ago, Mark Hammond, Microsoft general manager for Business AI and former Bonsai CEO was working as a systems programmer in a Yale neuroscience lab and noticed how scientists used a step-by-step approach to train animals to perform tasks for their studies. He had a similar epiphany about borrowing those lessons to teach machines.

That ultimately led him to found Bonsai, which was acquired by Microsoft. It combines machine teaching with deep reinforcement learning and simulation to help companies develop “brains” that run autonomous systems in applications ranging from robotics and manufacturing to energy and building management.

So Bonsai is Microsoft Autonomous Systems platform is an innovative framework for building, training, and deploying models by using machine teaching and simulations. It manages the full end-to-end machine teaching lifecycle.

Development and deployment with Bonsai has three phases: Build, Train, and Deploy. 

1.The Build phase consists of writing the machine teaching program and connecting to a domain-specific training simulator. Simulators generate sufficient training data for experiments and machine practice.

2. In the Train phase, the training engine automates Deep Reinforcement Learning model generation and training by combining high-level domain models with appropriate Deep Reinforcement Learning algorithms and neural networks.

3. The Deploy phase deploys the trained brain to the target application in the cloud, on-premises, or embedded on-site.

What is Inkling?

Inking is Bonsai’s special-purpose programming language for training AI. Is a declarative, strongly-typed programming language specifically designed for artificial intelligence.

But why a new language?  Microsoft says that the fundamental ideas that the language needed to express, and its fundamental objectives, were so different from any other programming language, that they required a new language … The language is designed to solve problems around machine training and teaching… Inkling has the idea of a concept, and no other language has that. Inking has the concept of a curriculum, and no other language has that.

What is Project Moab?

To onboard engineers and developers keen to begin experimenting with the Bonsai, Microsoft created Project Moab, a new hardware kit that’s available as a simulator in MathWorks. It’s a three-armed robot with a joystick controller that attempts to keep a ball balanced on a magnet-attached transparent plate, and it’s intended to give users an environment in which they can learn and experiment with simulations.

What’s the role of simulators?

Bonsai has Simulations that help train the models across different kinds of environmental conditions and scenarios, much faster and safer than in the real world. Experts can supervise the agents as they work to solve problems in simulated environments, and provide feedback and guidance that lets the agents dynamically adapt within the simulation.

Simulations are available across a broad range of industries and systems, including mechanical and electrical engineering, autonomous vehicles, security and networking, transportation and logistics, and robotics. Simulation tools include:

•Simulink, a graphical programming tool developed by MathWorks for modeling, simulating, and analyzing dynamic systems.

•Gazebo, a tool to allow accurate simulation of populations of robots in complex indoor and outdoor environments. Gazebo is open-source licensed under Apache 2.0

•Microsoft AirSim, an open-source robotics simulation platform and this is the one I will explain today.

Let’s recap

With that said, to realize these state of the art autonomous systems Microsoft suggests to have the following approach across four essential dimensions.

First, harness that Deep subject matter expertise and experience. Instead of focusing on how something like the brain or a neural network Works, we need to focus instead on what we are trying to teach and how best to teach it. This allows us to leverage our Deep domain expertise to efficiently and effectively guide the acquisition of intelligence just as you would for a new employee in terms that are understandable to you and your colleagues instead of only data scientists or machine learning practitioners.

Then, for your system to safely and rapidly gain experience following your machine teaching programs, Microsoft brings us powerful professionals simulation capabilities from Simulink, Gazebo and AIRSIM with massive scalability of Microsoft azure allowing your system to learn across hundreds or thousands of concurrent simulation instances running in the cloud.

Then, we need to focus on teaching these systems by abstracting away and automating the low-level mechanics associated with the underlying AI. When our machine teaching program compiles and runs, things like neural networks and reward functions are generated automatically on our behalf.

And finally, once we built our controller, Microsoft provides the runtime capabilities needed to deploy and integrate it with the rest of our software stack.

Conclusion

Machines have been progressing on a path from being completely manual to having a fixed automated function to becoming intelligent where they can actually deal with real-world situations themselves. When people think of autonomous systems, many go straight to the vision of the fully autonomous car that drives itself while you sit in the back seat and read a book. And even though we are on our way to get there, we still have a lot of scenarios we can consider today. And Microsoft’s autonomous systems platform helps us achieve that.