Hello and welcome to this new tutorial and most importantly to this very essential new code section
that we're about to implement and that is of course the training of the AI.
So we build the AI and basically we made all the tools that will allow us to implement all the steps
here of the augmented random search algorithm.
You know you understand that what we've done so far are just preparing the tools so that we can integrate
all of them in one same training.
And this is exactly what we'll do in this code section this nuke detection to do the training.
However this will still be a function.
That doesn't mean that a D.N. of this new code section about the training we will run the code.
No it is still a function.
However at the very end the very last code section of this implementation we will run everything by
you know preparing the environment connecting the environment to the AI and then execute this train
function to train the AI to walk inside this environment.
So we need to understand the approach we're doing.
We first made some tools to organize and structure everything.
And now we're about to make this huge training function which will use the previous classes and methods
we've done before to run this whole training.
So let's get started.
And let's go back to Python.
All right so as I said we will integrate the Holcim still inside a function.
So I'm starting with Def and the name of the function will simply be train and it will take as arguments
first the environment.
Then of course the policy which is of course our AI AI which will train inside this environment.
Then our normalizer is why the normalizer should be an argument of this function.
Well it's in the case you know you want to change your normalizer In fact when you build an AI AI when
you build a machine learning or deploying model you usually try several normalizer lasers.
There can be several of them so it's nice to still have the option to change it easily by sending it
as an argument of a function which is to try and function here.
And then the last argument we will use is H-P which will be our future object that will contain all
the hyper parameters there are going to be fixed during the training.
And again why do we use it as an argument that because we want to be able to do some Chuen easily in
the end meaning that we want to be able to try other values of the hyper parameters.
And indeed it will be simple for us to do that because since HP is an argument of this train function
well we'll simply need to change and tweak the values here to try some other ones and still keep this
hybrid parameter object in a function without having to change anything.
Basically if you want to try other values of your hyper parameters you only need to change them here.
So that's pretty practical.
And that said that the arguments will need for this strange function.
All right let's not forget the Collen and let's define what the string function has to do.
And so what does it have to do.
Well let's go back to the paper and let me show you exactly what we'll do in this train function will
basically do this whole while loop you know while ending condition that satisfy do all the stuff and
all this stuff will be very easy for us to do.
Thanks to all the functions we made.
So that's why the reason we're going to do is well it's not actually going to be a while loop because
we made this and the steps are horrible.
You know as one of our hyper parameters which I remind is the number of loops in the training or more
specifically the number of times we're going to update our moral you know with this update function.
So it's also the number of steps of great in the sense we're going to do to abate the weight of our
policy.
And since we have this variable that is going to set the number of training loops or the number of updates.
Well we don't need to do a while loop and it's actually better to do a for loop because indeed we're
going to loop over all the steps in the range starting from zero but we don't have to specify because
it's a default value you know the lower bound of the range function in Python.
So from 0 up to this total number of steps of the training and I didn't forget the HP here because the
steps is a hyper parameter of our implementation.
All right and now we are ready to start the fall.
So this training code section is quite long so I want to structure it as much as possible and more specifically
I want to highlight each of the different steps of the training we're going into.
So we're going to break this down in several tutorials and in the next one we're going to start with
the first step of the training which will be to initialize the perturbations Delta and the positive
rewards as well as the negative rewards.
So basically we will sample the Deltas using our simple deltas method from the policy class that we
made earlier.
So that will simple are dealt us then we will initialize the positive rewards.
That is the worst that will get by applying the perturbations in the positive directions.
And then we'll initialized the negative rewards.
That is the reward.
We're going to get by applying the perturbations in the negative.
Or remember it's actually the opposite direction.
So just an initialization step and then we'll move on to the next step.
They're going to be about eight steps in total for the train.
So as you can see it's pretty long but since we're going to break it down you'll be able to take it
step by step and most of the time I remind I recommend to take a step back in order to see the logical
flow of the implementation and hear the training.
So let's start the first step in the next tutorial.
And until then enjoy AI.