Hello and welcome to this new tutorial in the previous Statoil we initialized the variables of our normalized
class.
Therefore at the future normalizer objects and now we're going to make a new method which will call
the observed method and that will update and compute the mean and variance.
Each time we observe a new state.
So let's do this.
It's going to make much more sense when we start writing the can potations so deaf because it's a new
method.
We're going to call this method observe and it's going to take us and put self just to get the variables
here which are variables of the object and X which will be.
And you state because we apply this observed method each time we observe a new state of the environment
to gone.
Now here we go.
So first of all it's actually very simple.
We want to compute the mean and the variance.
You know the condition of the mean it's the sum of the values divided by the number of the values.
We have a variable that gives us the number of values.
It's n well and is actually a vector.
But that's because we want to compute the mean for all the different values of the input state vector.
OK so we're going to divide by N.
But before this we need to increment.
And of course each time we observe a new state because each time we observe a new state that's one more
state in total.
So that's the first thing we're going to do.
Self that end.
And the trick to increment a variable in Python is to do plus equals and then one point because we want
it to be a.
Since we're going to do a float computation.
So you have to understand that this is a vector and still we can increment it this way all the values
of this vector will be incremented by 1.
So that and we'll go from being a vector of end being put zeros to a vector of and when put once.
All right.
Now we have the total number of states that we've encountered up to now that now that we have this total
number of states we can compute the mean.
However it's not going to be the simple formula of the mean.
Summing all the values and dividing by the total number.
It's an online computation of the mean which means that you have your mean at each time you have the
previous mean and then by getting the next value you want to be able to compute the Neumann right.
So the online condition of the mean is that it's done you get a new value.
Well the new main will be the previous mean plus the value of the state minus the previous mean divided
by the total number of states.
That's the unknown computation of the mean.
However for the variance it's a little more complicated.
Each time you observe and you state the new variances the last variant plus the value minus the last
mean times the value minus the Neumann and then divided of course by the total number of inputs.
So we're going to write all this.
But Therefore since the computation of the new online variants you need to have the last min.
Well this is what we're going to get.
Now we're going to make a copy of the last I mean that is to mean that we had just before observing
this new state and I'm going to make a copy of that last minute because the new man is going to be updated.
You know self-direct mean it's going to become the new me now but since I need the last minute to compute
the variance.
Well I'm going to make a copy of that current.
I mean just before updating it.
OK.
So Lassman and to make up you can just take some of the main which is our current means just before
we updated.
And then you can add here that copy so that it will be like a temp variable getting the last minute.
OK.
And now with this counter incremental and the last min we have everything we need to compute the mean
and the variance.
So let's start with the mean the new mean which is going to be self.
That means because basically I'm updating the mean self that mean as I said it's going to be the previous
self that mean plus the value of the state X minus the previous mean divided by the total number of
states and them.
Therefore in order to get this we can just do another plus equal and then we'll as we just said it's
the value of the state X minus the previous mean self-determine because that mean is not a data yet
it's going to become dated once this is computed or we could even take less mean.
But that's the same here.
However for the vines we will have to take less mean because cells that mean will already have been
updated right now and right here it isn't yet so subduct mean.
And then divided by the total number of states and counted so far.
And that is exactly the online computation of the mean which means it is exactly the competition you
have to do when you already have a previous mean you observe a new value and you want to compute the
new mean.
This is how you compute the new Mean.
All right.
So it's good to know it's very important to know whether you're doing statistics or machine learning
deep learning or AI you will have to do that from time to time especially when you want to normalize
your values.
And now that we have the mean let's compute the variance that is let's do an online computation of.
Meaning the same that is we have the previous variants we observe a new state therefore a new value.
And we want to compute the new variants.
So in order to do this remember we want to break down first with the numerator that we called Self mean
def and that's what we want to online update first.
So I told you how we need to do that at the beginning of this tutorial.
The new numerator after observing a new value is the sum of the previous numerator.
That is this one just before it is dated.
Plus So I'm adding the plus equals here the value of the state minus the last mean and this time I cannot
take self mean because self-manage was already updated just with the previous line here.
So I have to take less mean so X minus less mean times the value of the state again minus the new Mean
and the new means of course self-determine because it was already updated.
This is the new mean this was the last min and this is the last.
So self that mean.
All right so now you clearly understand why we had to put in memory the previous mean in a separate
less maneuverable.
So this is the numerator.
This is the online update of the numerator.
That's all remember we want the variance.
And now it's going to be simple in order to get the new variance.
So self-taught var we don't have to do a sequel because we just had to do that for numerator but you
guess what we're going to have to do.
Now we are going to have to divide mindif by self-doubt and that's indeed the consideration of normal
variance you know online variance but the variance of some values it's the sum of the square differences
between the values and the mean divided by the total number of values.
Ok so now we just need to take again our self that mean if and divided by the total number of states
and counted so far which is even by that.
But then that's not all.
And this is not part of the consideration of the variance or the online variance.
It's just because in the next it's oil we're going to finally define the normalized function to normalize
the state because we have what we need will have the mean and variance.
But since when you normalize this state you subtract both the mean and then you divide by the variance.
What that means that variance must never be equal to zero.
And that's exactly what we're going to make sure of right now.
We're going to add a little trick here to make sure that the variance will not be equal to zero.
And this trick is just to add that clip and then in parenthesis mean equals 0.01 which we can right
this way.
One minus two.
That is one time stented the power of minus 2.
OK.
And this makes sure the variance will not be equal to zero and therefore we'll be able to divide our
values subtracted by the mean by the variance.
Perfect.
So we have the observed function ready which not only observes a new state but update online update
the mean and variance.
And so now we're ready to make the last method of this normalizer class which is of course to normalize
the state.
And this will be very easy because basically we have everything we need.
We just need the mean and the variance.
And you know decomposition so it will be a piece of cake.
So I'll see in the next it's oil and until then enjoy AI.