1
00:00:00,390 --> 00:00:06,690
Hello and welcome to this new tutorial in the previous Statoil we initialized the variables of our normalized

2
00:00:06,690 --> 00:00:07,260
class.

3
00:00:07,290 --> 00:00:12,480
Therefore at the future normalizer objects and now we're going to make a new method which will call

4
00:00:12,480 --> 00:00:17,570
the observed method and that will update and compute the mean and variance.

5
00:00:17,640 --> 00:00:19,940
Each time we observe a new state.

6
00:00:20,250 --> 00:00:21,150
So let's do this.

7
00:00:21,150 --> 00:00:27,960
It's going to make much more sense when we start writing the can potations so deaf because it's a new

8
00:00:27,960 --> 00:00:28,710
method.

9
00:00:28,830 --> 00:00:37,250
We're going to call this method observe and it's going to take us and put self just to get the variables

10
00:00:37,250 --> 00:00:42,460
here which are variables of the object and X which will be.

11
00:00:42,660 --> 00:00:50,000
And you state because we apply this observed method each time we observe a new state of the environment

12
00:00:50,430 --> 00:00:51,710
to gone.

13
00:00:51,750 --> 00:00:53,300
Now here we go.

14
00:00:53,330 --> 00:00:55,790
So first of all it's actually very simple.

15
00:00:55,790 --> 00:00:59,090
We want to compute the mean and the variance.

16
00:00:59,210 --> 00:01:04,820
You know the condition of the mean it's the sum of the values divided by the number of the values.

17
00:01:04,820 --> 00:01:08,170
We have a variable that gives us the number of values.

18
00:01:08,300 --> 00:01:10,660
It's n well and is actually a vector.

19
00:01:10,790 --> 00:01:16,520
But that's because we want to compute the mean for all the different values of the input state vector.

20
00:01:16,720 --> 00:01:19,330
OK so we're going to divide by N.

21
00:01:19,430 --> 00:01:22,130
But before this we need to increment.

22
00:01:22,370 --> 00:01:28,160
And of course each time we observe a new state because each time we observe a new state that's one more

23
00:01:28,160 --> 00:01:29,960
state in total.

24
00:01:30,020 --> 00:01:31,710
So that's the first thing we're going to do.

25
00:01:31,850 --> 00:01:33,730
Self that end.

26
00:01:34,160 --> 00:01:43,110
And the trick to increment a variable in Python is to do plus equals and then one point because we want

27
00:01:43,110 --> 00:01:43,860
it to be a.

28
00:01:43,940 --> 00:01:47,180
Since we're going to do a float computation.

29
00:01:47,180 --> 00:01:53,570
So you have to understand that this is a vector and still we can increment it this way all the values

30
00:01:53,570 --> 00:01:55,800
of this vector will be incremented by 1.

31
00:01:55,910 --> 00:02:05,050
So that and we'll go from being a vector of end being put zeros to a vector of and when put once.

32
00:02:05,150 --> 00:02:05,500
All right.

33
00:02:05,510 --> 00:02:11,740
Now we have the total number of states that we've encountered up to now that now that we have this total

34
00:02:11,750 --> 00:02:15,330
number of states we can compute the mean.

35
00:02:15,650 --> 00:02:19,490
However it's not going to be the simple formula of the mean.

36
00:02:19,520 --> 00:02:22,840
Summing all the values and dividing by the total number.

37
00:02:23,180 --> 00:02:29,840
It's an online computation of the mean which means that you have your mean at each time you have the

38
00:02:29,840 --> 00:02:36,600
previous mean and then by getting the next value you want to be able to compute the Neumann right.

39
00:02:36,890 --> 00:02:42,450
So the online condition of the mean is that it's done you get a new value.

40
00:02:42,680 --> 00:02:49,910
Well the new main will be the previous mean plus the value of the state minus the previous mean divided

41
00:02:49,910 --> 00:02:51,530
by the total number of states.

42
00:02:51,590 --> 00:02:53,610
That's the unknown computation of the mean.

43
00:02:53,840 --> 00:02:58,350
However for the variance it's a little more complicated.

44
00:02:58,460 --> 00:03:05,090
Each time you observe and you state the new variances the last variant plus the value minus the last

45
00:03:05,090 --> 00:03:11,350
mean times the value minus the Neumann and then divided of course by the total number of inputs.

46
00:03:11,600 --> 00:03:13,180
So we're going to write all this.

47
00:03:13,320 --> 00:03:19,500
But Therefore since the computation of the new online variants you need to have the last min.

48
00:03:19,730 --> 00:03:21,680
Well this is what we're going to get.

49
00:03:21,680 --> 00:03:31,190
Now we're going to make a copy of the last I mean that is to mean that we had just before observing

50
00:03:31,190 --> 00:03:37,870
this new state and I'm going to make a copy of that last minute because the new man is going to be updated.

51
00:03:37,910 --> 00:03:42,740
You know self-direct mean it's going to become the new me now but since I need the last minute to compute

52
00:03:42,740 --> 00:03:43,460
the variance.

53
00:03:43,670 --> 00:03:46,310
Well I'm going to make a copy of that current.

54
00:03:46,320 --> 00:03:48,470
I mean just before updating it.

55
00:03:48,650 --> 00:03:49,460
OK.

56
00:03:49,540 --> 00:03:56,360
So Lassman and to make up you can just take some of the main which is our current means just before

57
00:03:56,360 --> 00:03:57,370
we updated.

58
00:03:57,770 --> 00:04:05,430
And then you can add here that copy so that it will be like a temp variable getting the last minute.

59
00:04:05,970 --> 00:04:06,750
OK.

60
00:04:07,190 --> 00:04:13,670
And now with this counter incremental and the last min we have everything we need to compute the mean

61
00:04:13,760 --> 00:04:14,930
and the variance.

62
00:04:14,930 --> 00:04:20,250
So let's start with the mean the new mean which is going to be self.

63
00:04:20,270 --> 00:04:27,920
That means because basically I'm updating the mean self that mean as I said it's going to be the previous

64
00:04:27,920 --> 00:04:35,450
self that mean plus the value of the state X minus the previous mean divided by the total number of

65
00:04:35,450 --> 00:04:37,710
states and them.

66
00:04:38,030 --> 00:04:45,530
Therefore in order to get this we can just do another plus equal and then we'll as we just said it's

67
00:04:45,770 --> 00:04:53,150
the value of the state X minus the previous mean self-determine because that mean is not a data yet

68
00:04:53,360 --> 00:04:58,060
it's going to become dated once this is computed or we could even take less mean.

69
00:04:58,070 --> 00:04:59,410
But that's the same here.

70
00:04:59,420 --> 00:05:04,160
However for the vines we will have to take less mean because cells that mean will already have been

71
00:05:04,160 --> 00:05:09,140
updated right now and right here it isn't yet so subduct mean.

72
00:05:09,140 --> 00:05:15,920
And then divided by the total number of states and counted so far.

73
00:05:16,370 --> 00:05:23,060
And that is exactly the online computation of the mean which means it is exactly the competition you

74
00:05:23,060 --> 00:05:28,830
have to do when you already have a previous mean you observe a new value and you want to compute the

75
00:05:28,830 --> 00:05:29,630
new mean.

76
00:05:29,690 --> 00:05:33,190
This is how you compute the new Mean.

77
00:05:33,230 --> 00:05:34,000
All right.

78
00:05:34,130 --> 00:05:39,080
So it's good to know it's very important to know whether you're doing statistics or machine learning

79
00:05:39,080 --> 00:05:44,930
deep learning or AI you will have to do that from time to time especially when you want to normalize

80
00:05:44,930 --> 00:05:46,530
your values.

81
00:05:46,530 --> 00:05:53,900
And now that we have the mean let's compute the variance that is let's do an online computation of.

82
00:05:54,170 --> 00:05:59,660
Meaning the same that is we have the previous variants we observe a new state therefore a new value.

83
00:05:59,900 --> 00:06:02,030
And we want to compute the new variants.

84
00:06:02,030 --> 00:06:09,410
So in order to do this remember we want to break down first with the numerator that we called Self mean

85
00:06:10,430 --> 00:06:14,590
def and that's what we want to online update first.

86
00:06:14,600 --> 00:06:18,650
So I told you how we need to do that at the beginning of this tutorial.

87
00:06:18,770 --> 00:06:26,780
The new numerator after observing a new value is the sum of the previous numerator.

88
00:06:26,780 --> 00:06:29,010
That is this one just before it is dated.

89
00:06:29,600 --> 00:06:36,960
Plus So I'm adding the plus equals here the value of the state minus the last mean and this time I cannot

90
00:06:36,980 --> 00:06:41,500
take self mean because self-manage was already updated just with the previous line here.

91
00:06:41,630 --> 00:06:51,770
So I have to take less mean so X minus less mean times the value of the state again minus the new Mean

92
00:06:52,040 --> 00:06:56,020
and the new means of course self-determine because it was already updated.

93
00:06:56,030 --> 00:06:59,590
This is the new mean this was the last min and this is the last.

94
00:06:59,750 --> 00:07:03,680
So self that mean.

95
00:07:03,710 --> 00:07:09,530
All right so now you clearly understand why we had to put in memory the previous mean in a separate

96
00:07:09,620 --> 00:07:11,400
less maneuverable.

97
00:07:11,420 --> 00:07:13,730
So this is the numerator.

98
00:07:13,730 --> 00:07:16,830
This is the online update of the numerator.

99
00:07:16,940 --> 00:07:19,740
That's all remember we want the variance.

100
00:07:20,090 --> 00:07:23,800
And now it's going to be simple in order to get the new variance.

101
00:07:23,810 --> 00:07:29,470
So self-taught var we don't have to do a sequel because we just had to do that for numerator but you

102
00:07:29,470 --> 00:07:30,950
guess what we're going to have to do.

103
00:07:30,950 --> 00:07:39,110
Now we are going to have to divide mindif by self-doubt and that's indeed the consideration of normal

104
00:07:39,110 --> 00:07:45,350
variance you know online variance but the variance of some values it's the sum of the square differences

105
00:07:45,650 --> 00:07:49,780
between the values and the mean divided by the total number of values.

106
00:07:49,910 --> 00:07:59,600
Ok so now we just need to take again our self that mean if and divided by the total number of states

107
00:07:59,720 --> 00:08:02,580
and counted so far which is even by that.

108
00:08:03,110 --> 00:08:04,630
But then that's not all.

109
00:08:04,790 --> 00:08:08,860
And this is not part of the consideration of the variance or the online variance.

110
00:08:09,020 --> 00:08:14,810
It's just because in the next it's oil we're going to finally define the normalized function to normalize

111
00:08:14,810 --> 00:08:18,720
the state because we have what we need will have the mean and variance.

112
00:08:19,160 --> 00:08:25,930
But since when you normalize this state you subtract both the mean and then you divide by the variance.

113
00:08:26,100 --> 00:08:29,890
What that means that variance must never be equal to zero.

114
00:08:30,170 --> 00:08:33,060
And that's exactly what we're going to make sure of right now.

115
00:08:33,050 --> 00:08:38,340
We're going to add a little trick here to make sure that the variance will not be equal to zero.

116
00:08:38,540 --> 00:08:47,330
And this trick is just to add that clip and then in parenthesis mean equals 0.01 which we can right

117
00:08:47,330 --> 00:08:48,280
this way.

118
00:08:48,560 --> 00:08:50,770
One minus two.

119
00:08:50,780 --> 00:08:53,470
That is one time stented the power of minus 2.

120
00:08:53,750 --> 00:08:54,410
OK.

121
00:08:54,680 --> 00:09:00,110
And this makes sure the variance will not be equal to zero and therefore we'll be able to divide our

122
00:09:00,110 --> 00:09:03,700
values subtracted by the mean by the variance.

123
00:09:03,770 --> 00:09:04,220
Perfect.

124
00:09:04,220 --> 00:09:11,900
So we have the observed function ready which not only observes a new state but update online update

125
00:09:12,200 --> 00:09:13,770
the mean and variance.

126
00:09:13,970 --> 00:09:21,380
And so now we're ready to make the last method of this normalizer class which is of course to normalize

127
00:09:21,530 --> 00:09:22,260
the state.

128
00:09:22,490 --> 00:09:25,790
And this will be very easy because basically we have everything we need.

129
00:09:25,790 --> 00:09:27,300
We just need the mean and the variance.

130
00:09:27,410 --> 00:09:31,030
And you know decomposition so it will be a piece of cake.

131
00:09:31,160 --> 00:09:34,600
So I'll see in the next it's oil and until then enjoy AI.