Math Note on Lecture 3

We are glossing over a very important point in the mathematics of probability and statistics called the Law of Large Numbers which basically says that (under certain conditions) a statistical average from a sample should converge to the the theoretical probabilistic mean of the underlying random variable as the sample size goes to infinity.

A common example of this is the idea that if you toss a fair coin [i.e. with P(heads)=0.5] ten times it wouldn't be so unusual to have, say 8 heads and 2 tails, but if you toss it 1000 times, it is extremely unlikely that you would get 800 heads and 200 tails. In fact, it's very likely that you would have between say 450 and 550 heads.

So the result of the Law of Large Numbers is that the steady state average number of heads per toss is the same as the expected number of heads per toss.

The exact same argument applies to the number of losses per cell or per slot in this example.

As a side note, there is a generally untrue folklore version of this often referred to as "The Law of Averages" which says (in the context of the fair coin tossing example above) that if the last (say) 8 tosses have been heads, then the next toss is more likely to be a tail. This is certainly not true for i.i.d. random variables like coin tossing. OTOH, the Law of Large Numbers says in this case that over a very large number of tosses, the number of heads and the number of tails should certainly even up more.


URL for this page = http://networks.ecse.rpi.edu/~vastola/bbn/foils/lec6_2-3/lln.html