Lecture 9: Expectation, Indicator Random Variables, Linearity | Statistics 110

TL;DR

Expectation in statistics is the average of a random variable, calculated by summing each possible value multiplied by its probability. Key tools for finding expected values include indicator random variables, linearity of expectation, and symmetry. The fundamental bridge connects probability with expectation, allowing conversion between these concepts.

Transcript

PROFESSOR: We've been talking about random variables and their distributions. And the main topic for today is averages-- that is, how do you compute the average? First of all, what is an average mean, and how do you compute the average of a random variable? OK. But before we do that, I want to say a little bit more about CDF because CDF-- I think i... Read More

Key Insights

• Expectation is the average of a random variable, calculated by summing each possible value multiplied by its probability.
• The Cumulative Distribution Function (CDF) is a function that describes the probability that a random variable takes on a value less than or equal to a certain point.
• Linearity of expectation means the expected value of a sum is the sum of expected values, even if variables are dependent.
• Indicator random variables are used to express probabilities as expected values, providing a bridge between probability and expectation.
• The geometric distribution describes the number of failures before the first success in repeated Bernoulli trials.
• The expected value of a binomial distribution can be easily calculated using linearity of expectation.
• The hypergeometric distribution's expected value can be found using indicator variables and symmetry, despite dependent trials.
• CDFs have three important properties: they are non-decreasing, right-continuous, and approach 0 or 1 at the extremes.

Questions & Answers

Q: How to calculate the expectation of a random variable?

The expectation of a random variable is calculated by summing each possible value of the variable multiplied by its probability. This provides the average or expected outcome of the variable. It's a fundamental concept in probability and statistics, used to predict the central tendency of a random variable's distribution.

Q: What is the Cumulative Distribution Function (CDF)?

The Cumulative Distribution Function (CDF) of a random variable is a function that represents the probability that the variable will take a value less than or equal to a certain point. It is non-decreasing, right-continuous, and approaches 0 as x approaches negative infinity and 1 as x approaches positive infinity.

Q: What is linearity of expectation?

Linearity of expectation states that the expected value of the sum of random variables is equal to the sum of their expected values. This holds true regardless of whether the variables are independent or dependent. It simplifies calculations of expected values in complex scenarios.

Q: How do indicator random variables work?

Indicator random variables are used to represent the occurrence of an event as a random variable. They take the value 1 if the event occurs and 0 otherwise. This simplifies the calculation of probabilities as expected values, providing a bridge between probability and expectation.

Q: What is the geometric distribution?

The geometric distribution models the number of failures before the first success in a series of independent Bernoulli trials, each with the same probability of success. The probability mass function is given by p(1-p)^k, where k is the number of failures, and p is the success probability.

Q: How to find the expected value of a binomial distribution?

The expected value of a binomial distribution can be calculated using the formula E(X) = n * p, where n is the number of trials and p is the probability of success on each trial. This is derived using the linearity of expectation, which simplifies the calculation process.

Q: How is the hypergeometric distribution's expected value calculated?

The expected value of a hypergeometric distribution is calculated using indicator random variables and symmetry. Despite the dependency between trials, these tools allow the expected value to resemble that of a binomial distribution, calculated as the number of draws times the probability of success for a single draw.

Q: What are the properties of CDFs?

CDFs have three important properties: they are non-decreasing, meaning they never decrease as x increases; they are right-continuous, meaning they approach the function value from the right; and they approach 0 as x approaches negative infinity and 1 as x approaches positive infinity.

Summary

In this video, the speaker discusses random variables, their distributions, and how to compute the average result. They start by explaining the cumulative distribution function (CDF) and its properties. Then, they talk about independence of random variables and how to compute probabilities using CDFs and PMFs. The main topic of averages is introduced, and the concept of expected value is defined. The speaker provides examples of finding the expected value of different distributions, such as the Bernoulli and hypergeometric distributions. They also introduce the geometric distribution and show how to find its expected value. The video emphasizes the use of indicators, linearity, and symmetry in computing expected values.

Questions & Answers

Q: What is the cumulative distribution function (CDF)?

The CDF is a function that assigns probabilities to values of a random variable. It gives the probability that a random variable is less than or equal to a given value. The CDF is well-defined for all real numbers and has properties such as being increasing, right continuous, and approaching zero as the variable goes to negative infinity.

Q: How can the CDF be used to compute probabilities?

From the CDF, probabilities of events can be computed by subtracting the CDF values at the endpoints of the event. For example, to find the probability that a random variable is between two values, subtract the CDF value at the smaller value from the CDF value at the larger value.

Q: What are the properties of a CDF?

A CDF has three important properties. First, it is increasing, meaning that as the input value increases, the probability also increases. Second, it is right continuous, meaning that the probability does not jump abruptly at each value but rather continues smoothly. Third, as the input value goes to negative infinity, the CDF approaches zero, indicating that the probability of the random variable being smaller than any negative value is zero.

Q: How can the PMF be recovered from the CDF?

The PMF, or probability mass function, gives the probabilities of each individual value of a discrete random variable. In the CDF, the size of the jumps represents the probability of each value. By summing up the sizes of the jumps, the PMF can be recovered.

Q: What is the definition of independence for random variables?

Two random variables are independent if the probability of their joint event equals the product of their individual probabilities. This can be written as P(X <= x, Y <= y) = P(X <= x) * P(Y <= y). Independence means that the occurrence of one event does not affect the likelihood of the other event.

Q: How can the expected value of a random variable be computed?

The expected value, or mean, of a random variable can be computed by summing the product of each value and its corresponding probability. This can be done using the PMF of a discrete random variable or in some cases, using indicators and linearity.

Q: What is the expected value of a Bernoulli random variable?

The expected value of a Bernoulli random variable equals the probability of success. This means that if a Bernoulli random variable has probability P of being 1 and 1-P of being 0, its expected value is P.

Q: How can expected values be computed for distributions involving indicators?

For distributions involving indicators, the expected value can be computed by considering the probabilities of the events corresponding to the indicators. The expected value is equal to the probability of the event occurring.

Q: What is the expected value of a hypergeometric distribution?

The expected value of a hypergeometric distribution can be computed by multiplying the probability of the event of interest by the total number of trials. For example, if we are interested in the number of aces in a five-card hand, the expected value is 5 times the probability of drawing an ace.

Q: What is the expected value of a geometric distribution?

The expected value of a geometric distribution is equal to the reciprocal of the probability of success. In other words, for a geometric distribution with parameter P, the expected value is 1/P. The geometric distribution represents the number of failures before the first success in a series of independent Bernoulli trials.

Q: What is linearity of expectation?

Linearity of expectation is a property that allows the expected value of a sum of random variables to be computed by summing the expected values of each random variable individually. It states that the expected value of the sum of two random variables equals the sum of their expected values, even if the random variables are dependent.

Summary & Key Takeaways

• Expectation is the average of a random variable, found by summing each value times its probability. Key tools include indicator variables, linearity, and symmetry. The fundamental bridge connects probability and expectation, enabling conversions. The geometric distribution models failures before success in repeated trials.

• CDFs describe the probability a variable is less than or equal to a point. They are non-decreasing, right-continuous, and approach limits at extremes. Linearity means expected values of sums equal sums of expected values, even with dependencies.

• Indicator variables express probabilities as expected values, linking probability and expectation. The expected value of a binomial distribution is easily calculated with linearity. The hypergeometric distribution's expected value uses indicator variables and symmetry, despite dependencies.