Lecture 20: Multinomial and Cauchy | Statistics 110

TL;DR

This analysis explores joint distributions, specifically using the Cauchy distribution as an example, and discusses methods for finding the PDF (Probability Density Function) and CDF (Cumulative Distribution Function) of a Cauchy distribution.

Transcript

So we're talking about joint distributions, right? And there's a lot more to do with that, so to just continue. So last time, we calculated the expected distance between two iid uniforms, okay? So I wanted to do this analagous problem for the normal. Because I think that's another nice related example that has a different approach that makes it eas... Read More

Key Insights

• 💨 Joint distributions provide a way to model the relationship between multiple random variables.
• 🔨 The properties of the multinomial distribution make it a useful tool for modeling categorization processes.
• 🖤 The Cauchy distribution is notable for its lack of a well-defined mean or variance.

Questions & Answers

Q: What is the difference between joint distributions and univariate distributions?

Joint distributions involve more than one random variable, while univariate distributions involve only one random variable. Joint distributions provide information about how multiple variables interact with each other.

Q: What is the difference between the expected value of absolute difference and expected value of the sum of two variables?

The expected value of absolute difference measures the average difference between two variables, regardless of their signs. The expected value of the sum measures the average sum of two variables, taking into account their signs.

Q: What is the lumping property of the multinomial distribution?

The lumping property refers to the ability to combine multiple categories of the multinomial distribution into one category, resulting in a simplified distribution with fewer dimensions. This is useful when there are several small categories that can be grouped together without significantly affecting the overall distribution.

Q: Why is the Cauchy distribution considered "evil"?

The Cauchy distribution is considered "evil" because it does not have a well-defined mean or variance, making it challenging to work with mathematically. Additionally, averaging a large number of Cauchy variables does not converge to a different distribution, complicating statistical analysis.

Summary

In this video, the concept of joint distributions is explored further. The expected distance between two iid uniforms was calculated in the previous video, and now the same problem is approached for the normal distribution. The example focuses on finding the expected value of the absolute difference between two iid standard normal variables. The speaker emphasizes the importance of understanding the structure of the problem and how properties of the difference of normals can be used to simplify it. The speaker also introduces the multinomial distribution as a discrete multivariate distribution and explains its relation to the binomial distribution. The properties of the multinomial distribution, such as the marginal distribution and the lumping property, are discussed. Finally, an example involving the Cauchy distribution is presented, and the process of finding its probability density function (PDF) is explained.

Questions & Answers

Q: How is the problem of finding the expected value of the absolute difference between two iid standard normal variables approached?

The problem is approached by considering the structure of the problem and thinking about the properties of the difference of normals. Instead of immediately using the joint probability density function (PDF), the speaker simplifies the problem by recognizing that the difference between two standard normal variables is also a standard normal variable.

Q: How does the speaker prove that the sum of two independent normal variables is also normal?

By using moment generating functions (MGFs), the speaker proves that the sum of two independent normal variables with means and variances (mu1, sigma1^2) and (mu2, sigma2^2) is a normal variable with mean mu1 + mu2 and variance sigma1^2 + sigma2^2. The proof involves finding the MGF of the sum and showing that it matches the MGF of a normal variable with the desired mean and variance.

Q: What simplification is made when considering the expected value of the absolute difference between two iid standard normal variables?

The simplification made is recognizing that the absolute difference between two standard normal variables can be thought of as the scale factor (sqrt(2)) multiplied by a standard normal variable (Z). Therefore, the problem can be transformed into finding the expected value of sqrt(2) * |Z| and can be solved using a one-dimensional law of the unconscious statistician (LOTUS) integral.

Q: How is the multinomial distribution defined and what does it represent?

The multinomial distribution is a generalization of the binomial distribution for more than two categories. It represents the distribution of categorizing n objects into k categories with each object being independently assigned to a category. The multinomial distribution requires two parameters: n, the number of objects, and a probability vector p, which specifies the probabilities of each category.

Q: What are the properties of the multinomial distribution that were discussed?

The properties of the multinomial distribution include the marginal distribution and the lumping property. The marginal distribution of one category in the multinomial distribution is a binomial distribution with parameters determined by the number of objects and the probability of that category. The lumping property allows for merging certain categories together, resulting in a new multinomial distribution with fewer dimensions.

Q: Why is the Cauchy distribution considered "evil"?

The Cauchy distribution is considered "evil" because it lacks certain properties that are typically associated with distributions. It does not have an expected value or a variance, and its strange behavior when averaging multiple IID Cauchy variables leads to unusual results. Despite its unconventional properties, the Cauchy distribution is still useful in certain applications where ratios are a natural occurrence.

Q: How is the probability density function (PDF) of the Cauchy distribution derived?

The PDF of the Cauchy distribution, denoted as T, can be derived by finding its cumulative distribution function (CDF) and then taking its derivative. One approach is to directly write down the integral of the joint probability density function (PDF) and evaluate it using techniques such as changing variables and simplifying expressions. Another approach is to use the law of total probability and condition on either X or Y to simplify the integral. Both methods ultimately yield the same result: a Cauchy distribution PDF of 1 / (pi * (1 + t^2)).

Takeaways

The joint distributions of random variables can be analyzed by understanding the structure of the problem and utilizing the properties of the variables involved. In the case of the expected value of the absolute difference between two iid standard normal variables, recognizing the properties of the difference of normals simplifies the problem. The multinomial distribution is a useful multivariate distribution for categorizing objects into multiple categories, with properties such as the marginal distribution and the lumping property. The Cauchy distribution is a unique distribution with unconventional properties such as the lack of a mean and variance. Different approaches, such as integrating the joint PDF or using the law of total probability, can be used to find the PDF of the Cauchy distribution.

Summary & Key Takeaways

• Joint distributions are discussed and compared to univariate distributions.

• The example of finding the expected value of absolute difference between two iid standard normal variables is used to demonstrate the simplicity of the problem when simplified using the properties of the difference of normals.

• The multinomial distribution is introduced and defined as a multivariate distribution where objects are independently categorized into different categories.

• The properties of the multinomial distribution, such as its marginal distribution and lumping property, are discussed.

• The Cauchy distribution is defined as the distribution of the ratio of two independent identically distributed standard normal variables.

• The PDF of the Cauchy distribution is derived using a double integral and the derivative of the CDF.