L10.9 Mixed Bayes Rule

TL;DR

Mixed Bayes rule handles a discrete random variable K and a continuous random variable Y by combining a PMF with a PDF. It is derived by applying the multiplication rule in two orders to the joint event, approximating probabilities over a small interval of length delta, and taking the limit as delta approaches 0. Read on to see how the denominator is evaluated and what information each inference direction requires.

Transcript

We have seen two versions of the Bayes rule-- one involving two discrete random variables, and another that involves two continuous random variables. But there are many situations in real life when one has to deal simultaneously with discrete and continuous random variables. For example, you may want to recover a discrete digital signal that was se... Read More

Key Insights

• 📏 Bayes rule can be extended to situations involving both discrete and continuous random variables.
• 😑 The probability of two events happening can be expressed using the PMF and PDF functions.
• 👻 The Bayes rule for discrete and continuous random variables allows for making inferences about one variable given observations of the other.
• ❓ Inferences about discrete variables require knowledge of the unconditional distribution of the variable and the model of the noisy observation.
• ❓ Inferences about continuous variables require knowledge of the conditional densities of the variable under different scenarios for the related observation.
• 🍉 The total probability theorem is used to evaluate the denominator term in the Bayes rule equation.
• 🫡 The integral of the conditional density with respect to the variable must be equal to 1, ensuring the validity of the total probability theorem.

Questions & Answers

Q: How is Bayes rule extended to mixed discrete and continuous random variables?

For a discrete random variable K and a continuous random variable Y, apply the multiplication rule in two orders to the event that K equals k while Y lies in a small interval. Express the terms using a PMF and conditional or unconditional PDFs, cancel the interval length delta, and take the limiting case as delta approaches 0.

Q: What does mixed Bayes rule calculate when K is discrete and Y is continuous?

One version calculates the conditional probability distribution of K when Y takes a specific value. It is useful for inferring a discrete quantity from a continuous observation, such as recovering a discrete digital signal corrupted by continuous noise.

Q: Why does the derivation use a small interval around Y instead of a point probability?

Y is continuous, so the derivation considers the probability that it lies within a small interval. For small delta, that probability is approximately the relevant PDF value multiplied by the interval length, and the approximation becomes exact in the limiting argument as delta approaches 0.

Q: How are PMFs and PDFs combined in mixed Bayes rule?

The event K equals k is represented by the PMF of K evaluated at k. The event that Y falls in a small interval is represented approximately by a PDF value times the interval length, using a conditional PDF when K is given.

Q: What information is needed to infer K from a continuous observation Y?

You need the unconditional distribution of the discrete random variable K. You also need a model for the continuous observation Y under every possible value of K, meaning the conditional distribution of Y for each discrete scenario.

Q: Can mixed Bayes rule infer a continuous variable Y from a discrete observation K?

Yes. Rearranging the same equality produces a version for making an inference about the continuous random variable Y when the value of the related discrete random variable K is known.

Q: How is the denominator in mixed Bayes rule evaluated?

The denominator is evaluated with a suitable version of the total probability theorem. The density of Y is obtained by combining the conditional densities of Y under the different values of K, weighted by the probabilities of those discrete scenarios.

Q: Why are the two multiplication-rule expressions equal in the derivation?

Both expressions describe the same joint event: K takes a specified value while Y falls within a specified small interval. The multiplication rule can order the two events either way, so equating the resulting expressions provides the basis for deriving mixed Bayes rule.

Summary & Key Takeaways

• Bayes rule can be extended to situations where one variable is discrete and the other is continuous.

• The probability of two events happening can be expressed in terms of the probability mass function (PMF) and probability density function (PDF).

• The Bayes rule for discrete and continuous random variables allows for making inferences about one variable given observations of the other.