L14.6 Discrete Parameter, Continuous Observation

TL;DR

Bayesian inference can also be applied to situations where the observation is continuous, and the MAP (Maximum A Posteriori) rule remains the optimal way to estimate the true value.

Transcript

In the next variation that we consider, the random variable Theta is still discrete. So it might, for example, represent a number of alternative hypothesis. But now our observation is continuous. Of course, we do have a variation of the Bayes rule that's applicable to this situation. The only difference from the previous version of the Bayes rule i... Read More

Key Insights

• ❓ Bayesian inference can be applied to situations where the observation is continuous, not just discrete.
• 👻 Shifting the PDF of a random variable allows us to obtain the conditional PDF of the observation.
• 📏 The MAP rule is still used to estimate the most likely value, regardless of whether the observation is discrete or continuous.

Questions & Answers

Q: How does Bayes rule apply to situations with continuous observations?

In the case of continuous observations, the PMF (Probability Mass Function) is replaced by a PDF (Probability Density Function), but the rest of the Bayes rule remains the same.

Q: How is the conditional PDF of the observation obtained?

By adding a constant to the random variable and shifting its PDF, we can obtain the conditional PDF of the observation.

Q: What is the MAP rule?

The MAP (Maximum A Posteriori) rule is used to estimate the most likely value of the parameter based on the given observation.

Q: How is the probability of error calculated?

The probability of error can be calculated using the total probability theorem in two ways: by averaging the conditional probabilities of error over all possible values of the observation, or by taking a weighted average of the conditional probabilities of error for each possible value of the parameter.

Summary & Key Takeaways

• Bayes rule can be used to calculate the conditional probabilities of different choices when the observation is continuous.

• The conditional PDF of the observation can be obtained by shifting the PDF of a random variable.

• The MAP rule is used to estimate the most likely value, and the probability of error can be calculated using the total probability theorem in different ways.