# Introduction to Bayesian Statistics Course Website: https://sjster.github.io/introduction_to_computational_statistics/docs/index.html Git repo: https://github.com/vivek-s/intro_to_bayesian_coursera ## Bayesian vs Frequentist Inference In Inference, we want to measure the probability of a hypothesis given certain data. Frequentists choose a hypothesis and determine the probability of the data given this hypothesis. They then use this as evidence of the hypothesis. Bayesians go one step further to invert this to get the probability of the hypothesis given the data. # Introduction to Bayesian Statistics [[Probability Theory]] ## Bayes Theorem $P(A,B) = P(A|B) P(B)$ Also, $P(A,B) = P(B|A) P(A)$ Equating the two $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$ This is known as Bayes' Rule. Left hand side P(A|B) - posterior Right side first term P(B|A) - Likelihood P(A) - Prior P(B) - marginal probability of B. many times hard to calcuate and this is a constant and is ignored through use of proportions Posterior = Likelihood * Prior / Marginal likelhood (evidence) # Priors Choice of the prior is as much about what is currently known about the parameters as it is about the goal of the analysis. This makes the choice of prior subjective and often contested. How do we choose priors? Two broad categories could include: 1. noninformative 2. informative The priors can also be proper, ie conform to the rules of probability and integrate to 1, or improper. As an example, consider the uniform distribution as a prior to the mean in a normal distribution, we want to apply equal weights to all possible values of 𝜇 Often choice of prior is subjective contested. ## Conjugate Prior Let us say you assume the Prior is a Bernolli distribution with parameter $\theta$ Then you might want to assume a Beta distribution for $\theta$ . This is conjugate prior Parameters of the "conjugate function" are called hyperparameters The general approach to using priors and models is really to start with some justification, run the analysis, and then come up with competing priors and examine what your results are and how your conclusions might have changed. Common conjugate priors by likelihood type: likelihood - prior Binomial - beta Poisson - Gamma Normal (known 𝜎2) - Normal Normal (known 𝜇) - inverse-gamma Multivariate normal (known Σ) - inverse-Wishart # General references: - Mathematical Statistics and Data Analysis (9788131519547): Rice, John A. - Statistical Inference (9780534243128): Casella, George, Berger, Roger L. - Probability Theory and Statistical Inference: Empirical Modeling with Observational Data (9781107185142): Spanos, A. - Bayesian Models: A Statistical Primer for Ecologists (9780691159287): Hobbs, N. Thompson, Hooten, Mevin B. - A First Course in Bayesian Statistical Methods (0387922997): Hoff, Peter D. - Bayesian Data Analysis by Andrew Gelman, John Carlin, Hal Stern, David Dunson, Aki Vehtari, and Donald Rubin.