Generalized linear (mixed) models etc.
The last 2 scripts introduce generalized linear (mixed) models and some extensions:
- First script: bayes_winbugs_jags_5.r; contents:
- generalized linear models
- Poisson “t-test” (simulated data, R analysis, WinBUGS / JAGS analysis)
- binomial “t-test” (simulated data, R analysis, WinBUGS / JAGS analysis)
- binomial ANCOVA (simulated data, R analysis, WinBUGS / JAGS analysis)
- binomial GLMM (simulated data, R analysis, WinBUGS / JAGS analysis)
- Second script: bayes_winbugs_jags_6.r; contents:
- GLMMs that take into account inter-annotator disagreement (simulated data, R analysis, WinBUGS / JAGS analysis)
- ordinal probit “t-test” (based on joint work with Jakub Dotlacil): simulated data for an ordinal probit “t-test”, i.e., an ordinal probit regression with only one predictor (a factor with 2 levels); frequentist analysis in R using the “ordinal” package; Bayesian analysis using WinBUGS / JAGS
Plan for the Feb. 7, Feb. 9 and probably Feb. 14 classes (again, feel free to bring your laptops to class and work through the scripts on your own as we go through them in class):
- First script: bayes_winbugs_jags_3.r; contents:
- simple linear regression (simulated data, R analysis, WinBUGS / JAGS analysis)
- goodness-of-fit assessment in Bayesian analyses (posterior predictive distributions and Bayesian p-values)
- interpretation of confidence vs. credible intervals
- fixed-effects 1-way ANOVA (simulated data, R analysis, WinBUGS / JAGS analysis)
- random-effects 1-way ANOVA (simulated data, R analysis, WinBUGS / JAGS analysis)
- inferring binomial proportions with hierarchical priors (random-effects for “coins”, i.e., basically, random-effect “binomial” ANOVA)
- Second script: bayes_winbugs_jags_4.r; contents:
- 2-way ANOVA w/o and w/ interactions (simulated data, R analysis, WinBUGS / JAGS analysis)
- ANCOVA and the importance of covariate standardization (simulated data, R analysis, WinBUGS / JAGS analysis)
- linear mixed-effects models—random intercepts only, independent random intercepts and slopes, correlated random intercepts and slopes (simulated data, R analysis, WinBUGS / JAGS analysis)
Intro to WinBUGS / JAGS
Plan for the Jan. 31 & Feb. 2 classes; feel free to bring your laptops to class and work through the scripts on your own as we go through them in class:
- First script: bayes_winbugs_jags_1.r; contents:
- the mean model: simulated data, R analysis, WinBUGS / JAGS analysis
- the structure of WinBUGS / JAGS models, reexpressing parameters, number of chains, number of iterations, burnin, thinning, the Brooks-Gelman-Rubin (BGR) convergence diagnostic (a.k.a. Rhat), graphical summaries of posterior distributions
- binomial proportion inference with WinBUGS / JAGS instead of the Metropolis algorithm we built “by hand” for this purpose
- comparison of 3 models for the same binomial proportion data with different uniform priors: posterior estimation with WinBUGS / JAGS and computing the evidence / marginal likelihood for each model based on the WinBUGS / JAGS posterior samples
- inference for 2 binomial proportions with WinBUGS / JAGS instead of the Metropolis algorithm we built “by hand” for this purpose
- Second script: bayes_winbugs_jags_2.r; contents:
- essentials of linear models (focus on design matrices)
- t-tests with equal and unequal variances: simulated data, R analysis, WinBUGS / JAGS analysis
Plan for the Jan. 24 & 26 classes:
- wrap up examples of inference for binomial proportions with conjugate Beta priors and with grid-discretized priors
- intro to Bayes for cognitive science; slides: intro_bayes_2.pdf
- introduction to Markov Chain Monte Carlo (MCMC) and the Metropolis family of algorithms bayes_MCMC_intro.r; on a couple of occasions, you will need the following 2 scripts (by John Kruschke; with very minor modifications here): plotPost.R and HDIofMCMC.R
Cox (1946) and Jaynes (2003)
The slides for Robert Henderson’s excellent presentation of Cox (1946) (“Probability, Frequency and Reasonable Expectation”) and Jaynes (2003) (“Probability Theory: The Logic of Science”), chapters 1 and 2 are available here.
Intro to Bayesian inference (part 1)
Plan for the Jan. 17 class:
- intro to Bayesian inference (part 1); slides: intro_bayes_1.pdf
- examples of inference for binomial proportions with conjugate (Beta) priors — based on Kruschke (2011), chapter 5: examples_Bern_Beta.r; to run the examples, you will need the following 2 files: BernBeta.R and HDIofICDF.R
- examples of inference for binomial proportions with grid-discretized priors — based on Kruschke (2011), chapter 6: examples_Bern_Grid.r; to run the examples, you will need the following 2 files: BernGrid.R and HDIofGrid.R
Intro to probability — slides
The slides we discussed on Jan. 12 in the semantics seminar are available here. You can also take a look at Kruschke (2011), chapters 3 and 4.
“Doing Bayesian Data Analysis” – Now in JAGS
John Kruschke has created JAGS versions of all the programs in “Doing Bayesian Data Analysis”.
Unlike BUGS, JAGS runs on MacOS, Linux, and Windows. JAGS has other features that make it more robust and user-friendly than BUGS. I recommend that you use the JAGS versions of the programs.
For more info, go here.
From CLG to LaLoCo (Dec. 28, 2011)
The Corpus Linguistics Group (CLG) has evolved into the Language, Logic & Cognition (LaLoCo) lab. A description of LaLoCo’s overarching research goal is available here.
The CLG materials (summaries, scripts etc.) have already been posted on the LaLoCo page:
- [Winter 2010] Tutorial on quantitative corpus linguistics and R
- [Spring 2010] Regression etc.
- [Fall 2010] Quantifier Scope, Intro to GLMs and GLMMS
- [Winter 2011] NLTK syn & sem
- [Spring 2011] GLMs & GLMMs (ctd.)
- [Spring 2011] Intro to Bayesian data analysis
- [Fall 2011] Ordinal probit ‘t-test’
Most of the LaLoCo / CLG meetings this quarter (winter 2012) will probably be folded into AB’s seminar in semantics.
Winter 2012 Seminar in Semantics (Linguistics, UCSC):
- Statistical & Cognitive Modeling for Formal Semantics
See the syllabus and AB’s teaching page for more information.
