Bayes Nets in Educational Assessment: Where Do the Numbers Come From?

Educational assessments that exploit advances in technology and cognitive psychology can produce observations and pose student models that out-strip familiar test-theoretic models and analytic methods. Bayesian inference networks (BINs), which include familiar models and techniques as special cases, can be used to manage belief about students’ knowledge and skills, in light of what they say and do. BINs for assessments that add new tasks to their item pools and measure different students with different items can be assembled from building-blocks fragments. A student-model BIN (SM-BIN) fragment contains student model variables, which characterize aspects of knowledge. Evidence model BIN fragments (EM-BINs) contain observable variables and pointers to student model variables. Joining EM-BIN fragments to an SM-BIN fragment permits one to update belief about a student as observations arrive in a setting the EM-BIN was constructed to handle. Markov Chain Monte Carlo (MCMC) techniques can be used to estimate the conditional probabilities in the BINs from empirical data, supplemented by expert judgment or substantive theory. Details for the special cases of item response theory (IRT) and multivariate latent class modeling are given, with a numerical example of the latter.