We present a model of the dynamics of adaptive attention allocation in the AX Continuous Performance Test (AX-CPT), a simple context dependent decision making task of interest to the research communities concerned with cognitive control, schizophrenia, anxiety and aging (Braver et al., 2001; Cohen et al., 1999; Eysenck et al., 2007). We ground it in our recent theory of decision making under dynamic context, that assumes humans use sequential Bayesian inference to combine information from multiple sources in perception and (optionally) memory over time. The theory generalizes the well-known diffusion decision model of single-stimulus decision making (DDM; Ratcliff, 1978). Our first result is a new analysis that shows how memory encoding and retention can yield a variable initial condition for either a single- or multiple-stimulus decision, providing a theoretical grounding to the assumption of variability in initial condition previously shown to improve data fits for the DDM. Our second result is using this model to decode attention allocation from behavioral data in a novel quantitative payoff manipulation in the AX-CPT, showing that our model can capture the differences in how subjects encode and retrieve contextual information when the relative emphasis on task speed and accuracy is changed.