The Visual World
Data analysis for "visual world" experiments
This website was created in September 2007 to accompany my article "Analyzing 'visual world'
eyetracking data using multilevel logistic regression" for the special
issue of the Journal of Memory and Language on Emerging Data
Analysis and Interferential Techniques (to appear in Nov. 2008). This site is updated regularly.
A draft of the article is available here as a pdf file.
When analyzing categorical data, different results are sometimes obtained using logistic regression instead of ANOVA or a t-test on proportions. This raises the question: which is the "right" analysis? Defenses of logistic regression often appeal to statistical authority &mdash i.e., we should do logistic regression because that's what statisticians tell us to do &mdash rather than any principled argument in favor of the log odds scale. To justify use of the log odds scale, I resurrect an analysis from one of the foundational papers on probit analysis by Bliss (1934). The mystery was why each unit increase in dosage gave rise to a curved rather than a linear mortality function.
(read more...)
Posted by Dale Barr on Sat 13 Sep 2008 at 2:43 PM | Link
One problem with eyetracking data sets is that the response is oversampled. The term oversampling is usually used in the context of audio sampling, rather than in statistics, but it captures an important idea. We record frames of eye data at a very high rate (60-250 Hz), and the time between two adjacent frames (17 ms or less) is possibly shorter than the frequency with which decisions to move the eyes (or to remain in place) are made and sent to the oculomotor system. Even though a listener's beliefs about the identity of the target may be rapidly changing based on the incoming speech, it takes time both to program an eye movement (around 180 ms or so) as well as to move the eyes through space from one region to another. These factors create dependencies in the data set, making it difficult to apply standard statistical techniques. One trick to overcome this problem is to aggregate frames over time and over multiple trials, calculating a single independent number that will stand in for many dependent individual observations. The use of aggregation for these ends is characteristic both of the Mirman et al. 'Growth Curve' approach (which uses a proportional transformation) and the quasi-MLR approach (which uses empirical logit).
The question is: how much aggregation is needed to achieve independence?
(read more...)
Posted by Dale Barr on Tue 06 May 2008 at 9:22 PM | Link
Visual world studies investigate hypotheses of the form "Does constraint X influence the processing of linguistic fragment Y?" One necessary feature of such studies is that information corresponding to constraint X must be presented temporally prior to Y. Consequently, there may be effects of X on a participant's looking behavior independently of whether X imposes any constraint on the processing of Y. These anticipatory effects, when not appropriately controlled, can cloud the interpretation of results.
In the MLR paper, I discussed a statistical approach to controlling anticipatory effects; it is also possible to control them experimentally. Here I discuss (as yet unpublished) results from a study investigating whether listeners use knowledge about a speaker's perspective to constrain reference resolution. I show how the inclusion of an appropriate experimental baseline can be used to help untangle linguistic from nonlinguistic effects. The analysis is much simpler and more transparent then the curve fitting exercises that I've explored in the MLR paper or in previous postings.
The data files can be downloaded here [format: zip (Windows) or tarball (UNIX, Mac OS)].
This entry and the corresponding dataset were originally posted on Oct. 9th but were revised on Oct. 18th after I discovered an error in the original data set.
(read more...)
Posted by Dale Barr on Tue 09 Oct 2007 at 11:18 PM | Link | Comments (3)
Update Jan 23 2009: The bug noted below was fixed as of version 0.999375-27 of the R package lme4. It is no longer necessary to use lmer2; instead, you should upgrade to the most current version of lme4 (which as of this writing is 0.99375-28) and use the function lmer.
When running an weighted empirical logit regression, it is suggested to weight each observation by 1/v, where v = 1/(y+.5) + 1/(n-y+.5) (Gart & Zwiefel, 1967; McCullagh & Nelder, 1989). In the current implementation of the R function lmer (lme4 version 0.99875-8), the "weights" option of the lmer function has no effect. (I have tried and verified this result myself). However, the development version of the lmer function, called lmer2, seems to weight the observations properly. So I recommend using lmer2 instead of lmer until this problem is corrected.
Posted by Dale Barr on Tue 09 Oct 2007 at 3:07 PM | Link | Comments (4)
The subset of the design of Kronmüller and Barr (2007) that we will be looking at had a 2x2 design, with Speaker (Same, Different) crossed with Cognitive Load (None, Load). Essential background details for this walkthrough can be found in the MLR paper. You'll also need to download the files from my previous post.
(read more...)
Posted by Dale Barr on Wed 12 Sep 2007 at 1:35 PM | Link | Comments (5)
Analyzing eyetracking data requires handling two sources of nonindependence: the first due to the fact that multiple observations are sampled from the same individual (sampling-based dependencies), and the second arising out of the nature of eye movements (eye-movement-based dependencies). Sampling-based dependencies are easily handled by standard multilevel (a.k.a. "mixed-effects") regression. Handling EM-based dependencies, however, requires special treatment, without which the standard errors in the model can be underestimated—which, in turn, can inflate the Type I error rate. The two remedies I have recommended are (1) logistic regression using 'robust' standard errors and (2) empirical logit weighted regression, which uses aggregation to filter out dependencies.
Here are the files from a re-analysis of Kronmüller & Barr (2007) using empirical logit regression: (tarball) (zip). Details of the reanalysis are reported in the MLR paper.
I will post a 'walkthrough' of the code sometime soon.
Posted by Dale Barr on Sat 08 Sep 2007 at 6:38 AM | Link | Comments (2)
Disclaimer: The purpose of the website
is to offer possible solutions for analyzing data from visual world
experiments, and to encourage discussion/criticism of those solutions.
I am not a statistician, and the advice presented on this website
should be taken as just that—advice. It has not been subject to
peer review, it may contain errors (which I hope that commentary from
others will correct), and may be revised in the future.