Eyetracking

Plate Tectonics

Ideal learners are active, self-motivated, critical, inquisitive, and they search for answers to thought-provoking questions. However, research over the past 25 years indicates that most students do not become ideal learners in typical school settings. There is ample evidence that people do not typically engage in explanatory reasoning, nor do they seek a deep, complex understanding of a topic. Especially in schools, tasks are often constructed to emphasize shallow and simplistic memorizing rather than learning with understanding through critical reading and reasoning processes. What is needed is for learners to take a "critical stance" as opposed to a passive approach to learning. For example, the internet includes a huge amount of information on a vast array of topics, but unfortunately the reliability or creditability of this information is often questionable. An ideal learner will ask questions such as "What evidence is there for this claim on this website?" "Is this claim really correct?" and "Does the creator of this website have any deep expertise?" In such circumstances, it is vital to understand how, when, and why readers adopt a critical stance toward scientific information. Given an understanding of how critical thinking develops, interventions can be developed to train students how to become more critical thinkers.

Researchers at the University of Memphis and the University of Illinois, Chicago are working together to understand critical thinking in the context of locating information on the World Wide Web. In one experiment, participants were asked to learn about plate tectonics by searching various web sites. The web sites varied in terms of the reliability of the site; some were reliable (e.g., hosted by NASA) while others were unreliable (e.g. a site that claims that oil drilling causes earthquakes). Participants were asked to write essays based on their searches and to rate the quality of the sites. The eye-tracker was used to determine which pages were visited, and what types of information on a particular page was attended to. Assessment of the eye-fixations allows us to track the differences between efficient critical learners and inefficient passive learners. This information can be used to develop training programs designed to facilitate critical thinking. This project was funded by the National Science Foundation. The project web site can be reached at: http://mnemosyne.csl.psyc.memphis.edu/Plate_Tectonics/index.htm

AutoTutor

In an ideal learning situation, the teacher interacts with the student on a one-on-one basis and tailors the learning environment to suit the student's special needs. However, in traditional public education, this "idealized" form of training is not feasible due to limited funding and high student-to-teacher ratios. In addition, most families cannot afford to send their children to private schools, or to hire specialized tutors. The tutoring research group at the University of Memphis has developed a web based computer tutor called AutoTutor that can be used for a variety of subject areas. AutoTutor simulates the dialog of effective human tutors in conversationally appropriate ways. The Tutoring Research Group has developed two versions of AutoTutor, one for computer literacy and one for conceptual physics. The computer literacy AutoTutor is designed to help students learn basic computer literacy topics covered in an introductory course (e.g., hardware, operating systems, and the Internet). The conceptual physics AutoTutor is designed to help students learn Newtonian physics. AutoTutor works by having a conversation with the learner. AutoTutor appears as an animated agent that acts as a dialog partner with the learner. The animated agent delivers AutoTutor's dialog moves with synthesized speech, intonation, facial expressions, and gestures. Students are encouraged to articulate lengthy answers that exhibit deep reasoning, rather than to recite small bits of shallow knowledge. The eye tracking system is used to both evaluate and improve the AutoTutor software. This project was funded by the Department of Defense, the Office of Naval Research, and the National Science Foundation. You can learn even more about AutoTutor by visiting the web site at www.autotutor.org.

QUAID

People are routinely bombarded by questions in conversations, legal situations, surveys, testing and everyday activities. Recent research has shown that the manner in which a question is asked can have a dramatic impact on how it is answered. QUAID is a software tool that assists survey methodologists, social scientists, and designers of questionnaires in improving the wording, syntax, and semantics of questions. The tool identifies potential problems that respondents might have in comprehending the meaning of questions on questionnaires. These problems can be scrutinized by researchers when they revise questions to improve question comprehension and, thereby, enhance the reliability and validity of answers. QUAID detects five problems: unfamiliar technical term, vague or imprecise relative term, vague or ambiguous noun phrase, complex syntax and working memory overload. The output of QUAID was compared with ratings of language experts who evaluated a set of questions on the five classes of problems. The system was tested on 505 questions with 11 surveys developed by the US Census Bureau. Analyses revealed that QUAID was able to identify each of the five problems with questions. The eye-tracking system can be used to identify the parts of question where people spend their time and what parts they skip. Obviously, the validity of an answer is compromised if important words in the question are never read. QUAID in conjunction with eye tracking can be used help isolate the areas of the question that pose the difficulties. For example an eye-tracking experiment was conducted on a series of questions from the Census Bureau. Participants tended to focus on nouns that QUAID considered problematic, but tended to skip adjectives and adverbs that were problematic. When QUAID considered a question to contain complex syntax or pose a heavy load on one's working memory, participants tended to skip the questions, demonstrating an early exit strategy. In sum, the valuable information gained from the eye-tracker can be used to both improve, and validate the QUAID tool. The project has been funded by the National Science Foundation, the Office of Naval Research, and the U.S. Census Bureau. You can access the tool and learn more by going to the QUAID homepage (http://mnemosyne.csl.psyc.memphis.edu/QUAID/aboutquaid.htm).

The Way Things Work

In everyday life, we routinely come across information that is presented in multimedia rather than a single medium. Printed texts, pictures, diagrams, and semiotic symbols are pervasive, as in the case of Web-pages, TV programs, textbooks and instructions. Researchers in cognitive science and educational technology are currently investigating a large number of questions about multimedia processing. How do people capture information from these different media? How do people construct a coherent understanding of the information? Do people with different cognitive abilities and personality traits vary in the ways they extract information from the multimedia? Do people who achieve different levels of understanding manifest differences in how they extract and integrate information from the multimedia? As cognitive scientists, we asked ourselves how we could provide answers to these questions.

Our recent research has documented that an excellent way to demonstrate understanding of technical content is to ask good questions when confronted with obstacles or "cognitive disequilibrium." In our studies, participants read illustrated texts on everyday devices (cylinder lock, electrical bell, car temperature gauge, dishwasher, and toaster), and subsequently diagnosed hypothetical breakdown scenarios (e.g., the key turns but the bolt does not move in the context of a cylinder lock) by generating questions that address the malfunctions. Eye movements were recorded throughout the comprehension and problem-solving phases. Afterwards, participants also received a device comprehension test and a battery of tests assessing individuals. We found that good comprehenders asked questions that focused on the salient aspects of the breakdown scenarios and that they examined the individual parts before they verbalized their questions. We also found that different media played somewhat different roles in information acquisition, integration and utilization. For example, printed texts guide the initial comprehension, while the pictures plays an increasingly important role as comprehension progresses. This project was funded by the Office of Naval Research.