UCLA Engineering: UCLA Engineer Fall 2004

UCLA Researchers Develop New Testing Tools for Teachers
Automated System Enables Consistency, Fairness in Scoring

By Marlys Amundson

Preschool and elementary teachers at several Los Angeles area schools may soon have more time available for teaching, while still meeting national and state-mandated educational priorities.

Experts in engineering and education at UCLA and other universities are developing a child-friendly testing system that measures and analyzes children’s reading and pronunciation skills over time. They are working closely with elementary school teachers in the community to design an effective assessment system.

TBall team members (from left): Eva Baker, Margaret Heritage, Abeer Alwan, Markus Iseli, Christy Boscardin, Xiaodong Cui and Hong You

“The increased emphasis on testing at the national and state level is leading to less time in the classroom for teaching,” said education professor Eva Baker. “Our proposed system will significantly reduce the amount of time teachers need to spend giving tests, allowing them to focus on what they do best.”

Baker and electrical engineering professor Abeer Alwan are leading efforts to meet the growing demand for early testing of reading and pronunciation skills through an automated assessment system. Their initial efforts will focus on five- to eight-year-old native speakers of American English and non-native speakers of Mexican backgrounds.

“To date, no one has tracked the development of early speech in this group of non-native English speakers,” explained Alwan, a member of the UCLA Henry Samueli School of Engineering and Applied Science faculty. “And yet more than 60 percent of the students in some Los Angeles schools are native speakers of Spanish.”

The Technologically Based Assessment of Language and Literacy project (or TBALL) will allow researchers to tackle several fundamental research issues. For instance, the project’s acoustic and pronunciation modeling algorithms must address not only variability in speech from child to child, but also for a single child over time. It is also more difficult for computers to recognize children’s speech because of their different acoustic characteristics, including higher pitch and resonant frequencies.

The team is devising age-appropriate ways of displaying information to capture a child’s attention and elicit responses. They are also determining what criteria are appropriate to use in scoring the children.

“We’re trying to make a system that brings together fair and accurate testing of children,” explained Baker, director of the Center for the Study of Evaluation. “It’s very different with young children – they have associations that aren’t always clear and they are not consistent in how they see words.”

TBALL builds on a rich history of collaborative activities between engineering and education at UCLA, including improved curriculum feedback mechanisms, the Smart Kindergarten project, and the UCLA/LAUSD Computer Science Institute. The project will draw on the expertise of researchers in electrical engineering, computer science, and education at UCLA; electrical engineering, linguistics and neuroscience at the University of Southern California; and education at UC Berkeley.

UCLA’s Center for the Study of Evaluation is well known for development of scientifically based evaluation and testing techniques, and for promoting the accurate use of technology for improved decision making.

“Currently, a teacher has a series of flash cards or a testing sheet and shows the child an image or word and asks him or her to say it aloud,” explained Alwan. “Then he manually scores the child’s pronunciation. There is little consistency in scoring from teacher to teacher, and the test offers only minimal assessment of a child’s skill level.”

The TBALL team hopes to create consistency in scoring pronunciation, fluency and comprehension levels across classrooms through their automated system. Information derived from the project will also aid educators in determining which teaching methods are most effective with children of varying backgrounds.

“We have a team of advisors in education and linguistics who are helping us determine appropriate scoring criteria,” noted Alwan. “For instance, in some dialects there is no distinction between b and v in Spanish, so the question becomes, ‘Is it an error or an acceptable pronunciation?’”

At UCLA, Alwan is heading efforts in automatic speech recognition and acoustic analysis.

TBall team members (from left): Richard Muntz, Carlo Zaniolo and Yirong Yang

As part of the project, computer science professors Richard Muntz and Carlo Zaniolo are building a longitudinal database that will help teachers track a student’s progress throughout the year and over a multi-year period. The novel database will also support query-based data mining, generating new types and amounts of information and providing critical feedback to educators. This data will help researchers determine if there are early predictors for later performance in reading.

Participants in the project include professors Shrikanth Narayanan and Elaine Anderson at USC, David Pearson at UC Berkeley, and Alison Bailey, Christy Boscardin, and Margaret Heritage at UCLA’s Center for Research on Evaluation, Standards and Student Testing. Patti Price serves as a consultant on the project.

In addition to its impact on testing standards, TBALL also will serve as a vehicle to train graduate and postdoctoral students, as well as elementary school teachers in new interdisciplinary research areas of technological and educational interest.

The TBALL team will receive more than $3 million under a collaborative research grant from the National Science Foundation to support their research. They have also secured Research Experience for Teachers supplements from the NSF to support teacher training. If the platform proves successful, the team plans to create parallel tracks for math and science assessment.

“Our number one priority is to meet the needs of both the teachers and the children,” said Alwan. “It’s something, though, that can only happen through the coming together of technology and education.”

For additional information on the project, please visit http://diana.icsl.ucla.edu/Tball/.