Contributed by: RAND
RAND collaborated with the researchers from the University of California,
Santa Barbara; the Far West Laboratory; and the California State
Department of Education to develop and administer several science
exercises to students in elementary, middle, and high schools in
1993 and 1994.
This hands-on performance task was administered in various combinations
to samples of 5th graders in California during 1993 and 1994. The
task was administered by project staff under controled conditions.
Portable partitions were used so that students could not observe
or otherwise interact with one another during the test sessions
unless working in a group portion of the testing. Testing often
occured in the cafeteria or other large room at the school so that
each student had enough space to work with a task's equipment and
materials.
Using the scoring guides provided, RAND was able to achieve a high
degree of agreement between readers using these scoring guides.
The mean correlation between two readers on a hands-on task ranged
from 0.83 to 0.98. The mean was 0.95. The exceptionally high agreement
probably stemmed from a combination of factors, including designing
and testing the scoring rubrics in conjunction with task development,
having several separately scored segments within each measure, the
use of detailed semi-analytic scoring guide, reader selection and
retention policies, and extensive reader training and supervision
by project staff.
Incline was administered to a total number of 1,100 fifth grade
students in 1993 for purposes of field testing, task development
and refinement.
|