Does Testing Deserve a Passing Grade?

▪ 2002

Introduction

Charles D. Claiborn

High-Stakes Testing.

      As the term suggests, high-stakes testing is the use of educational and psychological tests to make decisions of often considerable consequence to individuals and institutions. Some tests assess the achievement or competencies of students at specific grade levels to determine whether they should be advanced to the next grade or, upon completing the secondary-school curriculum, be awarded a high-school diploma. Results of these tests additionally may be taken as an indicator of how well particular schools are educating their students and may in turn be used in allocating resources to schools or determining whether changes in their governance are warranted. Other tests assess the aptitude of applicants to be successful in college or graduate school and are used to make admissions decisions that dramatically affect the educational and professional futures of individuals. The differential impact these tests have on various racial, ethnic, and socioeconomic groups makes high-stakes-testing practices highly controversial.

Characteristics of High-Stakes Tests.

      According to some, high-stakes tests are “cognitively loaded” in that they measure the primarily cognitive constructs of knowledge and skill and, in some cases, potential or aptitude for gaining further knowledge and skill. The tests are also standardized— developed according to accepted practices of test development, such as those put forth jointly in 1999 by the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education—and have thus been validated for their intended purpose and normed for populations with which they will be used. The psychometric adequacy of a test depends on the extent to which these practices have been followed.

      The validity of a test is the adequacy of the test to perform a specific function. The types of validity that should be established for high-stakes tests thus vary according to the function of the test. For competency tests, such as minimum-competency tests used for grade advancement or graduation decisions, content validity is of particular concern, since it is important for the test to represent a designated domain of knowledge and skill adequately. A content-valid test of 10th-grade mathematics knowledge and skills, for example, is one that fairly and representatively reflects the range of mathematics topics and problems learned in the 10th grade, as determined by professionals in the area and, in some cases, the public at large. Different interest groups—a teachers union and a state legislature, for example—may naturally have different ideas about what a particular test should contain and who should determine that content. Content validity of competency tests can clearly be a source of controversy.

      A second type of validity, criterion-related validity, is important for tests used in the selection of students. The value of a college entrance examination, notably the ACT (American College Testing Program) or SAT (Scholastic Assessment Test), depends on its ability to predict academic performance, which is the criterion of interest. The usefulness of any test for screening or selecting applicants for a position is based on the test's ability to predict job performance, the criterion in this case. It would be highly problematic, scientifically and legally, if a test used for selection or screening of applicants measured something that was not clearly related to criteria of school performance. The test-criterion relationship is the very heart of validity for this sort of test. It would also be problematic if the relationship between test scores and performance differed for different groups within the population, such as ethnic minority groups. The use of a test in such circumstances would constitute bias, though some experts have indicated that standardized tests used in selection do not generally suffer from this sort of distortion.

High-Stakes Testing in Selection—the Diversity Dilemma.

      Even when high-stakes tests have established validity, they are still open to controversy, especially with respect to issues involving ethnic diversity. In a recent review it was argued that the weight of the scientific evidence supports the validity of high-stakes tests used in selection. Standardized tests of knowledge and skill are indeed effective in predicting performance, at least within the cognitive domain. However, the authors of the review and others have also noted the well-established findings that African Americans and Latinos consistently score lower than whites on such tests and that Asian Americans score higher than whites on measures of quantitative ability and lower than whites on measures of verbal ability. Such ethnic-group differences are typically confounded with socioeconomic status, with members of lower socioeconomic groups typically scoring lower on such tests than members of higher socioeconomic groups. Nevertheless, such findings present a dilemma, that of choosing between the goal of using the most valid tests—those making the best predictions of performance—and the goal of having a more diverse student body or workforce. Several ways of resolving this dilemma have been proposed, though none has been researched thoroughly enough to merit recommendation.

Competency Assessment in Education.

      The widespread and growing use of competency assessment in schools brings high-stakes testing into the public and political spotlight. Minimum-competency tests are now used in some 23 states to determine grade advancement and graduation. In December the U.S. Senate passed a landmark education bill that would require mandatory annual state math and reading tests for all students in grades three through eight. In addition, the results of such tests are used to assess the performance of teachers, schools, and school districts and for this reason are made available to the public and are subject to scrutiny by state legislatures and agencies. The rationale for state-mandated minimum competency testing is generally to hold teachers and schools accountable for the education they are providing and to improve education by holding education professionals to a higher standard, namely, that imposed by the state. The practical effect of such practice is to reward those teachers and schools who do well, through financial incentives and public recognition, and punish those who do not.

      Criticism of minimum-competency testing as a means to improve education has been considerable. First, there is little evidence to suggest that such testing really improves education. A 1990 report found that the use of minimum-competency tests is associated with higher dropout rates, though the reason for this is unclear. A number of researchers have documented the negative effects of minimum-competency testing on the curriculum and instruction. These include narrowing the curriculum to what is covered on the test (“teaching to the test”), taking time away from instruction in order to prepare students for the test, and limiting instruction to the types of knowledge and problem solving required by the test format (for example, emphasizing the recognition of information as emphasized on multiple-choice tests). Second, minimum-competency-testing policies often take important educational decisions away from professional educators and place them in the hands of those with little or no expertise—legislators or school-board members. These include decisions about test content and format as well as about standards for passing and the consequences of failure. Inexpert decisions about test development and use can undermine test validity and make unfair testing practices more likely. Third, minimum-competency tests, like all high-stakes tests, have a disproportionately negative impact on ethnic-minority students, students from lower socioeconomic groups, and students with learning disabilities. Finally, though the rationale for minimum-competency testing is to improve education, the focus of testing is often not in line with the instructional goals of particular teachers and schools. Consequently, the results of such tests are not particularly useful as feedback regarding how well teachers and schools are meeting the goals they set for themselves and their students. Though the use of minimum-competency testing has considerable public relations value by appearing to provide hard data on how well or how poorly schools are doing—with an accompanying set of high standards to which students, teachers, and schools are held—the reality of such testing falls short, in regard to both the flawed tests themselves and the often unhelpful, even hurtful, use to which the test results are put.

Charles D. Claiborn is professor of psychology in education at Arizona State University. He is the author of more than 60 articles and book chapters on psychotherapy process, including “Feedback” in Psychotherapy Relationships That Work: Therapists' Relational Contributions to Effective Psychotherapy.

* * *