The potential for system-friendly K-12 reform.

• Author: Merrifield, John
• Position: Report

Numerous empirical models connect individual student test scores or average test scores to theoretically plausible policy and socioeconomic variables. Although the models were created to test for the effect of a specific factor like funding levels or teacher training on student performance, the fully specified models also have important implications for the ability of the current K-12 school system to significantly improve its performance. This article examines various student performance models and the potential for systemfriendly K-12 reform.

Substituting potential, best-case values of independent variables into fully specified models of average student performance allows one to discover how much improvement might be achieved within the constraints imposed by factors that cannot be represented by explanatory variables. The fixed factors are the governance and funding processes of the school systems that generated the numbers that are the basis for the parameter estimates. The specific aim of this article is to reveal what existing student performance models say about the potential for improvement, and thus what they say about the significance of the fixed factors.

This is a suggestive assessment of the qualifying studies. Since the major school reform policy issues are aggregate performance measures like average test scores, I largely excluded models with individual student performance as the dependent variable. I mention the two that assess the significance of determinants of the National Assessment of Education Progress (NAEP) scores that are the most trusted consistent measures of student skill. Since my concern is the absolute level of test scores, I 'also excluded "value-added" models-that is, models that explain changes in performance. Incomplete documentation of results or unavailability of one or more variables means the forced exclusion of some otherwise qualifying studies. (1)

The next section uses hypothetical highly favorable (definition depends on available data) values for the independent variables to solve the regression equations for the predicted average level of each qualifying study's student performance measure. The result is a best-case outcome estimate of the performance variable. I also make a second performance variable estimate with control variables at their median or mean values, and only the policy variables at their highly favorable levels. The third section assesses the predicted best-case values of the student performance variables and discusses interpretation issues. For example, suppose the predicted maximum NAEP score is 300. The total possible score on a NAEP exam is 500, so a score of 300 is only 60 percent. This score is in the "advanced" range for fourth grade math and reading scores, but only in the "proficient" range for eighth grade scores. (2) Indeed, the hypothesis that motivated this article is that large gaps between the predicted "best case" and the total possible score will be the norm. The fourth section discusses some of the costs of changing real-world average values to the highly favorable values inserted into the equation. The final section provides some concluding remarks.

Predicted Maximum Student Performance

Blair and Staley (1995) used data for 266 Ohio urban school districts. Their dependent variable (SCORE) is a three-year (fourth, sixth, and eighth grades), three-test (reading, math, and language arts) district average composite test score ranging from zero to 100. Multicollinearity problems prompted them to omit some theoretically plausible variables. The model that included the remaining explanatory variables is

(1) SCORE = 20.98 + 0.34SAL- 0.13PTRATIO + 0.15AGI -3.73MINOR- 17.28ADC + 0.43NEIGHBOR - 0.80COUNT.

Where (highly favorable value = average +/- 2 standard deviations, in parenthesis):

SAL = Average teacher salaries in thousands (38.53)

PTRATIO = Pupil-teacher ratio (14.4)

AGI = Family adjusted gross income, thousands (57.35)

MINOR = % Minority (0 > av. - 2 s.d.)

ADC = % getting "Aid to Families with Dependent Children" (0 > av. - 2 s.d.)

NEIGHBOR = Average SCORE of contiguous school districts (66.7)

COUNT = Number of adjacent school districts (9.66 = av. + 2 s.d.).

To avoid the bias that might result from excluding insignificant variables, the predicted academic performance calculations exclude only variables with an implausibly signed coefficient. So, in the Blair and Staley (1995) model, only COUNT did not factor into the estimate of the predicted best-case value of SCORE. Substituting the highly favorable values into all of the independent variables with plausibly signed coefficients yields a predicted, best-case SCORE = 69.5 percent. With the control variables AGI, MINOR, and ADC set at their average values, and only the policy variables set at highly favorable values, the predicted best-case SCORE = 58.7 percent.

Gamrat (2002) examined 1999 Pennsylvania System School Assessment (PSSA) scores for the fifth, eighth, and eleventh grades. His model is

(2) PSSA = 1,259.86007 + 0.00735OEPPS - 2.37297LI -0.00012ENR + 0.00086ATS + 1.45744SPCT - 20.87379DLMA + 1.24332JU.

Where (highly favorable = actual max or min, in parenthesis):

OEPPS = Operating expenditures per pupil spending ($13,170)

LI = Percentage of low-income students (3.9)

ENR = District enrollment (1,715)

ATS = Average teacher salary ($64,338)

SPCT = Students per classroom teacher (12)

DLMA = Whether or not the district is located in a metro area (0)

JU = District gives its teachers the option of joining a union (1).

Solving for PSSA with the designated highly favorable values, the predicted average PSSA score is 1,403.8 out of a total possible score of 2,200 to 2,500; (3) roughly in the middle of the range of scores Pennsylvania now defines as proficient. With only the policy variables set at their highly favorable values, the predicted PSSA is 1,328.6, which is in the lower end of the range Pennsylvania currently defines as proficient.

Mensah, Schoderbek, and Werner (2005) used 2000-2001 New Jersey school district data to specify models for per pupil expenditure and test scores at various levels. Since the most important test score is the one for the oldest children, I report only the TEST3_AV model that is, district average scores for combined language and mathematics for the eleventh grade. The model is

(3) TEST3_AV = 5.923 + 0.01YR01 - 0.022POOR + 0.002STU_POP + 0.071EXPPP + 0.195CS_INST - 0.171CS_ADMIN - 0.302CS_OPMAIN + 1.124CS_EXTCUR - 0.054ABBOT_DIST - 0.022LOWING - 0.150COSTIDX + 1.325ATTD_RATE.

Where (highly favorable values = observed row: or rain, in parenthesis):

YR01 = Dummy variable that equals one for year...