Monday, February 11, 2008

Conservatism small c-

Sol Stern has an important article in the current City Journal on the limits of school choice.

Recent developments in both public and Catholic schools suggest that markets in education may not be a panacea—and that we should reexamine the direction of school reform.

One such development: taxpayer-funded voucher programs for poor children, long considered by many of us to be the most promising of education reforms, have hit a wall. In 2002, after a decade of organizing by school choice activists, only two programs existed: one in Milwaukee, the other in Cleveland, allowing 17,000 poor students to attend private (mostly Catholic) schools. That year, in Zelman v. Simmons-Harris, the Supreme Court ruled that limited voucher programs involving religious schools were compatible with the First Amendment’s establishment clause. The 5–4 decision seemed like school choice’s Magna Carta. But the legal victory has led to few real gains. Today, fewer than 25,000 students—compared with a nationwide public school enrollment of 50 million—receive tax-funded vouchers, with a tiny Washington, D.C., program joining those of the other two cities.

Proposals for voucher programs have suffered five straight crushing defeats in state referenda—most recently in Utah, by a margin of 62 percent to 38 percent. After each loss, school choice groups blamed the lobbying money poured into the states by teachers’ unions, the deceptive ads run by voucher foes, and sometimes even voters’ commitment to their children. When the Utah results came in, the principal funder of the pro-voucher side, businessman Patrick Byrne, opined that the voters failed “a statewide IQ test” and that they “don’t care enough about their kids.” If vouchers can’t pass voter scrutiny in conservative Utah, though, how probable is it that they will do so anywhere else? And denouncing voters doesn’t seem like a smart way to revive the voucher cause.

Voucher prospects have also dimmed because of the Catholic schools’ deepening financial crisis. Without an abundant supply of good, low-cost urban Catholic schools to receive voucher students, voucher programs will have a hard time getting off the ground, let alone succeeding. But cash-strapped Catholic Church officials are closing the Church’s inner-city schools at an accelerating rate [see “Save the Catholic Schools!,” Spring 2007]. With just one Catholic high school left in all of Detroit, for instance, where would the city’s disadvantaged students use vouchers even if they had them?

Even more discouraging, vouchers may not be enough to save the Catholic schools that are voucher students’ main destination. Archbishop Donald Wuerl of Washington, D.C., recently announced plans to close seven of the district’s 28 remaining Catholic schools, all of which are receiving aid from federally funded tuition vouchers, unless the D.C. public school system agreed to take them over and convert them into charter schools. In Milwaukee, several Catholic schools have also closed, or face the threat of closing, despite boosting enrollments with voucher kids.

During the 15 years since the first voucher program got under way in Milwaukee, university researchers have extensively scrutinized the dynamics of school choice and the effect of competition on public schools. The preponderance of studies have shown clear benefits, both academically and otherwise, for the voucher kids. It’s gratifying that the research confirms the moral and civil rights argument for vouchers.

But sadly—and this is a second development that reformers must face up to—the evidence is pretty meager that competition from vouchers is making public schools better. When I reported on the Milwaukee voucher experiment in 1999, some early indicators suggested that competition was having just that effect. Members of Milwaukee’s school board, for example, said that voucher schools had prompted new reforms in the public school system, including modifying the seniority provisions of the teachers’ contract and allowing principals more discretion in hiring. A few public schools began offering phonics-based reading instruction in the early grades, the method used in neighboring Catholic schools. Milwaukee public schools’ test scores also improved—and did so most dramatically in those schools under the greatest threat of losing students to vouchers, according to a study by Harvard economist Caroline Hoxby.

Unfortunately, the gains fizzled. Fifteen years into the most expansive school choice program tried in any urban school district in the country, Milwaukee’s public schools still suffer from low achievement and miserable graduation rates, with test scores flattening in recent years. Violence and disorder throughout the system seem as serious as ever. Most voucher students are still benefiting, true; but no “Milwaukee miracle,” no transformation of the public schools, has taken place. One of the Milwaukee voucher program’s founders, African-American educator Howard Fuller, recently told the Milwaukee Journal Sentinel, “I think that any honest assessment would have to say that there hasn’t been the deep, wholesale improvement in MPS [Milwaukee Public Schools] that we would have thought.” And the lead author of one of the Milwaukee voucher studies, Harvard political scientist Paul Peterson, told me: “The research on school choice programs clearly shows that low-income students benefit academically. It’s less clear that the presence of choice in a community motivates public schools to improve.”

Then see Jay P. Greene's rebuttal here.

Stern claims that “the evidence is pretty meager that competition from vouchers is making public schools better.” This cuts to the heart of the matter: do competitive pressures from vouchers or other incentive reforms improve the performance of traditional public school systems? To answer that question, Stern considers whether Milwaukee and New York City, places where he suggests that different incentive reforms have been adopted, show positive results.

Since Stern’s entire case hinges on this claim, it is surprising that he musters such weak evidence and neglects a considerable body of rigorous research. The crudest and least persuasive analysis of whether competition improves public school performance is simply to assess the overall performance of places where competition has been expanded, neglecting to control for other factors that influence school performance. After all, it’s always possible that things would be even worse if not for competition.

For Milwaukee, Stern references his only rigorous study, by Caroline Hoxby, and acknowledges that “Milwaukee public schools’ test scores also improved—and did so most dramatically in those schools under the greatest threat of losing students to vouchers.” But then, Stern claims, those gains “fizzled.” The only evidence that he presents on this central point is that, even after 15 years of school choice, “Milwaukee’s public schools still suffer from low achievement and miserable graduation rates, with test scores flattening in recent years.” But of course this neglects the question of whether things would have been worse if not for expanded competition.

A large body of research exists on this issue. Rajashri Chakrabarti, an economist at the Federal Reserve Bank of New York, extended Hoxby’s initial analysis. Rather than finding that gains had fizzled, as Stern claims, Chakrabarti concludes: “Using a difference-in-differences analysis in trends and Wisconsin data from 1987 through 2002, the paper shows that these shifts led to a much larger improvement in the second phase compared to the first phase. This result is robust to alternative samples and specifications, and survives several sensitivity checks.” That is, the benefits of competition on Milwaukee public school performance didn’t fizzle: they increased after 1998, when the program expanded.

Milwaukee is not an isolated instance of incentive reforms’ producing significant improvement. In Florida, where wide-ranging incentive reforms have been in place for several years, there have been four rigorous analyses of the effects of competitive pressure on the public school system. All four, from groups as disparate as the Manhattan Institute and the Urban Institute, agree that public schools made exceptional improvements in response to competitive pressure. (Read them here, here, here, and here.) Oddly, Stern makes no mention of Florida’s experience. He does not perform the crude analysis, which would show that overall Florida student achievement has been increasing since incentive reforms were under way, nor does he refer to any of the four rigorous analyses. If Stern means to conclude that competition has yielded “disappointing results,” one would think he’d have to address this evidence.

Rather than discuss Florida, Stern chooses to highlight New York City to support his argument. This is especially curious because New York has only recently pursued incentive reforms (such as merit pay for principals and teachers), has no voucher program, and has only a modest-sized charter effort. The merit-pay programs haven’t even been implemented yet, let alone subjected to rigorous evaluation. Stern’s comparison of NAEP scores between 2003 and 2007 can’t possibly speak to the effects of programs not yet in effect. Stern somehow knows the verdict not only before the jury has heard the evidence, but even before the crime has been committed.

In addition to evidence about the competitive effects of voucher programs, studies done in Arizona, Michigan, and Texas show that competition from charter schools improves the academic performance of nearby traditional public schools. A fairly large body of research also exists on the effects of public school districts’ competing with each other. Clive Belfield and Henry Levin at Teachers College, no friends of school choice, conducted a systematic review of over 200 analyses in that literature, concluding: “The above evidence shows reasonably consistent evidence of a link between competition (choice) and education quality. Increased competition and higher educational quality are positively correlated.”

Paul Peterson’s comment in Stern’s article—“It’s less clear that the presence of choice in a community motivates public schools to improve”—is relative to the evidence that we have on the effects of choice on those who participate in those programs. On the latter issue, we have ten analyses of random-assignment experiments (the gold standard of research design), nine of which show significant benefits for at least some subgroups of students who get to choose private schools. Understood in context, Peterson is not denouncing the quality of evidence showing the benefits of competition for public schools. He’s simply emphasizing the remarkably high-quality evidence available on participant effects.

Given multiple studies showing positive competitive effects from voucher programs in Milwaukee, several showing the same in Florida, several positive results from studies of charter competition, and more than 200 analyses of competition among public schools, Stern is simply mistaken to assert that incentive reforms fail to produce improvements by public schools—especially when he fails to discuss all but one of those studies.

02/11 10:47 AM