Yesterday’s analysis of exclusion data from the reading portion of this year’s National Assessment of Educational Progress revealed that far too many states were excluding numbers of children in special education ghettos and English Language Learner programs far above what is allowed under federal law. But none of those revelations are a stark as what Dropout Nation learned from analyzing the reading exclusion data from the federal exam’s Trial Urban District Assessment of big-city school systems.
There isn’t much good news to report. Not at all. Fourteen of the 23 districts participating in this year’s edition of NAEP — including Dallas Independent School District, the perpetually-woeful Detroit Public Schools, and even Boston’s reform-minded district — excluded 15 percent or more of fourth-graders condemned to special ed from taking NAEP’s reading test. Eleven of the districts — including Houston Independent School District (a two-time winner of the now-defunct Broad Prize for advancing systemic reform) — excluded more than 15 percent of eighth-graders in special ed.
Meanwhile D.C. Public Schools, which has garnered good press this week from the continuing improvements in achievement on NAEP, excluded as many as 44 percent of ELL fourth- and eighth-graders from the exam. Considering that ELL students accounted for 11 percent of DCPS’ students (as of 2013-2014, the latest year available), the traditional district simply engaged in massive and unacceptable test-cheating.
As your editor wrote yesterday, this form of academic fraud does little more than perpetuate myths that districts are serving children and taxpayers well. And this data is just another reminder that reformers must be eternally vigilant in holding districts and other school operators to account.
This isn’t to say there aren’t any districts on the list who have admirably followed NAEP’s restrictions on excluding 15 percent or more of ELL students and kids in special ed. New York City, for example, excluded just five percent of fourth-graders in special ed, six percent of eighth-graders in special ed, and 11 percent of ELL fourth-and eighth-grade students. Another model district is Albuquerque, which excluded only as much as nine percent of kids in special ed and ELL students. And Hillsborough County, Florida’s district excluded fewer than 11 percent of students in ELL and kids in special ed.
Milwaukee Public Schools managed to achieve the impossible: It excluded no ELL students or kids in special ed from the test. But this is because Milwaukee bowed out of participating in NAEP’s TUDA assessment this year. Probably because the dismal results would be predictable — or, some would say, why help Detroit avoid the status of being the nation’s worst traditional district.
But for the most part, many districts engaged in some rather shameless test-cheating in order to boost their performance.
Dallas topped the Dishonor Roll when it came to excluding kids in special ed. It excluded 44 percent of fourth-grade kids in special ed, leading in that category, and ranked second behind the notorious Baltimore City school system (36 percent), by excluding 29 percent of eighth-graders who were special ed and had other disabilities. Dallas’ test-cheating, along with that of Houston and Austin, explains why Texas was a leading state in excluding these most-vulnerable children from NAEP this year.
Detroit Public Schools took third place this year in excluding fourth- and eighth-graders in special ed. It kept 35 percent of fourth-graders in special ed from taking NAEP reading, while refusing to let 27 percent of eight-graders take the test. The exclusions didn’t exactly help the perpetually-failing district improve its performance. Seventy-three percent of fourth-graders in Detroit read Below Basic on NAEP 2015, three percentage points worse than two years earlier, while 56 percent of eighth-graders were functionally illiterate, two percentage points worse than in 2013.
When it came to excluding ELL students, no district did so with less shame than D.C. Public Schools, which got plenty of high-fives from reformers this week for improving performance on both NAEP and its Common Core-aligned exams. The district excluded 44 percent of eighth-grade ELL students from the exam, topping that category; this is 19 percentage points higher than its exclusion levels on NAEP reading two years ago. DCPS also came in second behind Jefferson County, Ky. (25 percent) in excluding fourth-graders in ELL, keeping one out of every five such students from taking the test.
Houston Independent School District was particularly shameful on the NAEP exclusion front this year. It excluded 25 percent of eighth-graders in special ed from the reading portion of this year’s NAEP, ranking fourth after Baltimore City, Dallas, and Detroit; it also excluded 20 percent of fourth-grade students condemned to special ed from NAEP this year, ranking fifth after seventh on that particularly lengthy dishonor roll. This isn’t shocking. As with Baltimore City and Jefferson County, Ky., Houston was one of the worst offenders on Dropout Nation‘s list two years ago.
Meanwhile two districts — Miami-Dade, and Philadelphia — managed the dubious honor of being leaders in all four categories of excluding our most-vulnerable children from NAEP this year. Miami-Dade raced to the bottom by being 10th place in excluding fourth-grade kids in special ed, third in excluding ELL fourth-graders, fifth in excluding eighth-graders condemned to special ed, and second place in keeping eighth-graders in ELL from taking the test. For Miami-Dade, excluding ELL students is especially egregious because those students made up 20.3 percent of the district’s enrollment (as of 2013-2014).
The virtually-busted School District of Philadelphia ranked fourth in excluding fourth-graders in special ed, sixth in excluding ELL fourth-graders, sixth in excluding eighth graders condemned to special ed, and third in excluding ELL eighth-grade students. Kids in special ed made up 16.6 of Philly’s student population in 2013-2014, while those in ELL accounted for 8.6 percent of students.
All of this matters because evidence — including research from Edward Haertel of Stanford University — shows that high exclusion rates can make a tremendous difference in performance. The more ELL students and kids in special ed excluded, the better the performance. For example, in an analysis by the U.S. Department of Education of exclusion rates, requiring all ELL students and kids in special ed to take NAEP in 2013 would have resulted in as much as an 8.5 point decline (or a nearly full-grade decrease) in average scales scores on the exam for eight of the participating cities.
In excluding fourth-grade kids in special ed, Dallas, Baltimore City, Detroit, Philadelphia (22 percent), and Jefferson County are joined by Atlanta (22 percent), the aforementioned Houston, Los Angeles Unified (19 percent), San Diego Unified (19 pecent), Boston (18 percent), Cleveland (18 percent), Miami-Dade (17 percent), and Duval County, Florida (15 percent). As mentioned at the top, 14 districts excluded 15 percent or more of fourth-graders in special ed from taking this year’s federal test.
On the fourth-grade ELL exclusion, Jefferson County and D.C. Public Schools were followed by Miami-Dade (20 percent), Cleveland (18 percent), Charlotte-Mecklenberg County (16 percent), and Philadelphia (15 percent). Altogether, six districts excluded 15 percent or more of their ELL students from the exam.
When it comes to eighth-grade special ed exclusion, Baltimore City, Dallas, Detroit, and Houston were followed by Miami-Dade (19 percent), Philadelphia (18 percent), Fresno Unified (18 percent), Jefferson County (17 percent), Boston (16 percent), and Atlanta Public Schools (15 percent). Altogether, 10 districts excluded 15 percent or more of eighth-graders in special ed.
On excluding eighth-grade ELL students, D.C. Public Schools was followed by Miami-Dade (26 percent), Philadelphia (22 percent), Cleveland (22 percent), and Charlotte-Mecklenberg (16 percent). Five districts, altogether, excluded 15 percent or more of ELL eighth-graders from NAEP this year.
All in all, the test-cheating is indefensible. It must stop. Ideally, states would crack down on these exclusions. But given that the states benefit as much from this test-cheating as the districts they oversee, this isn’t possible. So it is up to the federal government, particularly the U.S. Department of Education, to hold the line on these exclusion levels. Highlighting the bad actors in NAEP data would be one key step on that front.
But as it has been stated before, reformers must be vigilant in holding all districts, especially those reform-minded ones favored by the movement, accountable for their exercises in academic fraud.