The Commonal

Democracy Prep Revisited

UPDATE: Democracy Prep responded. See below.

I like data. Data are great for astrophysics and epidemiology. In education, data give me the willies. There they turn children into numbers. And the growing fixation on numbers reduces complex social and cultural phenomena to points on graphs.

But there are useful data in education, and we should heed them. These include socioeconomic levels, the percentage of youngsters learning English, and rates of disability, to name a few. Schools and districts should scrutinize these.

Other people, like the kind folks at New York charter network Democracy Prep, heed other data. Primarily, they fixate on “achievement,” as measured through standardized tests. They choose to gloss over student population characteristics like the percentage of English language learners (ELLs). Critics say the blindness of charters to student characteristics undermines their claims of “achievement.“

Cue this Twitter exchange, prompted by this New York Daily News article by Ben Chapman revealing that yet again, charter schools in New York City serve fewer students with special needs and English language learners (ELLs). The Twitter tussle involved charter network Democracy Prep, NYC parent-activist Leonie Haimson and Beth Fertig, education reporter for WNYC who has connected high attrition at Democracy Prep with its “no excuses” discipline model. We enter in media res.

Harlem D5 is the district in which DP schools reside.

This volley went unreturned. Seven hours passed. So I served again:

Again, crickets.

Let me reiterate that I don’t love data. Combing through student demographic databases, such as NYSED’s, isn’t how I’d spend a Saturday night. But Democracy Prep, about whom I’ve previously fulminated, basically begged me to do so. And it was a Sunday.

So what do the data tell us about Democracy Prep Charter School (grades 6-11, henceforth DP) and its district, Harlem’s D5? I tracked cohorts entering 6th grade in the years 2006-2007 through 2009-2010 – from DP’s inaugural year to the year with the most recent data. For each, I examined what percentage of the students had disabilities or were English language learners.

Here’s what the average cohort looks like. The blue is the district percentage, red is that of Democracy Prep.

In both cases, the percentage of students in the subgroups decreased markedly within Democracy Prep, while remaining steady district-wide. DP’s disability and ELL rates plunge between 6th and 7th grade, a trend consistent throughout all four cohorts. (Graphs for each cohort are at the end of this post.)

Also note that in no case did DP welcome a 6th grade whose demographics matched the district’s. They begin slightly unequal, and the disparity widens. They are never "higher than the district’s,” as DP claimed.

Here’s how each cohort within DP changed over its full three years. This chart shows a percent change for students with disabilities (blue), and English language learners (red).

(Note: an earlier version of the above was misleading.)

In each cohort, in almost every year for which data was available, Democracy Prep lost significant numbers of students with special needs and ELLs. In the 2009-2010 cohort, 9.5% of the students with disabilities and 6.6% of ELLs disappeared over the course of 3 years.

In short, Democracy Prep Charter School serves a population unlike that of the district, and that gulf grows year by year.  

There’s no cherry-picking of data happening on my end; I used all data available for a 3-year range at this school. Whether there’s cherry-picking happening at the school remains unknown. The data suggest there is, though who knows the actual processes by which Democracy Prep systematically loses the students who score lowest on standardized exams.

DP’s Twitter pilot suggested that the percentage of ELLs drops year by year because these students pass the NYSESLAT, the horrifically acronymed state exam for ELLs. Passing the test relieves a student of her ELL label.

DP’s explanation is problematic for a couple of reasons. First, the cohorts entering DP in 6th grade already consist of fewer ELLs. Second, at no point was the percentage of ELLs at DP 18%, as they claimed. Not even remotely.

Finally, I checked how many students reportedly took the NYSESLAT on DP’s yearly assessment data. Bafflingly, no students were recorded even taking the NYSESLAT at DP between 2006 and 2012. This is so puzzling I won’t even try to impute any nefarious motives, but it certainly gives the lie to claim that DP’s ELL totals dropped because of stellar NYSESLAT performance.

Is it creaming? Is it skimming? Is it some other as-yet-unnamed dairy analogy for jettisoning the students with the highest needs while holding fast to the best test-takers? In my experience, it matters little what you call it. There exist subtle and pervasive pressures in these types of schools to maintain stellar, “miraculous” numbers, and certain students fall victim to this drive. I’d be thrilled if Democracy Prep explained the data I’ve presented, but I’m sure they can wriggle out of it by claiming, quite honestly, “We have no policy of kicking out students because of their needs or test scores.”

I begrudge DP for its disingenuousness, but not for the underlying trends it epitomizes. These trends emanate from the “accountability” movement that holds public schools liable for test scores while turning a blind eye to baldly inequitable demographics. “High-performing” charters like Democracy Prep march as the standard-bearers of the accountability movement simply because they produce high test scores. If our definition of “high-performing” involved maintaining a stable student base while educating high numbers of kids with special needs, we’d have a different set of model schools.

Individual cohort data (excuse the lazy formatting)

Update (July 29, 4:00p)

Whatever its faults, Democracy Prep remains a plucky Twitter presence. After I posted this, the charter network took up its trademark peppy defense.

Let’s take these piece by piece.

1) “Retain in-grade at high level.”

This could be a valid explanation for some dip in the numbers of ELLs or students with disabilities, particularly if they’re held back upon first coming to the school. If this were the case, however, you’d see a jump in the following year’s numbers. So if the 2008 cohort’s ELL population was partly retained, 2009 would show that retention in an above-average percentage of ELLs. But every year, from its initiation onward, Democracy Prep has lower-than-district-average numbers of ELLs and kids with disabilities, and that remains consistent year to year. If the effects of retention were so starkly visible in the dropoff from 6th to 7th grade, they would be equally visible in the following 6th grade class.

When I pushed back on this, DP responded, “Always have higher ELL/IEP in G6 than G8.” This is precisely my point; the trend suggests selective attrition, not retention. It’s remotely possible they meant “All schools always have higher ELL/IEP in G6 than G8.” But this is not the case throughout the district (see charts above).

2) “IEP/ELL removed with strong academic performance”

Before DP responded to my queries, I talked to a friend who previously taught at a different charter. She recalled seeing disability-free kids coming into kindergarten with IEPs. This is unfortunately common, particularly for black boys. The school would carefully assess the students, meet with parents, and phase out the IEP. I figured DP would claim they do something similar.

Alas, they didn’t. When I asked them whether the school really lifts about 40% of students with IEP and ELL designations out of these labels, they responded “ELL-yes, IEP-no.” So, there’s still no explanation for IEPs other than retention, which see above. As for ELLs, this would require students to score proficiently on the NYSESLAT test for English language learners. As I wrote above, the records indicated NO students taking this test at DP between 2006 and 2012. Who knows why.

As for losing an ELL designation, I’m not enough of an expert to know how long it takes for an English-learning youngster to shed the ELL label. Something tells me that a year or two doesn’t suffice, though. I’d love some guidance in this regard.

3) “Mobility very high with poverty”

This argument doesn’t even believe itself. All of the comparisons in this post were between DP and its district, which is roughly as high-poverty as the school. If DP’s diminishing number of challenging students could be explained through mobility, then you’d see the same trends district-wide. You don’t.

DP uses “mobility” as a kind of all-purposes explanation for the trends in their enrollment. But there’s a reason why we compare schools to their home districts. When mobility occurs within a district, it’s students moving around between public schools. When it occurs at a charter, it’s almost always from the charter to the public schools. When that so-called “mobility” sees disproportionate numbers of needy kids leaving charters, it corresponds with disproportionately high numbers of needy kids entering already-stressed traditional public schools.

That’s precisely why I’m making such a fuss.

***

Gary Rubinstein informed me that the indefatigable Juice Fong, internal communications head for TFA, sits on the board of Democracy Prep. Juice had this advice, indebted I think to Lewis Carroll.

I’d be fiffled pink if he could do so.