A Question of Evidence: Does I3 Rulemaking Reveal Weakening of the Obama Administration’s Reform Efforts?
One of the arcane aspects of federal education policy is the structuring of competitive grant programs such as the Obama administration’s Race to the Top. Key to structuring the grants are the rules, called General Administrative Rules (or EDGAR), which can be just as important in shaping how grants are awarded (and who benefits). For reformers, especially Beltway types, redlining (or crafting) EDGAR can be as important as structuring the competitive grants themselves.
So one can expect that plenty of attention is being given to a series proposed amendments to the current EDGAR rules released to the public by the Department of Education this past month. Most of the proposed rule changes are innocuous or seemingly so, including a requirement that future rounds of Race to the Top and other competitive grants are reporting performance measures as required by the Government Performance and Results Act of 1993. But one key change, focused on the evidence needed by potential reform efforts to be considered for another key Obama Administration initiative — the Investing in Innovation Fund — will get more attention.
Currently, the Obama administration can only give consideration to proposed I3 projects that have “strong evidence” of “statistically significant” improvement on student achievement (including stemming achievement gaps) provided through at least one study that meets the research design standards of the federal What Works Clearinghouse (the federal center for determining scientific validity of educational research) “with or without reservations”. Those proposals that didn’t meet the standard could not get funding. Under the proposed rule, the Obama administration would be able to give special consideration to proposed projects that are either supported by “strong” or “moderate” evidence of effectiveness.
What is that evidence? The current standard for strong evidence largely remains and in fact, now allows for two studies that meets federal What Works Clearinghouse standards. But now, a proposal can also site “moderate evidence” that includes one study that meets those standards “with reservations” and uses a sample that overlaps with the populations that would be served by the project through I3 funding. On its face, the new rules gives the Obama administration more flexibility in funding projects through I3 that may otherwise not meet its current standard. But the impact may lie for future rounds of I3 and other grant competitions beyond this year. Why? Because as part of the proposed rule, the administration lays out four definitions of levels of evidence. This includes something called “Strong Theory” which could allow for a project with a “logic model” demonstrating potential success to gain funding in future rounds, as well as “Evidence of Promise, under which a grant proposal can prove its worth with a “quasi-experimental” study that meets What Works Clearinghouse standards with reservations. Depending on what the Obama administration decides to do, this could open the door to making I3 grants less competitive efforts than simply funding any scheme it wants.
For reformers, the proposed I3 rule may be both troubling and opportunity. The fact that I3 may become a sort of piggy bank for reforms the Obama administration favors without regard to efficacy should force reformers and congressional leaders to demand the administration to put more stringent competitive grant processes in place. At the same time, the administration’s move to lay out standards of evidence should be appreciated; other competitive and formula grant programs should have at least that much. Either way, this will get interesting.