Problems With Value Added Assessment by Diane Ravitch

January 18, 2011 The Pitfalls of Putting Economists in Charge Of Education/Building Bridges Blog

Dear Deborah,

A few weeks ago, Mike Rose posted a list of his New Year’s resolutions. One of them was that we should “make do with fewer economists in education.

These practitioners of the dismal science have flocked to education reform, though most know little about teaching and learning.” Mike suggested that so few economists were able to give useful advice about the financial and housing markets that we should now be skeptical about expecting them “to change education for the better.”

 

I agree with Mike. It is astonishing to realize the extent to which education debates are now framed and dominated by economists, not by educators or sociologists or cognitive psychologists or anyone else who actually spends time in classrooms. My bookshelves are chock full of books that analyze the teaching of reading, science, history, and other subjects; books that examine the lives of children; books that discuss the art and craft of teaching; books about the history of educational philosophy and practice; books about how children learn.

Now such considerations seem antique. Now we are in an age of data-based decision-making, where economists rule. They tell us that nothing matters but performance, and performance can be quantified, and those who do the quantification need never enter a classroom or think about how children learn.

So the issue of our day is: How do we measure teacher effectiveness? Most of the studies by economists warn that there is a significant margin of error in “value-added assessment” (VAA) or “value-added modeling” (VAM). The basic idea of VAA is that teacher quality can be measured by the test-score gains of their students. Proponents of VAA see it as the best way to identify teachers who should get merit pay and teachers who should be fired. Critics say that the method is too flawed to use for high-stakes purposes such as these.

Last July, the U.S. Department of Education published a study by Mathematica Policy Research, which estimated that even with three years of data, there was an error rate of 25 percent. A few months ago, I signed onto a statement by a group of testing experts, which cautioned that such strategies were likely to misidentify which teachers were effective and which were ineffective, to promote teaching narrowly to the test, and to cause a narrowing of the curriculum.

None of these cautions has stemmed the tide of rating teachers by student test scores and releasing the ratings. Last year, the Los Angeles Times published an online database that rated 6,000 teachers as to their effectiveness (one of them, elementary school teacher Rigoberto Ruelas, committed suicide a few weeks later). New York City is poised to make a public release of the names and ratings of 12,000 teachers, if the courts give the go-ahead (in the first trial, a judge ruled that the data could be released even if it was inaccurate).

This trend did not just happen. It was encouraged by the Obama administration’s Race to the Top, which urged states to develop quantitative measures of teacher effectiveness. Secretary of Education Arne Duncan has issued statements endorsing the publication of teachers’ names and ratings, although few testing experts agree with this practice.

The bulk of studies warn about the inaccuracy and instability of these measures, but the Gates Foundation recently released a study called “Measures of Effective Teaching” (MET) that supports the use of VAA and VAM. As is customary for the Gates Foundation, it hired an impressive list of economists at institutions across the nation to give the gloss of authority to its work. Among its key findings was this one: “Teachers with high value-added on state tests tend to promote deeper conceptual understanding as well.” Ah, said the proponents of measuring teacher quality by the rise and fall of student test scores, this study vindicates these methods and effectively counters all those cautionary warnings.

But now comes a re-analysis of the Gates study by University of California-Berkeley economist Jesse Rothstein, which says that the MET study reached the wrong conclusions and that its data demonstrate that VAA misidentifies which teachers are more effective and is not much better than a coin toss. Even the claim that teachers whose students get high scores on state tests will also get high scores on tests of “deeper conceptual understanding” is flawed, writes Rothstein.

The mismatch between Duncan’s words, actions

From Valerie Strauss

This was written by educator Anthony Cody, who taught science for 18 years in inner-city Oakland and who now works with a team of science teacher-coaches that supports novice teachers. He is a National Board-certified teacher and an active member of the Teacher Leaders Network.

This post, which appears on his Teachers Magazine blog, Living in Dialogue, refers to Monday’s Day of National Blogging for Real Education Reform, in which scores of educators from around the country shared their perspectives. Education Secretary Arne Duncan blogged too, and here is how Cody responded to Duncan:

Secretary Duncan,
You may recall we spoke on the phone last May when I was part of a group called Teachers’ Letters to Obama that organized ourselves to share our concerns and ideas with your administration. Unfortunately we never heard back from you (as I describe here).

In your post [Monday], you wrote, “We in education spend too much of our time and energy focused on issues that divide us. We forget how important it is to move forward on what we agree on.”

The trouble I have is that we are truly confused by a mismatch between your words and your actions. This makes a real consensus impossible, and forces us to continually return to the core disagreements we have with your policies.

My question to you is that you frequently tell teachers of your conviction that we need to move away from “teaching to the test.” Yet you are aggressively encouraging states and districts to:

* Pay teachers based on the growth in those test scores.

* Evaluate teachers based on those test scores.

* Close schools or fire principals or teachers based on those test scores

* Evaluate the “effectiveness” of teaching credential programs based on test scores.

You often say that we must recognize teachers for their greatness. Unfortunately, the primary means you have been promoting to measure greatness is the same one that doomed No Child Left Behind.

In your blog post yesterday, you wrote:

“For education reform to be ‘real,’ we need to focus on what works. We need consensus on the right way to measure students’ progress. And then we all need to hold ourselves accountable–and recognize those educators who are especially effective.

When you say we must focus “on what works,” do you recognize the research that demonstrates that pay for performance does NOT work, even to raise test scores? Has this powerful evidence caused any reevaluation of this strategy within the department?

Do you recognize that asking states to remove any limits on the expansion of charter schools is a mistake, given that charter schools have been shown to be no more effective than regular public schools?

While there may not yet be a consensus on the best way to measure student progress, there IS a clear consensus on what does NOT work – which you yourself frequently join in. That consensus says that the tests currently in use are, to use your own words, “low quality bubble-in tests.” There IS a broad consensus among educators that says the over-reliance on these scores for accountability purposes is destroying the quality of education in our schools. When will you bring your policies in line with your rhetoric?

Teachers have a whole range of alternatives to these misguided policies, and we have offered them repeatedly, with the belief that our deep understanding of what works in our classrooms and schools is an essential — but missing — component of improving schools.

You have been in office now for almost two years. It is not just my perception that teachers are more alienated than ever from the Department of Education. Do you hold yourself accountable for any part of this broken dialogue?

 

Education Secretary Duncan and teachers unions announce summit

By Nick Anderson

Thursday, October 14, 2010 Washington Post

Conventional wisdom holds that teachers unions are stuck in the status quo and that there are few examples of labor-management teamwork on education reform.

On Thursday, Education Secretary Arne Duncan and leaders of the two largest teachers unions called for an education summit early next year to disprove those notions.

The announcement highlighted several initiatives across the country, including a teacher peer-review program in Montgomery County and a new labor deal in Baltimore that allows teachers to be get raises based on student performance and other factors, not just seniority.

It omitted mention of the much-discussed labor contract for D.C. public schools that gives administrators more power to remove ineffective teachers and provides performance pay for those who excel.

Duncan made the announcement in Tampa with Dennis Van Roekel, president of the 3.2 million-member National Education Association, and Randi Weingarten, president of the 1.5-million-member American Federation of Teachers.

Schools in the Florida city are beneficiaries of a $100 million grant from the Bill & Melinda Gates Foundation to reshape traditional teacher pay scales and evaluation systems – another project with labor support.

“In dozens of districts around the country – from Tampa to Pittsburgh to Denver – union leaders and administrators are moving beyond the battles of the past and finding new ways to work together to focus on student success,” Duncan said in the announcement.

The summit on labor-management collaboration, at a date to be named, will include superintendents, school board members and labor leaders, according to the announcement.

DOE School Reform Is Intellectual Dishonesty and Political Puffery

“…Duncan routinely urges “a great teacher” in every classroom. That would be about 3.7 million “great” teachers — a feat akin to having every college football team composed of all-Americans. With this sort of intellectual rigor, what school “reform” promises is more disillusion.”

School reform’s meager results

By Robert J. Samuelson
Monday, September 6, 2010; A15

As 56 million children return to the nation’s 133,000 elementary and secondary schools, the promise of “reform” is again in the air. Education Secretary Arne Duncan has announced $4 billion in “Race to the Top” grants to states whose proposals demonstrate, according to Duncan, “a bold commitment to education reform” and “creativity and innovation [that are] breathtaking.” What they really show is that few subjects inspire more intellectual dishonesty and political puffery than “school reform.”

Civil Rights Groups’ Strong Criticism of Obama Ed Plan

Civil rights groups skewer Obama education policy (updated)
It is most politely written, but a 17-page framework for education reform being released Monday by a coalition of civil rights groups amounts to a thrashing of President Obama’s education policies and it offers a prescription for how to set things right.
You won’t see these sentences in the piece: “Dear President Obama, you say you believe in an equal education for all students, but you are embarking on education policies that will never achieve that goal and that can do harm to America’s school children, especially its neediest. Stop before it is too late.”
But that, in other nicer words, is exactly what it says. The courteous gloss on this framework can’t cover up its angry, challenging substance.
The “Framework for Providing All Students an Opportunity to Learn” is a collaboration of these groups: Lawyers’ Committee for Civil Rights Under Law, Rainbow PUSH Coalition, Schott Foundation for Public Education, National Association for the Advancement of Colored People, National Coalition for Educating Black Children, National Urban League, and the NAACP Legal Defense and Educational Fund, Inc.
Leaders of these groups were scheduled to hold a press conference Monday to release the framework but it was cancelled because, a spokesman said, there was a conflict in schedules. The delay was, presumably, not connected to public appearances this week by Obama and Education Secretary Arne Duncan at the convention marking the 100th anniversary of the Urban League in Washington D.C. Obama is making a speech on Thursday; Duncan on Wednesday.
The framework’s authors start the framework seeming conciliatory, applauding Obama’s goal for the United States to become a global leader in post-secondary education attainment by 2020.
But quickly their intent is clear. They take apart the thinking behind the administration’s education policies, and note a number of times the differences between what Obama and Duncan say about education and what they do.
To wit:

About Race to the Top, the competitive grant program for states that is the administration’s central education initiative thus far, it says:
“The Race to the Top Fund and similar strategies for awarding federal education funding will ultimately leave states competing with states, parents competing with parents, and students competing with other students….. By emphasizing competitive incentives in this economic climate, the majority of low-income and minority students will be left behind and, as a result, the United States will be left behind as a global leader.”
Ouch.
About an expansion of public charter schools, which the administration has advanced:
“There is no evidence that charter operators are systematically more effective in creating higher student outcomes nationwide….Thus, while some charter schools can and do work for some students, they are not a universal solution for systemic change for all students, especially those with the highest needs.”
And there’s this carefully worded reproach to the administration:
“To the extent that the federal government continues to encourage states to expand the number of charters and reconstitute existing schools as charters, it is even more critical to ensure that every state has a rigorous accountability system to ensure that all charters are operating at a high level.”
Double ouch.
But there’s more.
The framework says that the reauthorization of No Child Left Behind, formally known as the Elementary and Secondary Education Act, “should seek buy-in from community advocates.” But it notes that Obama’s Blueprint for Education reform makes “only cursory mention of parent and community engagement in local school development.”
It blasts the administration’s approach to dealing with persistently low-performing schools, saying that closing them in the way now being advanced is wrong, and it says that the administration is not doing enough to close gaps in resources, alleviate poverty and end racial segregation in schools.
And it says that the government should stop using low-income neighbors as laboratories for education experiments:
“For far too long, communities of color have been testing grounds for unproven methods of educational change while all levels of government have resisted the tough decisions required to expand access to effective educational methods. The federal government currently requires school districts to use evidence-based approaches to receive federal funds in DOE’s Investing in Innovation grant process. So, too, in all reforms impacting low-income and high-minority communities, federal and state governments should meet the same evidence-based requirement as they prescribe specific approaches to school reform and distribute billions of dollars to implement them.
“Rather than addressing inequitable access to research-proven methodologies like high-quality early childhood education and a stable supply of experienced, highly effective teachers, recent education reform proposals have favored “stop gap” quick fixes that may look new on the surface but offer no real long-term strategy for effective systemic change. The absence of these “stop gap” programs in affluent communities speaks to the marginal nature of this approach. We therefore urge an end to the federal push to encourage states to adopt federally prescribed methodologies that have little or no evidentiary support – for primary implementation only in low-income and high-minority communities.”
This is really tough talk, and it is about time that America’s civil rights leaders are speaking up.
The only question is whether anybody in the Obama administration is actually listening.
-0-
UPDATE:
Now we know why civil rights leaders suddenly cancelled today’s press conference at which they were going to talk about their new powerful framework for education reform, which includes a withering critique of the Obama administration’s education policies.
They met instead with Education Secretary Arne Duncan.
Rev. Jesse Jackson Sr., head of the Rainbow PUSH Coalition, said in an interview that he and other leaders felt that meeting with Duncan to discuss policy differences was “a better use of our time” than holding a public press conference.
Considering that most press conferences are a waste of time, Jackson makes a point.
But in this case, the postponement — or, perhaps, cancellation — left the impression among some that the civil rights leaders chose not to publicly criticize President Obama’s education policies any more than the framework already does.
The press conference was originally called for 10 a.m., which, it turned out, was exactly the time that the Duncan meeting started.
Jackson said Duncan listened as he and other civil rights leaders explained their concerns about ensuring equitable resources for each child and about how education reform should be part of a comprehensive urban renewal strategy that involves the Departments of Justice and Labor.
If quiet diplomacy can actually get Duncan to change some of his ill-conceived policies, then we can applaud this effort.
But if it doesn’t, it will be incumbent upon the civil rights leaders to shout to everyone who will listen that this administration is not doing what it must to ensure an equal education for every student.
They have to be as tough on a president that they like as they would be on a president that they don’t.

Follow my blog all day, every day by bookmarking washingtonpost.com/answersheet. And for admissions advice, college news and links to campus papers, please check out our Higher Education page at washingtonpost.com/higher-ed Bookmark it!

By Valerie Strauss  |  July 26, 2010; 3:00 PM ET

Obama School Plan Riles Lawmakers

By: Kendra Marr
May 14, 2010 04:12 AM EDT

Race to the Top is hitting the wall.

President Barack Obama’s $4.35 billion grant competition — designed to encourage states to dramatically improve school performance — is running into resistance across the country, as state officials and teachers unions are clashing with the administration over the contest rules.

And now Congress is getting into the act — with lawmakers of both parties challenging the president’s tough-love approach to school improvement.

“There are some very tough feelings over this,” said Sen. Mary Landrieu (D-La.). “Some people are afraid to say it, but I’m not.”

Minnesota Rep. John Kline, the top Republican on the Education and Labor Committee, has said he will urge Congress to oppose Obama’s request for $1.35 billion to extend the program in his fiscal 2011 budget.

“We have a program here that’s not yet proven,” he said. “We haven’t figured out if $4.35 billion is being spent wisely. Why would we add $1.35 billion?”

The idea seemed simple: Hold a contest for states to compete for billions in federal aid, right at a moment when school systems are battling budget problems. To win the funding, schools would have to convince Education Secretary Arne Duncan’s team that they were prepared to instill more teacher accountability and tougher standards to help students learn.

Summary: Chaos in Chicago’s Public School System (Duncan’s Model For National Reform)

Education Mon May 03 2010

The Education Revolt: The Chicago Model’s Fallout

Teachers, parents, and students are not happy at Chicago’s education leadership–there is mounting frustration with the Board of Education, the CPS bureaucracy, and the Chicago Teachers Union (CTU) leadership. As President Obama and Secretary of Education Arne Duncan take the Chicago model of slow-burn privatization national, Chicago may just be seeing a full fledged revolt against it. With the recent revelation that there are now no educators among the CPS’ top leadership, scrutiny of a reform program dominated by entrepreneurs and private interests (including a Board of Education stacked with financiers and real estate developers) is likely to sour people further.

Teachers, Parents, and Students, Oh My

Chicago’s teachers are angry; but that matters less than the fact that even more are discouraged, leaving the profession, burning out and warning the next generation away from teaching all together. Teachers have been under a full assault by corporate interests and the disingenuous reformers they underwrite for decades, and this assault has only intensified since the election of Barack Obama to the White House and the elevation of former CPS CEO Arne Duncan to the top of the Department of Education. Obama and Duncan have undertaken to bring Chicago-style education reform to the level of national policy, without any evidence whatsoever that that reform works.

The (arguably illegal) Race to the Top program, which embodies Chicago Renaissance 2010 model of school turnarounds, privatization, and “pay-for-performance” incentives, is just getting underway, and teachers around the country are finding out, too late, that Obama et al are hostile to public educators.

But here in Chicago, where the method to this madness was born, teachers and parents are organizing revolts to protect their schools. Unhappy teachers are lining up to challenge a union leadership they characterize as ineffective or accommodationist and an insular Board of Education, as parents and students are fighting to keep their schools public and democratically controlled. And what happens here, at ground zero of school privatization, could presage what happens nationally as the federal government tries to strong arm school districts into dismantling their public schools; a policy instituted as a sop to “centrism” could end up sparking a serious fight in the moderate liberal wing of the Democratic Party as urban community groups and teachers union factions resist.

Duncan Can’t Figure out Why So Few Black Teachers in New Orleans

Below is a link to a story that ran
today about Arne Duncan visiting New Orleans and bemoaning the low
percentage of African American teachers in the United States .  He is
expected to give a commencement speech at Xavier University in New
Orleans today and encourage students to consider a career in teaching.

I hope that during Duncan ‘s visit someone reminds him that all 4,000
teachers in New Orleans , 90% of whom were black, were summarily fired
after hurricane Katrina and replaced with overwhelmingly white Teach For
America teachers with no certification and little more than six weeks of
training.

Why would young African Americans opt for a career in teaching after
seeing what the state did to the 4,000 black New Orleans teachers who
devoted themselves to educating challenging students?  Teaching has
become an even more perilous vocation as a result of Duncan ‘s Race To
The Top Program, founded on the specious argument that failing schools
are the sole responsibility of bad teachers, so bad teachers need to be
fired.

The New Orleans policy of relying on novice teachers has contributed to
an academically segregated two-tiered system of schools that
concentrates novice teachers in schools where students have the greatest
needs for skilled educators. I have attached an UNTO report on the lack
of experienced teachers in New Orleans in which the authors note that:

* Before Katrina only 10% of New Orleans Teachers were in their first or
second year teaching.  By 2008, 33% of the city’s teaches fall in this
novice category.
* Inexperienced teachers are dumped into the remaining state-run RSD
public schools, ensuring that the students with the greatest needs are
taught by the least prepared teachers.
* 40% of all teachers in the state-run RSD schools have less than 2
years experience, compared to only 18% in the charter schools that
opened in 2006.
* At-risk black students are nearly three times as likely to have a
novice teacher than non at-risk white youth.

If we want more young African Americans to rise to the challenge of
teaching the disadvantaged, then we need to reward altruism rather than
punish it.

Link to Duncan Article
http://bit.ly/cMaZzH

Lance Hill, Ph.D.
Executive Director
Southern Institute for Education and Research
Tulane University
MR Box 1692
31 McAlister Drive
New Orleans , LA 70118
(504) 865-6100
fax (504) 862-8957
Lhill@tulane.edu
www.SouthernInstitute.info

Chicago Scandal Emerges From Duncan Years: Practice of Saving School Seats for the Elite

When U.S. Secretary of Education Arne Duncan ran the Chicago Public Schools, he kept a secret list of those who hoped to clout children into the city’s top-tier public schools.

“We didn’t want to advertise what we were doing because we didn’t want a bunch of people calling,” CPS official David Pickens admitted to Tribune reporters Azam Ahmed and Stephanie Banchero, who broke the story.

Chicago Tribune

Mayor Daley denies role in lobbying schools

Logs show several admissions requests from his administration

Mayor Richard Daley denied Tuesday that his office had any role in an underground process to lobby on behalf of students applying to the city’s best public schools, even though secret logs indicate several admissions requests came from his administration.

Former schools chief Arne Duncan tracked admissions requests over several years, creating a lengthy, detailed compilation of politicians and influential business people who intervened on behalf of children during his tenure. The lists, used mostly in appeals cases, also show inquiries from unconnected parents looking to place their children.

There is no evidence that principals were forced to admit unqualified students. Indeed, many students were still rejected after making the lists. Documents obtained by the Tribune chronicle calls and requests coming into Duncan’s office in 2006 and 2008, though it’s unclear if the documents are complete for those years.

The nearly 40 pages of logs show requests from 25 aldermen, House Speaker Michael Madigan and his daughter, Illinois Attorney General Lisa Madigan. Applications that district officials say were backed by Duncan’s wife, mother and a personal trainer at a downtown athletic club where he played basketball appear, as well.

There also were references to a Daley education aide making requests, including a 2008 entry seeking admission for the daughter of a prominent zoning attorney to Augustus H. Burley School, a magnet elementary that focuses on writing and literature. The student was No. 5 on the wait list at the time and it’s unclear if she was admitted, according to the log.