Home ::
It seems that just about every decade, the Texas Legislature hatches a major education reform plan that has profound effects on our public education system. Anyone who has been involved in education for any length of time remembers well when no-pass, no-play, TECAT and the career ladder emerged in 1984, and a little more than ten years later our state accountability system was born. Now here we are slightly more than a decade after that, with major testing and curriculum reforms under way, including end-of-course exams and college readiness standards.
Although many of these reforms had a significant impact on Texas education, most would agree that the accountability system stands out as the reform that has most changed the way educators and schools operate on a day-to-day basis. And many would also argue that the resulting changes (at least in recent years) are not for the good. That is why legislation was passed last session to revisit and redesign our state accountability system, largely due to the efforts of the “author” of our original state accountability system, former Senator and Lt. Gov. Bill Ratliff. When one of the founding fathers of our state accountability system says we should “not try to fix” the accountability system “one piece at a time” but instead should “wipe the slate clean and start over,” people stop and listen.
As a result, the Select Committee on Public School Accountability was formed to review our current system and come up with recommendations for the Legislature about how to change the system for the better. TCTA is fortunate to have Susan Lewis, our state president-elect, serving on this extremely important committee. In fact, Susan is currently the only active teacher in a public school classroom on the committee, along with legislators, business owners, superintendents and a principal. The Select Accountability Committee is holding monthly hearings to take testimony from experts and the public on various facets of our accountability system in order to develop recommendations about what changes need to be made. (See related article on pages 7-8.)
Upcoming meetings:
June 16th - Dallas
July 14th - Brownsville
August 4th - West Texas (city to be determined)
Less emphasis on testing
It is clear from the testimony the Select Committee has heard that if there’s an obvious villain in our current accountability system, it is surely the Texas Assessment of Knowledge and Skills (TAKS) test. Much like the federal No Child Left Behind Act “may be the most tainted brand in America” (according to U.S. Rep. George Miller, D-CA, chair of the House Committee on Education and Labor), the TAKS test is the flashpoint for the most heated and passionate arguments about an accountability system gone awry. Because of the enormous weight our accountability system places on TAKS test results in assigning school and district ratings, it is no wonder that schools and educators are driven to focus on preparing students to pass the test.
The perverse effects of this kind of test-driven system are seemingly unending. There are numerous accounts of an insidious intrusion into instructional time spent preparing for the TAKS test; educators report directives from school leaders to spend dictated amounts of time drilling students on TAKS concepts, even if the concepts aren’t part of the curriculum for that class. Studies have confirmed that the curriculum in schools has narrowed in order to focus on tested subjects and allow more time for test preparation. Parents tell of students who are sick with worry and fear about the consequences of not passing the test (students in grades 3, 5 and 8 have to pass certain TAKS tests in order to be promoted to the next grade, and students cannot graduate unless they pass the exit-level TAKS test). Whole systems are in place for intensive TAKS preparation, ranging from daily TAKS preparation drills to tutoring before, during and after school. At the March 2008 Select Accountability meeting, testimony describing the magnitude of test preparation at the cost of instruction prompted Select Accountability Committee member Salem Abraham, a Canadian ISD school board member, to exclaim in exasperation, “You can’t fatten a pig by weighing it all the time!”
Our test-driven accountability system has other ramifications. When a test is as high-stakes as the TAKS, test security, integrity and validity become critical. Thus, the testing machine has to be fed – new tests must be developed routinely since the law requires the tests to be released every three years. This requires massive field-testing of the new tests, which then creates another serious intrusion into the already scarce instructional time. Extensive data systems must be in place to crunch numbers and report test results. Complicated and strict test administration procedures are required, with stern warnings to educators about certificate sanctions should they step out of line. And the cycle goes on and on.
It is for these reasons that one of the chief recommendations coming from educators and parents is to reduce the emphasis on testing. Others have argued that the state ought to get out of the testing business altogether and rely on “shelf” tests in order to assess student learning, rather than spend massive amounts of state funds and time developing our own. Still others have noted that the federal No Child Left Behind Act, which adds another layer of accountability to our state system, doesn’t require as much testing as our state system requires.
Credit should be given for student growth
Still, the majority of educators and the public seem to feel that some assessment of students is necessary in order to determine how well students are doing. But an important corollary discussion taking place, along with the need to reduce the emphasis on testing, is how tests are used to hold schools, educators and students accountable. An increasing chorus of voices is pointing out the need for our state and federal accountability systems to measure not just student attainment of standards, but student growth toward those standards. One of the chief criticisms that educators have about our current system is that it doesn’t take into account the point at which a student starts in the process of being measured. Most would agree that for fairness and a more complete picture of how students/schools are doing, any accountability system should include growth and attainment of standards.
Yet, growth measures are not the panacea some would make them out to be – the core troubling issues about using student test scores to evaluate school and educator performance remain. For example, in order to be fair and accurate in holding schools/educators accountable for student test scores, outside factors must be screened out. Although there is research suggesting that a quality teacher is the most important school-related influence on student achievement, a 2000 report from the Education Commission of the States notes that most researchers agree that home and family influences also play a strong role. In fact, research discussed in a report from the Consortium for Policy Research in Education suggests that teachers account for 20% of a student’s performance, with roughly 20% of the differences in student achievement associated with the schools children attend, and 60% associated with differences among the children in each classroom, including the effects of their prior achievement and socioeconomic background.
Some hold up a measure known as a value-added model (VAM, a complex, statistical measure that is said to isolate school/educator effects on student performance) as the answer to these concerns. Yet, to date, there is no widely available, scientifically validated value-added model in existence. The ones that are in existence are either too new to produce sufficient evidence to evaluate their effectiveness, or the evidence of their effectiveness is lacking. A report issued in 2004 by the respected Rand Corporation indicates that “the research base is currently insufficient to support the use of VAM for high-stakes decisions.”
Purpose of growth model is an important consideration
It is also important for policymakers to have a clear statement of intent when considering adoption of a growth model, because different purposes need different models.
It seems that it is often the case in this state that we enact major education reforms without first adopting a clear statement of the purpose of the reform. A good example of this is the recent push from various quarters for schools to be held accountable for the post-secondary readiness of students. Like a growth model, accountability for post-secondary readiness might be better than what we currently have, but there is a lack of consensus in this state as to what post-secondary readiness is. Although there’s research suggesting that the knowledge and skills needed for college readiness and workforce readiness are one and the same, still other research suggests that they are quite different; others argue that there is a core set of knowledge and skills that apply to both. Additionally, there are numerous entities working on the issue of post-secondary readiness, but no statewide coordination of these efforts. Consequently, we have a definition of post-secondary readiness that has emerged from the High School Completion and Success Initiative Council, a definition of college readiness that has been adopted by the Texas Higher Education Coordinating Board, college readiness standards developed by the Commission for a College Ready Texas, and college readiness standards developed by vertical teams of educators that have yet to be adopted by the Commissioner of Education. Obviously it seems premature to incorporate the concept of post-secondary success into our accountability system when we haven’t even agreed upon its definition.
Using a growth measure effectively
In order to use a growth measure accurately and effectively, certain conditions must be in place (source for the following: National Association of School Boards, Measuring Student Growth: A guide for informed decision making):
Tests: The tests used for measuring growth should: (1) document year-to-year growth on a single scale, (2) measure a broad range of skills, and (3) align the test content with state standards. Additionally, tests should be vertically aligned and scaled. Numerous experts have pointed out that the TAKS test is not such a test. That is partly why legislation was passed last session to replace TAKS at the high school level with end-of-course exams. But those tests won’t be in place until 2011-12, and the TAKS test remains for grades 3-8.
Data: Data systems also have to be in place to collect, store and analyze the data, but according to the 2007 Data Quality Campaign, Texas currently does not have a data system capable of sufficiently performing the tasks required to incorporate growth measures into our system.
Transparency vs. complexity: Growth models can be complex and very difficult for non-statisticians to understand. Therefore, some say professional development is a key factor for gaining buy-in. This is especially true for value-added models because there is no simple way to isolate the impact of teaching on student learning. Others believe that complex value-added models fail one of educators’ most important criteria: that the models be transparent.
Costs: It costs between $1 million and $3 million to implement a student ID system – which is required to calculate individual student growth – and there is an additional cost to actually make the calculations. While there isn’t much information on what it costs states to develop and run the calculations for a growth model, Ohio contracts with the SAS Institute in North Carolina for such purposes at $2 per student.
An enormous task
These factors illustrate the enormity of the task that the Select Accountability Committee and, ultimately, the Texas Legislature have in designing our new state accountability system. The importance of getting sufficient stakeholder input and structuring changes so that they occur in the proper order and sequence is essential to the success of our next accountability system. We stand at a crossroads in this state as we decide where to go from here, and it is essential for educators to be fully engaged in charting our course.
References and links to pertinent information:
College readiness definition adopted by the Texas Higher Education Coordinating Board.
College readiness standards developed by the Commission for a College Ready Texas.
College readiness standards developed by vertical teams of educators that have yet to be adopted by the Commissioner of Education.
Evaluating Value-Added Models for Teacher Accountability, the Rand Corporation, 2004.
In Pursuit of Quality Teaching: Five Key Strategies for Policymakers, Education Commission of the States, 21st Century.
Measuring student growth: A guide for informed decision making, 2007.
National Center for Educational Achievement , 2007 Data Quality Campaign .
Post-secondary readiness definition from the High School Completion and Success Initiative Council.
Using Student Progress to Evaluate Teachers: A Primer on Value-Added Models, Educational Testing Service, Sept. 2005.
Value-Added Assessment of Teachers: the Empirical Evidence, by Haggai Kupermintz, 2002.
What Large-Scale, Survey Research Tells Us About Teacher Effects On Student Achievement: Insights from the Prospects Study of Elementary Schools, Consortium for Policy Research in Education, 2002.
Web posted 06/06/08 from The Classroom Teacher, Summer 2008.
