I don't like writing tests. I know, I won't get any sympathy from my students who don't enjoy taking the tests I write. But I constantly struggle with writing a good assessment, and in my more generous moments, I give credit to the folks at College Board and ETS who write huge, national tests for a living.

The College Board announced today their planned changes for the SAT. I am no friend of the College Board, and it's too early to tell if their revision will amount to much, but their stated goals are not off base:

Current research shows us that students need to dig deeper into fewer topics — those topics that are most important for success in college and career. When we help students focus on the skills and knowledge that are essential for college readiness, we’re setting them up to succeed. We will work with classroom teachers to ensure that the daily work students do in grades 6–12 aligns with the demands of college and career readiness.

I have a complicated relationship with standardized testing like this. On the one hand, the idea that any single test can determine a student's fate is absurd. And I've seen first hand some of the unintended consequences of the testing culture. There are the students who spend so much time, money, and energy on learning how to beat the test that they may as well not be learning anything else in school. And having taught at a large public university, I certainly wouldn't say that these standardized tests have set the bar particularly high either. Having spent most of my career in private schools, I've avoided the weeks of state-mandated tests that my public school counterparts have to endure, and all the lost instruction time that entails, but I certainly understand those frustrations.

But at the same time, there has to be some accountability for the education system, and you need some kind of standardized assessment to get good data. The content of those assessments is debatable, but not their value. As Amanda Ripley writes in The Smartest Kids in the World: And How They Got That Way:

Tests helped schools to see what they were doing right and wrong, and who needed more help. That insight was a prerequisite, not a solution.

The problem comes, of course, when schools see testing as an end to itself. If the goal isn't, "Prepare students for life," but rather, "Prepare schools to have all students pass the tests," you end up with watered-down standards, poor tests, and results used to punish or reward schools, not to enact any meaningful changes.

Other countries have had success using national standardized tests appropriately to strengthen their education system. By drawing a line in the sand and saying, "Students must have these skills to succeed," it can really focus the schooling on those priorities. That hasn't happened in the US yet, probably because every state has wildly different standards, and even the national tests like the SAT and ACT are not particularly great tests of what is truly important for college and beyond. From Ripley again:

Matriculation exams like Finland’s helped inject drive into education systems—creating a bright finish line for kids and schools to work toward. Teenagers from countries with these kinds of tests performed over sixteen points higher on PISA than those in countries without them.

So is there a way to split this difference, to have the benefits of a rigorous, standardized exam, while not turning students into test-taking automatons? Can a single assessment possibly be comprehensive enough to achieve this? I recently saw this from Grant Wiggins about creating valid assessments:

Could the students do a great job on the task, but not meet the goal of the assessment?

Could the students do a poor job on the task but still provide lots of evidence that they can otherwise meet the goal of the assessment.

Then the proposed test is likely invalid for that goal.

That simple two-question test is hard enough for me to apply on a simple unit assessment in Latin class. Making a large, national test valid is a large undertaking, but not impossible. As Wiggins goes on to point out in another entry, this question of validity is one of the pieces the College Board is trying to address. The real problem, according to Wiggins, isn't validity, it's reliability, the "margin of error" in a test's result:

The irony is that critics of the test typically claim that the tests are not valid. But that’s where they probably go wrong. The better argument concerns the questionable reliability of a single high-stakes score.

It’s no accident, in fact, that the World Series is best four out of seven.

Most of the current crop of standardized tests, certainly the SAT and ACT, can be questioned in terms of both their validity and their reliability, which is a problem for the students and the schools being judged on these test results. Incremental improvements like what the College Board is enacting are a step in the right direction, but can any single three-hour test really fit all these criteria? I'm skeptical, but I'm also unable right now to propose exactly how to scrap it and start over. Or maybe we (teachers, administrators, colleges) need to, while tweaking the current tests, also be more cognizant of what the tests do (they tell us something about what a student has learned) and don't do (tests like this don't communicate everything about the student or their school).

I need to read more about this topic. It's important not only for big, national tests but also on a much smaller scale at my school.