Newsweek has released its annual list of America’s top high schools, using the same flawed formula as last year. Here’s what I wrote then:

Here is Newsweek’s formula:
“Public schools are ranked according to a ratio devised by Jay Mathews: the number of Advanced Placement and/or International Baccalaureate tests taken by all students at a school in 2004 divided by the number of graduating seniors.”

Both parts of this ratio are suspect. In the numerator, they count the number of students who show up for AP/IB tests, not the number who get an acceptable score. Schools that require their students to take AP/IB tests will do well on this factor, regardless of how poorly they educate their students. In the denominator is the number of students who graduate. That’s right — every student who graduates lowers the school’s rating.

To see the problems with Newsweek’s formula, let’s consider a hypothetical school, Monkey High, where all of the students are monkeys. As principal of Monkey High, I require my students to take at least one AP test. (Attendance is enforced by zookeepers.) The monkeys do terribly on the test, but Newsweek gives them credit for showing up anyway. My monkey students don’t learn enough to earn a high school diploma — not to mention their behavioral problems — so I flunk them all out. Monkey High gets an infinite score on the Newsweek formula: many AP tests taken, divided by zero graduates. It’s the best high school in the universe!

[Note to math geeks annoyed by the division-by-zero: I can let one monkey graduate if that would make you happier.]

Though it didn’t change the formula this year, Newsweek did change which schools are eligible to appear on the list. In the past, schools with selective admission policies were not included, on the theory that they could boost their ratings by cherry-picking the best students. This year, selective schools are eligible, provided that their average SAT score is below 1300 (or their average ACT score is below 27).

This allows me to correct an error in last year’s post. Monkey High, with its selective monkeys-only admission policy, would have been barred from Newsweek’s list last year. But this year it qualifies, thanks to the monkeys’ low SAT scores.

Newsweek helpfully includes a list of selective schools that would have made the list but were barred due to SAT scores. This excluded-schools list is topped by a mind-bending caption:

Newsweek excluded these high performers from the list of Best High Schools because so many of their students score well above average on the SAT and ACT.

(If that doesn’t sound wrong to you, go back and read it again.) The excluded schools include, among others, the famous Thomas Jefferson H.S. for Science and Technology, in northern Virginia. Don’t lose heart, Jefferson teachers – with enough effort you can lower your students’ SAT scores and become one of America’s best high schools.

Yesterday was the NBA draft. In the first round, eight high school seniors were taken, and only five college seniors. (The rest were overseas players and college underclassmen.) The very first pick was a high school senior, chosen over a very accomplished college player.

You have to be 16 to drive. You have to be 21 at drink alcohol (at least where I live). Should there be a minimum age for playing professional basketball? NBA commissioner David Stern favors a minimum age of 20 for NBA players. The NFL’s rule, banning players less than three years out of high school, withstood a court challenge from Maurice Clarett, who wanted to go pro after two years of college.

Nobody can argue, after seeing Kobe Bryant, Kevin Garnett, and LeBron James, that college is a prerequisite for NBA stardom. Sure, some high-school draftees wash out, but they may well have failed just as badly had they spent four years playing college ball.

Stern, and other proponents of the minimum age rule, argue that going to college is good for these kids. That’s probably true, if they become real students. But it’s hard to see the point in making them pretend to be students, which is what many of them would do were it not for the straight-to-the-pros path. It’s especially hard to see the point of making them mark time as pseudo-students until they pass some arbitrary age threshold, at which point they can drop their pseudo-education like a red-hot brick and jump to the pros.

Another, considerably more cynical, argument for an age limit is that forcing kids to play college sports is a clever way to subsidize university education. If college basketball is just minor-league pro ball with unpaid players, then it can serve as a profit center for universities, generating revenue to support other students who are actually being educated.

But all of this ignores the biggest losers in the trend towards professionalization of college sports: the true student-athletes. These are the players who don’t spend all day in the weight room, who study things other than game films. It’s very hard for them to compete against full-time athletes, and so they face intense pressure to slack on their studies.

It seems to me that professional football and basketball could learn a thing or two from baseball. The normal path in baseball has been for players to turn pro immediately after high school, with only a few players

One of my least favorite tasks as a professor is grading papers. So there’s good news – of a sort – in J. Greg Phelan’s New York Times article from last week, about the use of computer programs to grade essays.

The computers are surprisingly good at grading – essentially as accurate as human graders, where an “accurate” grade is defined as one that correlates with the grade given by another human. To put it another way, the variance between a human grader and a computer is no greater than between two human graders.

Eric Rescorla offers typically interesting commentary on this. He points out, first, that the lesson here might not be that computers are good at grading, but that human graders are surprisingly bad. I know how hard it is to give the thirtieth essay in the stack the careful reading it deserves. If the grader’s brain is on autopilot, you’ll get the kind of formulaic grading that a computer might be able to handle.

Another possibility, which Eric also discusses, is that there is something simple – I’ll call it the X-factor – about an essay’s language or structure that happens to correlate very well with good writing. If this is true, then a computer program that looks only for the X-factor will give “accurate” grades that correlate well with the grades assigned by a human reader who actually understands the essays. The computer’s grade will be “accurate” even though the computer doesn’t really understand what the student is trying to say.

The article even gives hints about the nature of the X-factor:

For example, a high score almost always contains topically relevant vocabulary, a variety of sentence structures, and the use of cue terms like “in summary,” for example, and “because” to organize an argument. By analyzing 50 of these features in a sampling of essays on a particular topic that were scored by human beings, the system can accurately predict how the same human readers would grade additional essays on the same topic.

This is all very interesting, but the game will be up as soon as students and their counselors figure out what the X-factor is and how to maximize it. Then the SAT-prep companies will teach students how to crank out X-factor-maximizing essays, in some horrendous stilted writing style that only a computerized grader could love. The correlation between good writing and the X-factor will be lost, and we’ll have to switch back to human graders – or move on to the next generation of computerized graders, looking for a new improved X-factor.