May 30, 2024

Archives for September 2003

Computers As Graders

One of my least favorite tasks as a professor is grading papers. So there’s good news – of a sort – in J. Greg Phelan’s New York Times article from last week, about the use of computer programs to grade essays.

The computers are surprisingly good at grading – essentially as accurate as human graders, where an “accurate” grade is defined as one that correlates with the grade given by another human. To put it another way, the variance between a human grader and a computer is no greater than between two human graders.

Eric Rescorla offers typically interesting commentary on this. He points out, first, that the lesson here might not be that computers are good at grading, but that human graders are surprisingly bad. I know how hard it is to give the thirtieth essay in the stack the careful reading it deserves. If the grader’s brain is on autopilot, you’ll get the kind of formulaic grading that a computer might be able to handle.

Another possibility, which Eric also discusses, is that there is something simple – I’ll call it the X-factor – about an essay’s language or structure that happens to correlate very well with good writing. If this is true, then a computer program that looks only for the X-factor will give “accurate” grades that correlate well with the grades assigned by a human reader who actually understands the essays. The computer’s grade will be “accurate” even though the computer doesn’t really understand what the student is trying to say.

The article even gives hints about the nature of the X-factor:

For example, a high score almost always contains topically relevant vocabulary, a variety of sentence structures, and the use of cue terms like “in summary,” for example, and “because” to organize an argument. By analyzing 50 of these features in a sampling of essays on a particular topic that were scored by human beings, the system can accurately predict how the same human readers would grade additional essays on the same topic.

This is all very interesting, but the game will be up as soon as students and their counselors figure out what the X-factor is and how to maximize it. Then the SAT-prep companies will teach students how to crank out X-factor-maximizing essays, in some horrendous stilted writing style that only a computerized grader could love. The correlation between good writing and the X-factor will be lost, and we’ll have to switch back to human graders – or move on to the next generation of computerized graders, looking for a new improved X-factor.

RIAA Files 261 Suits

The RIAA launched its long-awaited lawsuit storm today. John Borland at CNet news.com reports that 261 copyright infringement suits were filed against individual defendants.

Several of the suits have already settled, reportedly for around $3,000 each.

P2P Porn

Yesterday’s New York Times reported that recording industry lobbyists are shocked, shocked to find porn on popular peer-to-peer networks. Naturally, they think P2P should be heavily regulated as a result. Somebody should send them a copy of Michael Kinsley’s classic Slate editorial on the most dangerous of all media: paper.

RIAA to Grant Semi-Amnesty

The RIAA is reportedly planning to offer amnesty to file sharers. According to the reports, just after the RIAA launches its upcoming flurry of lawsuits against file sharers, it will offer a deal to everybody else: send a letter to RIAA admitting that you have infringed in the past, and in exchange RIAA will promise not to sue you for your past infringement. If you continue to infringe, though, the RIAA gets the right to sue you for wilfull infringement.

Mary Hodder at bIPlog points out that this may not be as good a deal as it might seem. The RIAA and its members don’t own all of the copyrights that cover recorded music. If you infringe, the RIAA and its members are not the only people who can sue you. An amnesty from them is worth something, but it doesn’t shield you fully from lawsuits. Given that, a written admission of infringement might not be a wise move.

Ruling in Garage-Door-Opener Case

An important ruling was issued yesterday in the Chamberlain v. Skylink lawsuit. (See this previous post for a summary of the case.)

The court denied Chamberlain’s motion for summary judgment. Although this is only a ruling on a preliminary motion, the judge used it to offer her analysis of how the DMCA applies to the apparent facts in the case. In short, she ruled that it is not a violation of the DMCA for Skylink to make a replacement remote control that can open Chamberlain-brand garage door openers.

Chamberlain uses a simple cryptographic protocol to authenticate the remote (the small push-button device you keep in your car) to the opener (the big unit attached to the garage ceiling). The purported purpose of this is to prevent bad guys from recording the signals sent by the remote, and replaying them later to open the door when the homeowner is gone. The protocol includes a resynchronization mode that is used when the remote and the opener somehow get out of sync. Skylink’s replacement remote uses the resynchronization mode every time. Chamberlain argued that by doing this Skylink was circumventing Chamberlain’s authentication protocol, and that the protocol controls access to the copyrighted software running in the opener. Chamberlain concluded that Skylink’s actions ran afoul of the DMCA’s ban on devices that circumvent (without permission) measures that control access to copyrighted works.

The judge ruled that Skylink was not violating the DMCA, essentially because consumers have permission to open their own garages. You might think this sensible conclusion was easy to reach, but it was not. The judge’s problem was that in a previous DMCA case (Universal v. Remeirdes) a court had ruled that consumers do not have permission to view their own DVDs, except on devices “authorized” by the copyright owner. (To be more precise, the Remeirdes court ruled that whatever permission consumers had did not create an exception to the DMCA.)

The toughest part of the Chamberlain judge’s opinion is the part that tries to reconcile her ruling with the previous Remeirdes ruling. (This is on pages 25 and 26 of the ruling, if you’re reading along at home.) I have to admit I don’t fully understand this part of the judge’s ruling. Ernest Miller at LawMeme is scornful, saying that the judge used tortured reasoning, based on artificial distinctions between the cases. Derek Slater says that the judge should have simply admitted that her ruling is inconsistent with Remeirdes. (She is allowed to be inconsistent, because Remeirdes was decided in a different circuit and so is not binding precedent for her.)

I’m not sure what to think about this. I hope the issue will become clearer after more discussion.