PDA

View Full Version : Reality Check?



Fruitbat
07-05-2011, 08:11 AM
Wild idea here, just wondering what people think...

If there was an essay test to broadly evaluate writing level, say like they have on college entrance exams, that gave a number, say 1 to 4 of writing ability, do you think it could/would be useful?

There is a large discrepancy between many writers' abilities and their perception of their abilities. Slush piles, contests, critiquing, all show that most aren't writing at anywhere near a publishable level, yet they firmly believe that they are. For one real world problem, it's probably why you can't send hardly anything to publishers without an agent anymore. It's made it too costly for them to sift through the slush. Also, of course delusional people are hard to work with.

If there was such a test, maybe some publishers would take direct work again, for example, if your test results were a "4." Or, if you scored a "2" maybe the sections you did poorly in would tell you where to focus your improvement efforts, or other useful applications.

Do you think there would be any value in such a test, and would you take it?

Cyia
07-05-2011, 08:50 AM
You're talking about the Dunning Kruger Effect. (http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect)

No, a standardized test wouldn't be any sort of indicator of writing ability because once it's standardized, it's no longer subjective. You can't quantify voice or style; they're unique to the writer. All you can do is improve technical skill, and sadly, one of the biggest obstacles to those writers you mention is the fact that they were told by teachers in high school or college, using those standard tests, that they possessed those skills.

Academic writing =/= commercial writing.

Fruitbat
07-05-2011, 08:55 AM
Thanks, Cyia. I've copied and pasted it here because it's interesting - it has a name! :)

The Dunning–Kruger effect is a cognitive bias (http://en.wikipedia.org/wiki/Cognitive_bias) in which unskilled people make poor decisions and reach erroneous conclusions, but their incompetence denies them the metacognitive (http://en.wikipedia.org/wiki/Metacognitive) ability to appreciate their mistakes.[1] (http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect#cite_note-morris-0) The unskilled therefore suffer from illusory superiority (http://en.wikipedia.org/wiki/Illusory_superiority), rating their ability as above average, much higher than it actually is, while the highly skilled underrate their own abilities, suffering from illusory inferiority. Actual competence may weaken self-confidence, as competent individuals may falsely assume that others have an equivalent understanding. As Kruger and Dunning conclude, "the miscalibration of the incompetent stems from an error about the self, whereas the miscalibration of the highly competent stems from an error about others" (p. 1127).[2] (http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect#cite_note-Kruger-1)
The effect is about paradoxical defects in cognitive ability, in oneself and others.

Mr. Anonymous
07-05-2011, 10:01 AM
agree with cyia to an extent. Writing is very subjective. Not entirely. But subjective enough to pose great difficulties for any kind of standardized measurement/quantification of writing ability.

The only way I could see it working would be to have a large sample (at least 20, the more the better) of qualified readers (agents, editors, writers, professors, regular people who are well read) each read the writer's work and rate it on, say, a scale of 1-10, then take the average.

Aside from this being logistically more-or-less impossible, an additional problem is that writers who take greater risks would probably be rewarded with lower ratings, not because their work is worse but because it does not have the same mass appeal.

So if a Harry Potter type story were pitted against a Lolita type story (assuming both are very well written), I'd put my money on the Harry Potter one.

Vespertilion
07-05-2011, 10:08 AM
There's another thread around here about Hemingway and commas. Even if you used a panel of dedicated copyeditors instead of writers (if we're judging technical skill only), then he'd probably be graded a 3 or so on his lack of proper comma usage, right? Maybe lower, if the panel thinks it shows a critical gap in his grasp of the language.

Does he get graded first, then "allowed" to do as he likes once Someone Else tells him he's good enough to do so? Or is the majority of the panel in agreement that he's an exception, and graded higher than he would be on technical skill alone? What does that do for the rest of the scores handed out by that panel?

What do you do with the writer who takes liberties, but is so damned engrossing she sells like hotcakes, because the readers don't care that she tested at a 2.5?

Technical skill may not be subject to personal opinion, but the perception of skill or talent very much is. Taste, peeve threshold, critical training, expectations--so many things affect a reader's experience, and no two will agree completely on what's acceptable in a published work.

ETA: cross-posted with Mr. Anonymous :)

Polenth
07-05-2011, 10:11 AM
Most of the people we get through SYW can write a correct sentence. The thing you can't easily test for is whether that sentence is interesting, or whether a story containing that sentence will sell.

...and no, I wouldn't take a test. I passed GSCE English well enough to show I have a basic grasp of English.

Cyia
07-05-2011, 10:16 AM
So if a Harry Potter type story were pitted against a Lolita type story (assuming both are very well written), I'd put my money on the Harry Potter one.

Except that at the time it was published, HP would have scored very low under those conditions. Fantasy was a dead zone and long children's books were unheard of. By commercial standards, it didn't have legs for mass appeal.

Jamesaritchie
07-05-2011, 06:30 PM
Writing ability is pretty easy to judge, and also mostly meaningless. Story and character are what sell fiction, and these are much, much tougher to judge in any sort of standardized way.

But, yes, such a test would weed out about forty percent of those you find in slush piles.

Writing may be subjective to a degree, but really horrible writing, lousy grammar, and poor punctuation are not subjective.

JimmyB27
07-05-2011, 07:17 PM
Writing may be subjective to a degree, but really horrible writing, lousy grammar, and poor punctuation are not subjective.

They can be. Witness A Clockwork Orange, Trainspotting and Feersum Endjinn, for example.

kaitie
07-05-2011, 08:09 PM
This is interesting considering my bout of insecurity last night. On the SAT and GRE, I had to do writing sections for both. For the latter, I actually was taking the test while ill, on four hours of sleep, and realized ten minutes before the end of the time limit that I'd written on the wrong topic and had to go back and start from scratch.

On both, I scored in the 98th percentile for ability. I've always been a fairly good writer. I made it through school turning in first-draft essays I'd thrown together in an hour without any prewriting whatsoever. Even later when my work required research, I'd spend two weeks on the research but then write the paper itself in no time. I always got A's.

At the same time, writing, particularly writing fiction, is about so much more than being able to write, particularly essay writing. Even if we switched it to "write a short story," the problem remains that some people are great at short stories and some people can't do it at all.

I had a professor tell me once before that I should never doubt my writing, but that he wasn't sure I could tell a story. Last night I was feeling pretty down wondering if I'm just not a storyteller.

There are so many elements required to make a good novel and a good author. You need an understanding of structure, pacing, character, dialogue, and you need to be able to tell an interesting story in an interesting way. It's not just about writing ability, and while that goes a long way, it's not the only qualification. You can be a good writer who can't write a publishable novel because of a failure in one of the other areas.

I think the best thing for people to do is have good, knowledgeable beta readers who can help them determine whether or not it's there yet.

Darkshore
07-05-2011, 08:39 PM
I don't think that writing ability is something you can test. I mean grammar, structure, and that sort of thing can be learned. But I don't think that you can learn to tell a good story. In my honest opinion you can either enthrall people with worlds, characters, and plots that live in your mind, or you simply cannot. For example I've always been a pretty good bullshitter. In my last History class we had to read a book about the 1920's and write an in-depth essay on it. Well I couldn't actually afford the book at the time and could find nowhere that I could borrow or read it for free. So I look up the basic plot online and received a 100% on the essay. So this sort of thing kind of proves to me that a test of writing ability will show almost nothing in most cases. EDIT: Excuse the rambling...it's early and I am still in need of coffee. :D

Little Ming
07-05-2011, 11:14 PM
They can be. Witness A Clockwork Orange, Trainspotting and Feersum Endjinn, for example.

Don't forget Daniel Keye's Flowers for Algernon and Mark Twain's works. I found Catcher in the Rye near unreadable, but for some reason lots of people like it. :Shrug:

Jamesaritchie
07-05-2011, 11:40 PM
They can be. Witness A Clockwork Orange, Trainspotting and Feersum Endjinn, for example.

There's a HUGE difference between intentional bad writing and grammar, and unintentional bad writing and bad grammar. It's always obvious which you're reading, and never subjective. No one reading a slush pile is ever going to confuse the bottom forty percent of what they find with something like A Clockwork Orange. Or even Huckleberry Finn.

Eddyz Aquila
07-13-2011, 02:44 PM
You're talking about the Dunning Kruger Effect. (http://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect)

No, a standardized test wouldn't be any sort of indicator of writing ability because once it's standardized, it's no longer subjective. You can't quantify voice or style; they're unique to the writer. All you can do is improve technical skill, and sadly, one of the biggest obstacles to those writers you mention is the fact that they were told by teachers in high school or college, using those standard tests, that they possessed those skills.

Academic writing =/= commercial writing.

Quite, I agree here with what Cyia said. Except that if you're a capable academic writer than at least you possess some sort of technical skill that can translate towards the commercial writing venue that you are attempting.

Nevertheless, it's all subjective. And I do believe that agents and editors are good enough tests without having to create others.