PDA

View Full Version : Results of My Online Focus Group



Jay Jennings
03-22-2010, 12:32 PM
I realize this is a long post, but if you're interested in a "different" way to get feedback on what you're writing it may be something you want to look at.

Let me share my results and then give you the "secret sauce" that makes it happen.

Last week I got 50 people to read the first chapter of my story and give me feedback. They were NOT the target audience, they were all adults, but I think what I got back will help me.

I asked them two questions:

First, what age range does this story appear to be for? Here are the answers:

Younger than 9: 4%
Ages 9 - 11: 30%
Ages 12 - 14: 58%
Ages 15 - 17: 36%
Older than 17: 6%

It adds up to more than 100% because people could choose multiple age ranges. Based on those results I'm writing a story that's not going to crossover into adults, but that's fine with me. I'm not writing it for them. =:)

The second question was, do you think a kid of the selected age range would enjoy the book.

Yes: 26%
Probably: 60%
Probably Not: 14%
No: 0%

I would have liked to see a higher Yes number, but I didn't get any No's and the Probably+Yes numbers are a nice 86%.

Then I asked for comments if they wanted to leave them. 37 of the 50 people left comments. Some were like this:


Disapprove of the story, words like "suck" are misused and turned into slang, felony behavior engaged in, and occultism is encouraged, which would make it perfect for Disney.

There were some like this:


I like it. I sort of wanted to keep going, and I'm 30. Kid brings up a good point. You can just fly. That's it. Not a superpower, but interesting.

-

The opening chapter is interesting enough that I think most kids would want to keep reading to see what happens and how the protagonist became a criminal. The writing is fairly fluid - the opening chapter flows pretty well and is easy to follow. I'm interested enough to want to know how the narrator got into the situation he's in.

And then there were some along these lines:


I thought there were too many asides in parenthesis which were distracting and tried to make the narrator sound too "cool" since the asides were kind of sarcastic. Also, I don't think you need to use language like "sucks" or "dude" just to appeal to kids, it makes the story seem too informal and that you're trying too hard to relate or sound like a teenager. The idea is OK, but I thought the narrative sounded too jokey and sarcastic.

-

While the story was interesting, I found the sidebars, constant asides in parentheses, and occasional comments in bold to be very distracting. Choose one of these tools to make your points, not all three. I know you are writing for kids who are used to quick action but they really don't need this much choppiness for you to keep their attention. Of the three, I felt the sidebars were the most annoying and really didn't add anything to the story. I'd like to see your story flow more smoothly and quickly, and taking away most of your asides would do that.

Those comments echoed some of what I got in the SYW section.

Okay, and here's the punchline for this...

...while this is all still preliminary and I don't know whether these results will mean anything long-term, it only cost me $5. That five bucks total, not $5 per person. I paid each person a dime to read the chapter and give me feedback.

It took 7 days to get all 50 critiques, but I got the first few within a couple hours, so I had almost immediate feedback. I used a service from Amazon called Mechanical Turk. I've used it *many* times in the past, but for marketing purposes.

The one day I was clicking Refresh over and over in the SYW forum and it hit me that while I might not get feedback from writers at MTurk, I could certainly get feedback from "readers."

I figured for $5 it was worth a try and I think the results are, at the very least, interesting and worth looking at.

That's it, just a quick look (in a loooong post) at something that might help if you're looking for more feedback on your writing.

Jay Jennings

Polenth
03-22-2010, 01:12 PM
Then I asked for comments if they wanted to leave them. 37 of the 50 people left comments. Some were like this:
Disapprove of the story, words like "suck" are misused and turned into slang, felony behavior engaged in, and occultism is encouraged, which would make it perfect for Disney.

How dare you encourage occultism like Disney!!!

One thing that struck me, as you were surveying adults for a children's book, you could ask them: "would you buy this book for a child of the selected age range?"

A lot of children's books are brought by adults as gifts for children, so it'd be a handy thing to know.

Jay Jennings
03-22-2010, 01:30 PM
How dare you encourage occultism like Disney!!!

I have relatives that could have written that comment. *sigh*


One thing that struck me, as you were surveying adults for a children's book, you could ask them: "would you buy this book for a child of the selected age range?"

A lot of children's books are brought by adults as gifts for children, so it'd be a handy thing to know.

Yeah, that's a great question to ask -- when I do this again I think I'll either add that question or swap it for the "would a kid like this" question. After all, if they would buy it for a kid they probably think the kid would like it.

Thanks!

Jay Jennings

Parametric
03-22-2010, 01:42 PM
This is intriguing, Jay. I think my worry would be that readers may not have an appreciation of exactly how competitive the market is for fiction. Not everything a reader enjoys reading is worth a substantial investment of time and money by a major publisher. So I'm not sure how much I'd read into the data about whether a young reader would enjoy the book. The critique comments seem good though.

Jay Jennings
03-22-2010, 01:54 PM
This is intriguing, Jay. I think my worry would be that readers may not have an appreciation of exactly how competitive the market is for fiction. Not everything a reader enjoys reading is worth a substantial investment of time and money by a major publisher. So I'm not sure how much I'd read into the data about whether a young reader would enjoy the book. The critique comments seem good though.

Oh yeah, I know what you mean. Stuff like that has to be taken with a grain (or shaker) of salt. But out of the 30-some comments I got, several mentioned the same thing as something that bothered them -- so as a "rough barometer" it might be good.

I'm going to try it again and see what happens next time. At only $5 a pop it's not like I'm gambling much. =:)

Jay Jennings

cameron_chapman
03-22-2010, 08:10 PM
That's a really great idea. I think I might try it out for some of mine. Especially since I have a few novels I'm debating whether to revise or just stick in a drawer...

dgiharris
03-23-2010, 01:44 AM
My first thought is to wonder where the 50 adults came from. It sounds like you paid for a service and 10 cents per reader sounds wayyyy too cheap to be true. So, I would wonder how Amazon is able to provide this service for so cheap.

Assumin that the service is on the up and up, then my next concern is you aren't talking to your target audience.

If i'm writing a story for women but then I survey men and ask men what they'd think what women would think... my data is going to be way off.

If your story is YA, then you need a YA targeting survey. Adults that read and specialize in YA would be ok to survey, but ultimately, you want your target audience to provide the feedback.

If I were to go about something like this, i'd probably go to a kid hang out, for the sake of argument, lets say an arcade.

I'd set up a booth with a dozen or so copies of my 1st chapter. I'd hang a sign up that says, 'Free Tokens for a Survey' or maybe even "Free Candy Bar for a Survey".

I'd then explain to the kids that I just want them to read a chapter, complete the survey, drop it in the box (so it is anonymous) and then I'd give them some tokens or something tangible like a candy bar.

Ultimately, it will probably cost you around $1 per survey.

Another alternative would be to pass your survey/first chapters out to a few kids you know and ask them if they could pass it on to their friends to help out.

Another route you can take is to jump online to some of the teen chat rooms (there are a lot of them) and you can just ask them for your help and post a link that has the first chapter and the survey. Kids love to be helpful and i'm sure you'd find a few that wouldn't mind reading and offering their opinion.

Basically, if you are going to do a survey, it simply MUST be with your target audience or at the very least, people who have a lot of experience reading what your target audience reads. Otherwise, the survey is wasted. Not trying to be mean here, just trying to help.

Mel...

thothguard51
03-23-2010, 02:05 AM
My question with surveys is always...what is the competency level of those taking the survey. If your goal is to get editors to read and buy your work, then general readers are not going to tell you what the editor wants for their publisher. Or agent.

The feedback on the critique might be more important than the rest of the survey, IMHO.

ChristineR
03-23-2010, 02:05 AM
How does this work? Are these 50 people just people who happened to sign up, or do they get paid?

I would think with a 1:50 ratio and only a $5 fee, they'd have a hard time getting enough people to do this. Or are there just lots of people who enjoy the process?

Jay Jennings
03-23-2010, 03:41 AM
Amazon's Mechanical Turk is an awesome resource for "crowd-sourcing" -- it's been around for years and every time I use it I'm amazed at how many people will do small tasks for literally pennies.

There are tens of thousands of MTurk workers and I probably could have offered 5 cents per person and filled all 50 slots -- but it might have taken a few more days.

I once asked for "ideas on budget-minded family activities" and got back 30 ideas that I then expanded on for a project. Another time I got "ideas to make your kid's party extra special," "tips that helped you lose at least 10 pounds," etc. In each case I paid just a few bucks for raw content I could then use for whatever I needed.

As far as the people doing it being the "wrong crowd," I could have specified only parents with kids between the ages of 10 and 15, for example. Are they the target audience? No, but in general, I think parents know what kind of books their kids would like so that might have helped make the data more reliable.

Again, I don't think you could take the data and believe it 100%, but it's another way to get feedback.

I'm not advocating it as an alternative to the SYW forum, just wanted to point it out as a resource that may be useful for some people. If not for feedback, for many other tasks that are time-consuming but need to be done.

Jay Jennings

Polenth
03-23-2010, 05:40 AM
My question with surveys is always...what is the competency level of those taking the survey. If your goal is to get editors to read and buy your work, then general readers are not going to tell you what the editor wants for their publisher. Or agent.

My family does market research tests for products sometimes. We don't take competency tests to do them. We're selected based on whether we fit the demographics for that product (usually age range, but it can be based on family size, number of children, etc).

It's a different perspective to getting a critique from peers.

You do need to be careful what you ask though. The craziest we were asked was: "if this cake bar were a person, what sort of person would it be?"

I wanted to reply "a tasty chocolately one", but I think I was overruled. I don't expect many people answered that question in a useful way.