First off, this isn't meant to be a genre vs literary thread.
So, I'm almost finished reading Andrew Miller's Pure, which won the 2011 Costa Prize [and believe is shortlisted for The Walter Scott Prize too.] It's old style Gothic [think Great Expectations or Wuthering Heights], set in the years preceding the French Revolution.
As I was reading last night, I was finding the plot becoming more, and more convoluted, my supension of disbelief going right out the window. At which point I thought, 'you wouldn't get away with this in a genre novel.'
Yet this book has won a big literary prize.
So what do you think? Do we let literary fiction stretch believability further than genre?