For me, horror is about atmosphere: Black nights; thunder and lightning; fog; distant animal noises; absolute, encroaching silence.
There are countless novels out there in the horror genre, full of ghouls, ghosts and all manner of Hammer regulars, that simply aren't scary. Some lack the right atmosphere (James Herbert's Once...) while others have the atmos, but read more like crime thrillers (Laurel K. Hamilton's Anita Blake series).
'Horror' has become quite a generic term and encompasses anything that has a certain type of charcter in it (i.e. vampires, ghosts, werewolves, etc). It's very rare you'll find a horror novel that's actually scary.
I tend to find the scary horror stories are those that centre around a perfectly normal charcter and build a sense of tension with just the right pace, then hit you with something when you least expect it (like the punchline of a well timed joke). The best example of this I can think of off hand is the opening to James Herbert's Ghosts of Sleath. A perfectly normal scene is being described, but there's an underlying sense of something not being right, which builds right up until the scene's climax. The story then moves on as if nothing's happened.