I use DreamHost for all of my sites, but aside from the very basic things, they're kind of higher up on the technical learning curve.
If you're looking for the easiest, most reliable hosting possible, you can purchase hosting through Wordpress.com. The packages vary, and it tends to be a more pricey option than a traditional webhost, but you'll have complete uptime, a CMS-backed website, and you won't need to worry about things like updates as they are all pushed from central.
The other benefit of hosting with Wordpress is that you can start free and add features as your site grows without having to really worry about things.
It's not the ideal solution for everyone, but it is something that's frequently overlooked.