Weighing in my own opinion here, but I don't think it's a must. If it was, I would have gone broke on the series I'm working on since the characters travel to Germany, Africa, Antartica, and a few other places in the US.
I think it IS important to do your research. All of the resources that everyone suggested are great ideas. If you can find someone online from the place you're going to, they could always read and check for inaccuracies.
That said, two people can go to the same place at the same time and pick up on different feels from it. They might notice different things, or may even read different things into certain landmarks, etc. Like my hometown for instance-there are a few different sides to it. On the one hand, it has charm and a history of being a circus town and a few other things that set it apart (i.e. an old brick road going through downtown). But then it's also a community surrounded by farms, so someone else might pick up on the farmer vibe. and then on the other hand, some of the population is made up of very low income immigrants (some whose residential status is less than legal) and then on the other hand, we have a lake that is surrounded by ridiculously large homes, most of them vacation homes.
So this is a small town and there are several dynamics going on here, depending on where you live and what your own personal history is. If someone were to write a novel about it, the characters would dictate what feel is given to the town and what is seen of it.
With bigger cities, this will be even more pronounced because the populations will be larger and more diverse. If writing about an exact neighborhood, it's important to get the facts right, but there's room to create your own neighborhoods, etc.
So while it's important if you're putting certain facts into the book (i.e. saying that such and such neighborhood is near such and such bay when in fact the neighborhood is miles away from the bay) then check those out. But I would say that for most things, do as much research as you can but feel free to take some creative license.
Anyways, that was a really long way to say that I don't think you need to break the bank visiting anywhere you want to set your story or only write about places you've been. You just need to be willing to do some research and fact checking