My response is . . . it depends. Sometimes I get ideas for characters and then build a world for them to play in. Sometimes, I get a "what if X or Y happened, how would people react to that" and then I build characters and a world to go with the plot. Sometimes, I sit down a just doodle up a map and something about the way the terrain and waters lie tell me the story of the people who live there.
Being the child of a highway engineer and having been taught to read maps since I was four years old, I do admit a fascination not only with creating maps, but also with researching old maps. One of my claims to geekdom back in HS (which wasn't nearly as cool as it sounds) was being able to not only name all the countries in Africa as a freshman, but to name the colonial map of Africa (pre-1960s).
How to build a world is a very mobile target. It really depends on how your approach to your own world works. Me? I prefer the outside in method. If I know I'm creating a secondary world fiction, I start with the outline of the continent/region/island and then figure out where the mountains are. Once you have seacoast and mountains, you can see where the rivers might be. Once you have mountains and rivers, you can see where cities might logically fall. Once you have cities and terrain features, you can see where country boundaries might fall AND might be disputed (automatically creating a tension point for the story). Then I start populating the world (and noting if it makes sense for them to be native or immigrant bodies of people (again, more tension), and then where and what resources might be available for any of the countries (as well as what they might be missing) which encourages trade and potential holds . . . (You have all that gold but you have little grain. If I don't ship you wheat and corn, how long can you eat gold? Oooh, more tension), and don't even get me started on potential religious disagreements.
*Will someone tell Tiddlywinks to pick up all these plot bunnies that are bouncing around here?*
That's honestly a quick synopsis of Terra Incognito, a book based on a collection of world building essays I wrote for a now-defunct e-magazine--I published it in 2015 and it's still my best selling small-press book. I'm going to be going up to MA in a few weeks to speak to a college about world building--they're using Terra as a text book for a Video Game Design class.