Most of my SF worlds are entirely human based. I rarely use aliens, besides the odd space opera, so creating a believable alien species is beyond my field.
When I create science, I try to base it on something slightly factual. This does not mean that its actually possible, but it becomes easier to accept. Today even science is seriously talking about SF terms like time travelling, teleporting and using dimensions to reach beyond our solar system. Its hard to create a concept totally which is unacceptable for the reader.
I know a lot of SF writers build their world separate to their story, but I tend to let the world grow, as i develop the story. This might not be a great idea, but it saves me from developing too many things that i don't need. It might also make my worlds slightly less believable, but I'm normally willing to sacrifice that for a the actual story.
Strangely enough I do the exact opposite, when i write fantasy. But i think its because I intend to tell many stories from the same fantasy world. Although i might not actually do it. Tolkien set the standard for fantasy worlds, a history, maps, mythology, even languages and cultures. I think you need those things to write good fantasy.
SF can be told with "In a galaxy far, far away." it needs no background. The extended universe of Star Wars was developed after the original story had been written. I seriously doubt, that Lucas spend much time writing about Mandalorian culture and history, when he introduced Boba Fett. All that was added later by other people. That's why we don't hear about midi-chlorians in the first three films. Its introduced later to explain how they can test Anakin. Don't get me wrong, I love the film, but the story is really all over the place. Like Obi-wan not knowing R2-D2 in "a new hope", when he obviously spend the most important part of his life, with the very same robot.
Maybe that's the main difference between SF and Fantasy. In SF you tell a story by means of the world you have created, and in fantasy you tell a story set in the world. I don't think, I have ever reused a SF world (besides using it in a serial), but my fantasy world can contain a lot of unrelated stories.
Telling SF stories is down to the famous "suspension of disbelief". One of my favourite worlds is the Verse from Firefly. Joss Whedon gives no explanation why his main character carries and ancient colt revolver, or why people have reverted to dressing like 19th century pioneers, they just do and its great.