When I began actively querying in 2009, it took me over two months of research to narrow down a list of 100+ agents who might be interested in fantasy, in all its subgenres. I am out of the agent game right now, but I update that list when I get more information.
I still haven't been published in SFF; I had to switch genres to LGBT erotica to get my debut space opera mms published. This was in late 2011 and early 2012, pre 50 Shades, and I only found 7 remotely reputable agents willing to look at M/M erotic romance. My previous agent, with whom I'd parted ways years ago, did not even want to deal with straight romance. I got my current agent based on a publisher offer, my own persistence, and a comedy of errors on my part. That she is willing to market both genres is luck I didn't deserve.
50 Shades opened the market considerably, but I am not sure many of the agents jumping on that bandwagon are qualified to make decent deals - even when the publishers are willing to work with them.
With respect to the OP, there are still far more agents looking for non-romance SFF. Finding them requires research.