What I find interesting is that even legitimate names can throw readers. 'Eibhlin' is one of my characters. It is an Irish name, a girl's name, and it's in current use as well as past use. Phonetically it's similar to Evelyn.
But it's the sort of name that readers get a bit hung up on, when they critique my excerpts.
I don't worry too much if people don't like the names I choose.
Star Wars has a Luke running around with an Obi-Wan. So, I can have a Hortentio running around with an Eibhlin and a Jack and a Hyiab.
I do get a bit knocked out when a world that has no Bible or Christianity has a bunch of Biblical names (likewise for names from other specific historical or mythological contexts that don't exist in said fantasy world), but of course it's possible to have a fantasy name that sounds similar to real-world names, but have a different origin, like Sam Gamgee not being Samuel but Samwise. This happens in real life too, where two different cultures just so happen to have the same name with different origins. So an occasional name that resembles a real world one is certainly plausible.
My issue with the ASOIAF names is that so many of them were Biblical.
It also gets complicated, because some names can be popular in one place and not another, even when they speak the same language, so a name that seems out there or weird to one person is common to another. I have a British friend who ran across the name Yvonne for the first time when he was an adult, and he thought it was pronounced Why vone ee for a long time. But it's a moderately popular name in the US and has been since I was a kid (A childhood friend was named Yvonne), and I occasionally get Yvonnes in my class, even today.
I think there are often historical reason for a fantasy country or region to have a great diversity of names, such as its being a crossroads or a place where many different peoples and cultures have moved in and out over the ages. Reality is often messier than idealized word building conventions. I had a critiquer ask why in a fantasy world, some of my names were things like "White Creek," while others were made up names, like Ialdan, and still others mixed made up names with English words like Port Vlick . He felt that was unrealistic and inconsistent, and I pointed out that both the US and
Britain have a hodgepodge of place names stemming from different languages and cultures, often right next to one another. He said that's because both countries had been occupied, invaded, and colonized by different cultures at different times in history, and my response was, "Yes, exactly, and that's what I am trying to get across about the history and culture of this region of my fantasy world."
But some name glitches stand out more in a pre-industrial culture than others. Even in a diverse country, it would be unlikely to have John, Yvlich, Valala, Pierre, Smort, and Aiko all in the same family. In the modern US, of course, parents often pick names for their kids from baby books with little understanding of the culture or history behind the names. So we may very well have the first names Hunter, Maria, Madison, and Cohen in the same family. But I am guessing this is a relatively recent thing.
Another way fantasy names depart from reality is that in real life, duplicate names are common, both for first and last names. Even nowadays when people have endless name lists to pick from, certain first names seem to end up being inordinately fashionable for periods of time, and of course certain first and last names come up frequently for specific historical and cultural reasons. As a college instructor, I've seen how certain first names appear on my roster over and over for a few years but may fade into the background as new names became popular (What I see runs 18-20 years behind baby naming fads). Duplicate last names are common too. I get a lot of Vangs, Johnsons, Garcias, Smiths, and Kaurs in my classes each semester. But writers are wary of confusing readers by having duplicate names, even for secondary characters.
Maybe the best advice is to think about why you pick the names you do and think in terms of there being some kind of rules behind naming patterns, whether the names be based on real-world cultures and places, or whether they are completely made up, or some mix of the two.