I can't claim any special expertise, but my impression has always been that a familiar (or "familiar spirit") is a particular being that forms some kind of long-term relationship with the magic user. In the KJV of the Bible, there's a reference to a woman who "had a familiar spirit," with the connotation that her state was akin to being demon-possessed (Paul and Silas cast out the spirit, according to Acts, and thus simultaneously freed the woman spiritually and deprived her of her occupation as a soothsayer). Then again, that's the classical Judeo-Christian take on the subject, so it might not be strictly applicable to Tolkienesque fantasy. But I think a familiar is a long-term "monogamous" relationship, generally.
Other magical beings might be summoned for temporary service, but they're not usually called familiars, in the stories I've read.