I looked up a bunch of articles on this, and from what I can tell, there are three big rules to follow:
1. No dead giveaways that fictional character A is actually real person B, such as name, physical appearance, or specific/unusual actions that the real person took in real life. "They hooked up at a party" is one thing. "They hooked up at a party in E-24, drank cherry-lime soda mix, played Leonard Cohen records (all on track seven, repeatedly) and admitted they were both bed-wetters before kissing" is another.
2. Mix n' match. Composite characters are what make real life people and their quirks usable but not recognizable.
3. Alter as much as you can. Birthplaces, race, weight, hobbies, etc. See #2.
4. Wash, Rinse, Repeat.
The litmus test is can an ordinary, non-celebrity wake up one morning, read your book, and immediately exclaim, "Omigod! That's ME!" and find out that it's hurt his/her livelihood or reputation? Was there some kind of humiliating secret fact told in confidence between you and the alleged 'victim' in the book, like revealing that s/he is HIV positive? Use common sense.
My impression--and I'm hardly an expert--is that libel in mainstream fiction is fairly difficult to prove. However, it's wiser to save yourself the trouble.
