Nobody's going to sue you. The only way you can be sued is by spreading falsehoods and presenting them as truth, and doing so in a negligent or malicious manner. If you're writing fiction, you are, by definition writing something that is untrue. Real people get used in fiction all the time -- presidents making bad decisions, rock stars kissing waitresses. You don't need to worry about being sued.
In addition, if the things actually happened to you, then truth is an absolute defense against libel. They'd have to prove it was false, prove that the fictional character you presented was based on them and that they were damaged by what was written. So no one is going to sue you.
Now, people getting angry at you and threatening to sue, that's a different story.
I have a pal who is a published, award-winning writer -- we're not close, but she takes my calls the few times I dial her. Everything she writes is thinly-veiled fiction -- she writes about her life and loves and changes the names (my father was one of them. I named one of my twin boys after him, and the other after the name of her character based on him). I went to a reading she gave a couple of years ago, and in the Q&A session someone asked her if people recognized themselves. "Sure," she said. "Does that cause problems sometimes?" "Sure." And that's all she said.
If you're going to write about things that are real -- even if they are fictionalized -- you have to expect people are going to be mad at you for it. if you're going to write about things that happened to you, you can expect your close circle will be mad about it. That's the price of offering private events to world scrutiny. You can either deal with that or you can't.
But you don't need to worry about actually being sued. Unless you're talking major best seller, even a desperate lawyer would be dumb to take it.