This just happened to me recently, and was both exciting and nerve-wracking.
When I had an offer on the table, I told the agent that others were also reading the manuscript and it was only fair to give them a chance to respond. She was very gracious and understanding about this. I had four full requests and I was up front with all of them abouth other agents reading it when each asked for it.
I then let the other agents reading my manuscript know immediately that I had an offer and wanted to let them know in case they were also interested in the project. I've heard several agents say they hate ultimatum timeframes, so I said I'd like to make a decision by X date (I gave about two weeks) and was that enough time for them to review the manuscript? I figured if they were interested they'd jump on it, but who knew what else they have on their plate and I wanted to give them time. All of them gave me dates as to when they could get back to me, and they were all fairly quick. If any needed more time I would have given it to them.
I told the original agent when I expected to get back to her. I tried to keep everyone in the loop and be up front about what was going on (No names though). I got two more offers, agonized over my decision, and made a choice. (Don't rush this no matter how exciting or scary it is. This is one of the most important decisions you'll make in your writing career). The whole thing took about ten days.
After I made my choice, I let the other agents know in a polite and professinal manner that I had chosen another agent that I felt was the best fit for me, and thanked them for their offer and time.
If you've accepted an offer without considering others (say your first choice makes an offer first), it's only polite (and professional) to let any agents reading your full know so they don't waste time reading your work. It's pretty annoying to spend a weekend reading a book, falling in love with it and then finding out the writer accepted an offer weeks ago and never told you. Partials you can skip notification unless it's a large partial or you had a personal connection to the agent. Say met at a conference face to face or something. Then a quick note is nice.