Coffee and writing. Coffee and artwork. Coffee and housecleaning, phone calls, breathing. If I could, I'd drink coffee while I sleep. So I've never noticed caffeine making a difference to my writing, mostly because I have nothing to compare it to. I've never written without it, and there are very few other things I do without it. Caffeine is always in my system. In fact, it's time for a nice evening cuppa right now.
But yes, even I, hardcore coffee addict that I am, know about jitters. I suffered that back when I had The Job From and In Hell, and I somehow fell without realizing it into a habit of downing up to 10 cups of coffee per 8-hour day, just to keep going. Yeah, that wasn't good. As soon as I quit, I cut that back to three cups a day, four if they're small.
Jitters are not fun. If you're getting jitters, cut back the caffeine. I heard a news report, oh, years ago, that said some study had shown that very crunchy foods, eaten in the afternoon, will perk us up. Crunch stimulates the central nervous system, apparently. Maybe switch that afternoon cup out for some pretzels and carrot sticks?