I had to cut characters out of my current WIP (which is out for beta) during the editing process. It was too long to be considered by anyone and so that meant getting rid of three secondary characters (including two I really liked) and several other tertiary characters who were removed as a result.

I think the number of characters is influenced by the length of the book (there's probably a formula you could work out in Excel! ). Smooshing characters together may be frustrating, but ask yourself, would you prefer it if your reader had to stop to think about whom you were referring instead and thereby disrupting the flow of reading? I'm revisiting some old Tom Clancy novels at the moment and there have been several points where I've thought "who the hell is this guy?" and had to stop to remind myself - although that may be to do with getting older!

Following on from Woollybear and ChaseJxyz, I would suggest a couple of techniques I have used.

Use something about the character as a cue. My WIP is set in the US, and so I've used nationality when describing a character as a reminder. I've also double-down by adding things like job title i.e. "The British detective", in the hope of reminding the reader who I'm referring to.

Catchphrases - for a minor character, introducing a catchphrase or verbal tick might be useful. I have a FBI agent who finishes most sentences with "I can tell you" i.e. "that guy is a son of a bitch, I can tell you"