I've just read a book called "Lady's Maid", which is about Elizabeth Barrett Browning (mid 1800s) told from the POV of her lady's maid. The maid is always addressed and spoken of as Wilson (her surname) and, although single, was called Mrs Wilson by people other than her employers. She retained this name even after she married.
Collectively, be they maids, butlers, housekeepers whatever, they were called servants.