Timing?
Does it have to be BOTH late afternoon/early evening? Because that timing seems unlikely to me. Short parties are held in the afternoon or early evening. Longer parties that include a full meal are held at meal times. (I used to be a caterer.) It would more likely be one or the other.
A late afternoon event would be a sort of modern-day tea. Alcohol might be served, and light refreshments (finger sandwiches, chocolate covered strawberries, quite possibly champagne.) The dress code would probably be dressy casual. Think polo shirts and nice slacks for men, and summer dresses or linen slacks & silk blouse in ice cream colors for woman. Only very simple (but very expensive) jewelry would be acceptable. It's also more likely to be mostly women, especially if it's held on a weekday, or for a charity like a garden club.
An early evening event is really just a cocktail party held outdoors. Full bar, canapes, maybe a few hot hors d'oeurves. Very little, very high-quality food. (The rich are big on staying thin, not on eating.) If it's dressy, yes, cocktail dresses and suits or tuxs for men. (Maybe it's being held before the opera or another event?) If it's "casual" then a similar dress code to the afternoon event, but with subdued prints and probably jewel-tone colors rather than pastels. The rich have lots of time to change clothes 5 times a day. More elaborate jewelry would be acceptable.
I guess the real question is, who is hosting this party, and why? That will determine the timing, attendance, dress code, etc.