The best way is to take a couple of successful examples and find out why they work. The first two that come to mind of female characters for boys are Tank Girl and Lara Croft. Okay, so one is a graphic novel character, the other a video game, but they both manage to hold boys' attention where Barbie doesn't. The reason is quite simple: they both play out typically male fantasies - adventure, action and revenge, with lots of danger, guns and things blowing up.
For the other side of the coin, the only example I can think of off the top of my head is "Lucas" by Kevin Brookes, where the MC is male, but he's badly done by, misunderstood, while also being caring, thoughtful and intelligent - all of which are stereo-typical girl traits, and fit with their favoured reading.
I fell into a bit of a hole with this very problem in that I wrote a thoughtful caring, intelligent male character, who was misunderstood and badly done by, and expected it to sell to boys. Although two editors loved the book, they both came back with the same point: a well written story targeted at the wrong audience. In short, it would never sell. The choice was simple, to rewrite it with female characters (not really possible with the plot I had) or to filter out the introspection and worries, and increase the pace, tension and sense of danger. Luckily, these did fit with the plot I had, so it wasn't a hard choice to make.
Did it work?
Still waiting to find out