As the managing editor of a historical fiction imprint, I'm telling you (my opinion) quite frankly--use Father.
Naturally, you're going to want to use words from the protagonist's tongue to flavor the manuscript, but with Latin, you're going to want to use words that impact the modern day reader and evoke that feel of ancient Rome--centurion, legion, proper names like the Via Appia, and Latin words that have passed directly into English. Not a word like "Pater" when most modern day readers aren't going to make the connection between pater and father unless you tell them.
However, that being said, if you're wanting to play up the concept of Roman duty from son to father and the traditions of honor, paterfamilias might be the direction you want to go. Use Father in conversation and internal dialogue, and paterfamilias for the concept in your narrative. That might be a better option for you.