I've never had a problem with indulgence but there you go, that's coming from a 26-year-old single man who is perhaps a little too happy in his own company.
Honestly, though, writing is a worthy pursuit. Some people even call it lofty or noble, but I'll leave that to you. As someone just said it's certainly more rewarding than watching television or lounging around. And while it may not seem as important as doing the chores, it is far more stimulating and creative and satisfying I think. If you really like writing, this could even be seen as a second career path for you, regardless of whether you make money from it or not. And even if not, it is still an enjoyable hobby and it's both educational and entertaining. So if that's indulging yourself, then indulge away! You have literally no one to answer to but yourself, you are your own keeper! If writing was getting in the way of your personal life then perhaps it would be a problem, but that's not the case with you.
I'm sure this guilt will pass. I only ever feel guilty if I'm not writing and I'm in the middle of a big project!