I would include it. This isn't high school. University magazines are a bit more picky. Just keep it to one line in your query letter: e.g. My short stories have appeared in (Name of magazine), and leave it at that.
but if the agent hasn't heard of it, the only thing it tells them is not "I pubbed in 'Anonymia'." It tells them "The closest thing I have to an actual credit is this bit of hand-waving..."
THAT'S why people are saying if it is a no-name you may as well ignore it. if it doesn't impress the agent, it can actually leave small misgivings, instead, and not having a track record doesn't preclude you from selling, so this may not offer enough pro to make up for the possible cons.
*edit: as an add-on, I used to love the Drew Carrey Show. One of my all-time favorite scenes had Ryan Stiles on the couch with something frozen lying on his pants; I forgot why. Drew opens the front door and surprises him, and when ryan hops up, Drew asks him what he was doing. Ryan's answer was "Well, I sure wasn't just freezing my crotch, why?"
I like the scene because it was a great illustration of how what you "say" can actually communicate a vastly different message; in this case, the exact opposite one, in the OP's situation, maybe desperation, perhaps even a bit of duplicity, perhaps reinforcing instead a sense of mediocroty ("thus far, my work proir work has only made it into this place you've never really heard of anyway..") and at best a bit of naivete....all for (assuming this isn't a highly regarded journal, although it may be) no real gain, because the agent doesn't know anything GOOD about this journal.
I'm not completely talking out of my ass here; I assume the same sort of rules apply here as in biological publishing--I may mention my first author paper in Journal of Biological Chemistry because it is considered a "second-tier" or "third-tier" journal, but would not put a publication in "Journal of Lithuanian Biological Sciences" because the only thing that really screams is "this article was not good enough to go into a journal you've heard of....."