deborah:
Welcome to AW!
I fully understand what you are saying. The road to publication is so long, with so much work (improving craft, learning story-telling, researching agents/editors, understanding the industry, learning the craft of letter writing, etc.), that you don't want to go down it until and unless you know you have some chance of success that depends on you and not on chance. I've been there, and to some extent am still there.
Unfortunately, I don't see any easy answer to this question. Critique groups, in real life and on-line such as here in "Share Your Work" have limited value. You don't really know if the people critiquing you know more than you, or if they "have it". Paid critiques at writers conferences, I have found, are not worth the money, because the critiquers rush through a bunch of them quickly and critique by rote, rather than by the true pros and cons of the piece. Beta readers are good, but you still have the concern about how much they know. Beta readers who are not writers, but who are in your intended audiences, may tend to overlook faults that would kill the piece to an editor. Beta readers who are writers may tend to focus on the trees and not help you with the forest--at least that's my experience.
So it seems to me about the only ways to know if you "have it", the talent to achieve publication, are 1) pay a mentor, 2) have something accepted for publication, or 3) the accumulated evidence of critiques, which includes comments from editors.
I have yet to find mentors who work for free, but possibly you'll find one. Many successful writers who are not quite earning enough to be full time will mentor for a fee. This takes a lot of research on your part, and possibly some luck of timing.
Beta readers who are writers don't work for free either. This normally requires reciprocity: you want someone to read your work in progrees, you must reciprocate and read theirs. So the cost is your time. Beta readers who aren't writers will do it for free, but these tend to be your colleaques and family, which somewhat diminishes the value of their critique, unless they can be truly honest.
Having something published before you know if you "have it" sounds a bit like an oxymoron. But maybe, while doing the work for the book-length piece, you can also pursue publishing shorter works in regional markets. Acceptance by them, while not guaranteeing anything, is some feedback for your decision making process.
To me, the best evidence is accumulated comments from agents/editors. If you submit your work to them, and receive feedback, this will really tell you something. Even a form rejection letter tells you something: the work wasn't good enough for the agent/editor to take the time write something personal. But if the agent says, "Your writing is strong, but I just don't know if I can sell this book," and if you hear something similar several times, then you probably do "have it", and just need to persevere.
Good luck,
NDG