It's more than just length. The prose used to write a picture book and that used for a chapter book are NOT interchangeable!
A picture book is designed to be read out loud by a proficient adult reader. The writer can use all sorts of big words and rich language, as long as the rhythm and story will make the meaning clear. After all, the kid can always stop the grown-up and say, "What does topiary mean?"
A chapter book is designed for a beginning reader. The syntax must be kept simple, the two- and three-syllable-words must be kept to a minimum, and the storyline must move forward in a very clear way. A kid who is listening doesn't have to work very hard; all their brain power goes to enjoying the story. A kid who is learning how to read has to remember all of the 40 sounds of English, most of which have multiple spelling possibilities, not to mention the non-phonetic words like "their" and "who." The amount of brain power used to figure all that out limits the amount of memory left for keeping track of the story.
The length is not actually a problem, in and of itself. There are plenty of books (Arnold Lobel and Cynthia Rylant are the god and goddess of the age range) that fit in that length. But writing prose at the right DIFFICULTY for that age range is VERY, VERY hard to do well.
You also have to take into account the attention span of a young child. If your story is really one that would be enjoyed most by a 5 or 6-year-old, then find a way to trim it down, get to the point and move on. Believe me, even the advanced ones at that age can't sit still for very long. I tried to have my first-graders lie still for 5 minutes today. Not one of them made it.