Back from my day long moping session. Now I've got two questions.
1. Why didn't Jon recognize his brother's wolves? "Gee, two dires wolves, one black, one white, I wonder who those could belong to?"
2. Why exactly did he leave? He had dealt with the wildlings aside from one and we know his girlfriend was on his side, it just didn't make sense to run away.
I tried really hard not to have spoilers here, but there's a bit of discussion about concurrent stuff in the books.
I don't think we really had time to know if he recognised the wolves or not. In the book they all think by this point that Bran and Rickon were killed by Theon... they haven't said that explicitly here (I think it's just been said that they are "missing", presumed dead or captured), but I think the assumption is that, either way, the wolves got away, so even if Jon recognised them, he would assume they are wandering around loose without the boys. Remember he doesn't know about Osha, or that Hodor carries Bran around. The last he heard about Bran, the boy was bedbound for life.... he has no reason to think he would turn up hundreds of miles from his last known whereabouts.
Also, presuming that Jon is still loyal to the night's watch (I think after his actions here that's a fair assumption) his first duty is to get the heck over to them and let them know that Mance is approaching from the rear.
I don't know if you caught that they have WAY fewer men than he was letting on to the wildings (at some point early on someone said "less than 1000 in total" and that's across three castles and before half of them were killed beyond the wall... and of that, a lot are very old or very young) but when you factor in the huge numbers Mance is leading and the giants and other things he's got in his army.... well, even if Jon had mostly defeated the wildings in his group, he can't afford to be tied down guarding them or whatever... and frankly, he can't even really trust Ygritte. He
has to make it back. Or the wall may fall.
I wonder how much of what's going to happen in this storyline will play out in the next episode, actually...