Contemporary Fantasy, as I understand it, is where magic and magical beings infiltrate into the reality. In other words, where you have a modern story where various paranormal, supernatural, or occult elements are real.
Urban fantasy is the same thing only it takes place specifically in a city, thus urban.
I could be wrong though.