*feels like the only person in the world who still has a landline*
Th noise you get depends on whether you picked up the phone without a call, or someone hung up one you.
If you just pick it up you get a dial tone, then, after a few seconds you'll get a series of higher pitched notes, like 'beep, beep, BEep' and a voice telling you 'please hang up. please hang up. please hang up.' if you don't hang up then, after a minute or two, you'll get a really loud alarm tone played down the phone at you - a bit like a muted siren.
If someone hangs up and you are still on the line, you'll get a single 'durrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr' for a while, then the beep, beep,beeps and then the voice will come back and tell you to hang up.
Now, this is assuming that they have called you from a mobile or have completely terminated the call - if they have called you from another landline and hang up, you can stay on the line until someone picks up the phone on their end because you are still connected - ah, the hilarity.