Don't talk to me!

Here's a pet peeve of mine: People that talk to me while I'm in the restroom. I'm already a bit pee shy in public restrooms. It doesn't help when someone insists on having a conversation with me while I'm trying to do my business. Is this weird? Or am I right in saying that even if it is a public restroom, we should pretend that the stalls are private and no one should talk unless it is an emergency (you know, "spare a square" or "anyone got any feminine products on hand!").