I have a girl in my office and when we’re talking about our own lives, I’ll speak to her in Urdu. When we discuss work, it’ll be in English - but it’s because our colleagues know us and our dynamic and they know we’re not hiding anything work-related.
And absolutely, when I’m talking to my mum or sisters on the phone, just to be more discrete, we speak in Urdu (everybody doesn’t need to know our ghar kai masallay).
I guess I felt like a stranger was trying to personalize a formal working/professional interaction and I did not feel comfortable with his actions.