IF you really think about it, the clothing restriction is for prayer meaning going infront of Allah:swt:.. It does not say you u have to wear such and such because you will be praying infront of people but you wear such and such because you will be praying to Allah:swt:
When we go to job interview, we put on the best clothing then how in the world it can not be true that we can not dress up properly, modestly and decently when going in front of Allah:swt: 