Yep your observations are right, if it bothers you , you should transfer the contents to some other bottle with sturdy cap.
They do not want to change this style of the cap because then people will think that this Rooh afza bottle is not genuine and it is number 2 ka mall.
I will say what TLK has to say about a wine rack in your home.