Islam asks men to lower their gaze, irrespective of what the woman is wearing, and Islam also asks women to cover herself properly, irrespective of which type of na-mehram (in terms of character) she is going to come across.
As a scholar mentioned that according to a hadith, the peak of Islam after the Prophet (saw) would be the time when a woman would travel alone from one city to another while not wearing "proper" dress, but still not a single man would dare to raise his gaze and stare at her lustfully. This actually happened in the times of Hazrat Umar Farooq (r.a).