That indirectly proved that invaders did not have to do anything with spreading Islam. They just came to loot. Another reason may be that at that time mosques were bulit with sufi shrines and dargahs and probably Muslim population lived near those shrines and thats why we don’t find separate mosques.