It’s foolish to ignore rumours because it gives the person spreading the rumours more power and audaciousness to spread even more rumours. Most of us don’t care about the minor comments, but when it’s an attack on your character, don’t let it slide.
There is no right answer in how to combat rumours. In certain instances confronting the rumour-monger privately is the way to go, in other instances, speak to a mutual friend and clarify the truth so there’s a neutral party aware of the facts, and in certain egregious situations confront the rumour-monger in public with an audience so that one of the two is proven a liar.
But the only thing is not to stoop to the other person’s level. Maintain your dignity.