...because you don't like to hurt people's feelings, because you don't like conflicts, because you don't think it will make a difference, or because you think it will come back at you...
is that good or bad?
I say whatever there is to be said right at the person's face, weather s/he likes it or not.
I'd feel bad to hurt someone though. Conflicts can happen either way doesnt matter if its good or bad. Difference can also happen either way.
Whatever it is if your own judgement is right about the person and s/he've been treated fairly it'll never come back to you. :)