Yes! “you remind me of my brother/dad” works like a dream! 
I guess constantly mentioning past relationships, even if you have to make them up, then you can always remind him how he compares with these past boyfriends/imaginary boyfriends… I’m sure that would put him off.