What should I be doing ?

^ Never settle for a guy, for the reason you just gave.

and

Like someone else said, you should get to know him a little more, see what he’s really like.