This is a subjective question. Therefore, there’s no single exact answer to your question. You will find people who would say that the height is not so far apart for it to be a huge deal. And you will find others who will say that they prefer a taller guy.
I think what would look worse than height…would be an incompatible couple. If you find him compatible in all other ways and are able to overlook height…then go for it. If it’s hard for you to overlook this quality…then consider someone else.