We do not have any communication issue whatsoever and we are getting a long very well and we both speak English quite well. 
General understanding is very good too as we share the same values and ideas about life.
I think if you have already doubts, then leave it. It is not worth the headache… Love is not everything, there needs to be a common ground in my opinion.