PSquared - Email this boss of yours a very polite email, referring to the aforementioned incident, stating the date and time, and other people who were around.
Explain to your boss that you have construed his remarks to be offensive and do not understand why or think its fair that u were singled out in this way.
Further, mention that u have had a good working relationship with him so far but feel that it is deteriorating fast. Add that u do not have any negative attitude towards him etc etc.
To conclude, let him know that u do not expect to be on the receiving end of any more comments such as he has made in the past, and if there are any issues that he has with u, please can we resolve them in a meeting with the presence of a formal witness from HR.
If he has any further comments to make, u will indeed be referring this matter formally to HR.
CC the email to your private email address also for future reference.
This should definitley do the trick. Do not put up with nonsense like this from anyone. Trust me, when is starts as a joke, some ppl dont know where to stop and it can get nasty.