Hi
Try some over the counter topical numbing gel, suitable for oral use. I would suggest you must go and seek your doctor's opinion. There are numerous numerous illnesses, which can even be multi systemic, that cause ulcers in the mouth. Treating a symptom temporarily won't be of much help in long run. Please see your doc at the earliest, get it diagnoses appropriately, you might not even need a dentists' review at all.
Good luck !