I love sarees, but having to sit on the floor while wearing a saree is not the most comfortable attire, so unless you’ll be seated on a sofa/chair, I’d avoid wearing a saree. The only caveat being, if you sit down and then are not expected to get up again and again, then it might be okay.
Sahar’s idea of dhaka pajamas sounds like a great option.