I agree with you, but not with the “meaningless talk” part. It also depends on the kind of job it is. I can take personal calls at work, but they have to be VERY limited, because I have to get my work done. He can take calls a lot more easily than I can, because he has busy periods and then not so busy periods.
We have had many meaningful conversations when he was at work. Haha! We have even lounged under the sun and read together, while he was working.