Maybe he has a crush on you too...
That's a possibility, which has crossed my mind as well. But......it's not a thought that you should be entertaining, Reshma. Even if he did like you, he can't act on it because it's unprofessional and he has a responsibility to his wife n family. You don't want to be the woman who broke another woman's home.
Also, you'd be toying with your career and reputation at work. Even if an affair were to take place, it would more than likely blow up in your face and have greater consequences for you.
***Yes, I know you said that you don't flirt with him and nor do you act on your feelings. I'm just putting out the possible consequences for you.