Agreed, that nobody should ´spoil´ childhood bonds... But the man has also another bond..... a LIFETIME bond with his wife.....
Besides asking for personal space and time, not interfering in private time/life is not really a bad thing to do/ ask........ I bet the friends would want their own private time as well.......
If they really don´t understand your need for personal time with your husband, I suggest you return the favor...... in time!