That is an assumption, there is no proof that only those with Prophets in their offsprings are Ahl-e-Bait.