You must examine verses 80:1 to 80:3 together. "He" is used in the first 2 verses as in "He frowned ..." but the pronoun clearly switches to "You" in the third verse.
If you assume the "You" is referring to the prophet(pbuh), then the "he" is the other person.
Knowledge was imbued into the ahl-bait (pbuh) whom the vast majority of the ummat ignored. Without their gleaming guidance, no one can claim they understand the true meanings of the holy Qur'an.
May Allah (swt) hasten the appearance of Imam Mahdi (as).
In order to understand anything in Al-Qur'an and to be certain that one has the correct understanding, he must refer to the people of knowledge. In our case, reffering to the interperters of Al-Qur'an we understand that Surat Abassa began talking to Prophet Muhammad. Allah said in Surat Abassa ayah [1] which means:{(The Prophet) frowned and turned away, [2] Because there came to him the blind man (interrupting).}
According to Tafseer Al-Jalalayn: The ayah is reffering to the Prophet.
And according to Tafseer ibn Katheer: "Interperters of Al-Qur'an said that the Propeht was giving a speech to some of great Qurayshies about Islam. Ibn of Um Maktoom came; and had just embraced Islam, he asked the Propeht and incisted (interupting), than Allah revealed Surat Abassa Wa Tawalla".
Note: It was not a sin that Prophet Muhammad did. Because Porphets are impeccable against sins other than Kaba'ir and Sagha'ir (Major sins and small abject sins). And Allah knows best.
Shah Jahan,
Contrary to what you say,the Ayahs would become very confusing and unclear if we were to assume they were adressed to two different people.They are very clear as it is,and the meaning has been explained before(by EA,myself and Ahmad G).The Quranic language has it's own traditions.The change you mention is nothing unusual.And because of that it leads to no unclarity.