To be sure plenty of ECFs associated the keys with Peter or to the Church through Peter.
To the OP, the literary structure of the Isaiah and Matthew passages are quite striking when viewed side-by-side:
Isaiah 22:22 And I will place on his shoulder the KEY of the house of David; he shall open, and none shall shut; and he shall shut, and none shall open.
Matthew 16:19 I will give you the KEYS of the kingdom of heaven, and whatever you bind on earth shall be bound in heaven, and whatever you loose on earth shall be loosed in heaven.
As well, Mark Bonocore notes some worthy allusions in the early Church connecting the Isaiah passage to the Matthew passage, including from St. John Cassian and Aphraates the Sage, for example. Link to Bonocore’s article here.
*St. John Cassian (ca AD 362-435) “O Peter, Prince of Apostles, it is just that you should teach us, since you were yourself taught by the Lord; and also that you should open to us the gate of which you have received the Key (singular). Keep out all those who are undermining the heavenly House; turn away those who are trying to enter through false caverns and unlawful gates since it is certain that no one can enter in at the gate of the Kingdom except the one unto whom the Key (singular), placed by you in the churches, shall open it.”
Aphraates (ca 330 AD) “David handed over the Kingdom to Solomon and was gathered to his people; and Jesus handed over the Keys to Simon and ascended and returned to Him Who sent Him.”*
More importantly, and Bonocore touches on this, that the Matthew-Isaiah connection is consistent with the unique role of Peter and his successors. Certainly no Church Father denied such a comparison. So if the comparison was made later, in light of modern contextual criticism and literary structures, so what? It complements Tradition, not contradicts it. If genuine debate were to arise in the Church, who will settle the matter?.. :rolleyes: