I dissagree on your point of binding and loosing being connected with keys. To explain my phrase- I believe they are connected in Peter but not that Apostles received keys also. May I ask why do you believe Our Lord left out this part of phrase said to Peter when he talked to the Apostles? After all, I do not believe he also said “you are Rock and upon this Rock I will build my Church and gates of hell shall not prevail against it” to all Apostles when he implied binding and losing. After all, only one of them received name Peter (Rock)- one with keys, not everyone who would bind and lose. Based on this, I believe that keys are connected to being “Rock” and also Church being built upon Peter instead- of being connected to binding and losing, which was given to all of them. Of course, that is just my impression that right now became a theory.
Also, while Patristic scources claim that Apostles received keys they mostly claim it in context of Church having the keys (therefore all Apostles as guardians of truth having them, but in sense of having them through unity with Peter). Patristic scources also claim that to be united in Church, you should be in unity with See of Rome and submit to Bishop of Rome in his Cathedra estabilished by Our Lord.
Then again, I know Church Fathers are witnesses to the life of Church and are great models for us Christians- yet they are not infallible.