First we have to understand that the Bible is not a proof text for doctrines/dogma. Rather, it is the witness to Christ and his Church. The Church preceded the NT, was its author in its Apostles and Saints. Their writings were never meant to be exhaustive in matters of faith and morals. To use the Bible for that which it was not written or give it authority God never gave it is to misuse it, intentionally or unintentionally.
Peter’s authority to “bind and loose” came directly from Christ himself. To try to argue otherwise is to dispute the plain words of Scripture as well as the Early Church Fathers, of whom the earliest ones sat at the feet of the Apostles. One reference, is: [Mt. 16 And I tell you, you are Peter, and on this rock I will build my church, and the powers of death shall not prevail against it.
 I will give you the keys of the kingdom of heaven, and whatever you bind on earth shall be bound in heaven, and whatever you loose on earth shall be loosed in heaven."
How many references does it take to make Christ’s words effective?
As for priests having the authority to forgive sins in the person of Christ (not on their own authority as men) the reference is: Jn. On the evening of that day, the first day of the week, the doors being shut where the disciples were, for fear of the Jews, Jesus came and stood among them and said to them, “Peace be with you.”
 When he had said this, he showed them his hands and his side. Then the disciples were glad when they saw the Lord.
 Jesus said to them again, “Peace be with you. As the Father has sent me, even so I send you.”
 And when he had said this, he breathed on them, and said to them, “Receive the Holy Spirit.
 If you forgive the sins of any, they are forgiven; if you retain the sins of any, they are retained.”
The Apostles laid hands on others to carry on this commission and institution by Our Lord. It’s just that simple.
Only grave sins need be confessed to a priest/bishop because grave sins cut us off from God’s love and from the graces of the Church, which Jesus established in the sacraments.
We can certainly ask God directly for forgiveness of sins–an act of perfect contrition will even satisfy for grave sin if one cannot go to confession for good cause, such as at the point of death, although no one is to presume upon that for that in itself is a grave offense against God’s mercy.
We know that confession with the priest endows us with the grace of God’s forgiveness, so it is reassuring to go to confession. It is good to receive God’s grace for us and for everyone. It keeps us honest with ourselves to have to admit before another that we have sinned. It is very easy for us humans to excuse our sins, but when we go to confession we have to acknowledge our sins and our need of God’s grace.