I agree! It IS tough to accept. But think about it. YOU have the power to forgive those who trespass against you. God has the power to forgive those who trespass against Him. Through the sacrament of confession we are ABSOLUTELY ASSURED that He has heard and forgiven us since He gave the ability to ‘bind and loose’ to the Apostles (who passed on that charism through the laying on of hands).
Remember it is not the man who is the Priest that forgives you, it is God Himself acting through the Priest (and through the Bishop who ordained him and who ordained him going back to the Apostles) who forgives you.
As a convert from protestantism, I have to tell you that confession is an absolutely amazing and humbling experience. I had never experienced anything like it before, even coming from a ‘high church’ Anglican background!