Look at the Gospels.
When Christ forgives sinners, how long does it take?
Seriously. Do exactly that. Look at how long and how complicated Christ made things for people seeking forgiveness.
Then think of a modern day priest acting in persona Christi why should it take any longer?
Yes, we do have a set rite (ritual) which is not found directly in the Gospel, which makes things a little bit longer. Nevertheless, there’s no reason why confession & absolution should take a long time.
2 minutes is about right; for the average practicing Catholic.
One thing to note is that he should have asked you to make an act of contrition–even one of the short ones. Maybe he did but you just didn’t include that in your description. If he doesn’t, the penitent should pray one quietly while the priest says the absolution.