I heard a priest say once: “We are called to love everyone. We don’t have to like them.”
Think about how you can love this priest. First and foremost, you can pray for him. When I’m praying for someone who has angered me, or someone I don’t like, I ask God to give me the grace to love ALL His people even if they annoy me. As St. Josemaria Escriva says, “Do not think ‘oh, how he annoys me!’ but ‘oh, how he sanctifies me!’” Whenever someone gets on your nerves you can use it to glorify God by showing them love anyway.
Also, be careful that you do not dwell on this dislike or talk about his flaws in a gossipy way with others. There was a priest in our parish that was not very likable. I noticed that if I tried to look past his flaws and remembered that, flaws and all, he is still bringing the Eucharist to me, I felt more at peace. But if I started talking to my husband or inlaws about the things this priest did to offend me, I felt more and more dislike for him and it was harder to love him. Don’t let his flaws lead you into sin, in short.