I know that hatred is a very serious situation. Forgiveness would be the ideal solution, but what happens if this is not possible. My question is if hatred consumes you so much you begin to have self hatred how should one deal with this?
Yikes, you are in a dire situation. Fortunately, you realize it is serious.
Let me suggest this: try to forgive Jesus for what the other person did. Imagine that Jesus is guilty of what that other person did, and then see if you can forgive Jesus. After all, is this not what God did with our sins?
That you are starting to have self-hatred shows the justice of God in your situation. For the snare we lay for another is the one we will fall into.
Psalms 34:8 Let the snare which he knoweth not come upon him: and let the net which he hath hidden catch him: and into that very snare let them fall.
I commend you for facing this. Now lay hold of salvation by praising God for His justice (for this same justice will save you from the evil one in the future), and accept your trial while begging God for mercy to change your heart, for you are unable to. Ask for love for Jesus. Practice confidence by accepting any passing feelings of discomfort, despair, fear, etc. and rest in the fact that God does not wish for the sinner to die, but to live. But you must be fair and also wish for this other sinner you hate to live.
If you hate so much that you would like to see yourself condemned so as to avoid loving the other, then you are in a truly terrible state:
Ecclesiasticus 14:6 There is none worse than he that envieth himself, and this is the reward of his wickedness: 7 And if he do good, he doth it ignorantly, and unwillingly: and at the last he discovereth his wickedness. 8 The eye of the envious is wicked: and he turneth away his face, and despiseth his own soul. 9 The eye of the covetous man is insatiable in his portion of iniquity: he will not be satisfied till he consume his own soul, drying it up.
But there is hope, since you wish to repent. You have fallen deep, but little by little, you can be delivered. Start practising natural virtues that are simple: give money to those in need. Purchase Mass cards for yourself and others. Pray for the souls in purgatory. In all this, focus on your willingness, and not how you feel. Treat the feeling of hatred and envy as a pitbull latched onto your leg, which you must carefully endure until rescued. But you see, you must practice keeping that gate closed, for you opened it against your neighbor, and God did not allow you to succeed except against yourself. Praise God for His justice, even if you are the victim, knowing you deserve it and yet hope for deliverance by showing you will amend your life willingly.
Also, pray alot. Pray from your heart honestly, seeking a resolution pleasing to God: “I will forgive if God does. I will forgive if God helps me. I will forgive if God makes it up to me. I believe God can make it up to me. I believe God can help me. I believe God would probably forgive. I believe God most likely commanded me to forgive. I believe God will forgive me if I manage to forgive. I will accept forgiveness because Jesus paid so much for it, and deserves that I should accept it. I would like to want to give forgiveness since Jesus most likely loves that other person somehow. I believe Jesus accepted notoriously awful sufferings because of sins. I believe I am probably too weak to forgive, because I have been so hurt. I believe He will have to do the forgiving for me. I believe He is able to do the forgiving for me. God, I believe, help my unbelief.”
Pray and practice charity while nourishing hope for eventual deliverance.
O Come, O Come, Emmanuel!
And rescue captive Israel!
That mourns in lonely exile here;
Until the Son of God appear.
Rejoice! Rejoice! Immanuel
Shall come to thee O Israel!
I hope this helps.