St. Josemaria Escriva once said, “Choose mortifications that don’t mortify others.” I think, from your description of what happens when you fast, it would apply to you here.
As a husband and father, you need to be physically able to do your work and care for your family. Being grumpy and irritable is not part of that.
Also, any fasting beyond what is required by the Church, that is, Ash Wednesday and Good Friday, is best done under the advice and direction of your confessor or spiritual director.
There are many ways to be a mortified soul and make spiritual progress without doing a complete fast. You can:
Choose to eat foods you don’t care for, or leave off the butter or salt or other condiment.
Skip snacks and desserts - give your share to your children.
Prepare your coffee in a way you don’t like - drink it black if you like cream and sugar, or sweeten it if you prefer it black, etc.
Omit TV or computer time, or watch what others prefer and you don’t.
Fast from having the last word or making clever comments.
Don’t make unnecessary small purchases for yourself.
Put your own needs aside in favor of the others.
I know a priest who says that our families should look forward to Lent, because that’s the time when we treat them the best. We can generalize this to the whole subject of fasting and mortification. God will be much more pleased if you put yourself aside to make life better for your family than if you fast until you make yourself and everyone else miserable.
Ask your confessor what he thinks.