There is one thing your counselor said that is true: You DO need to have a backup plan for your own sanity. Consider the following factors:
Your children are watching you. You husband may be a good father, but is he a good husband? Are you willing to let your children grow through their formative years thinking that what he is doing you, his wife, and their mother, is morally acceptable? If he will not respond to God's love and mercy, as well as yours, then there may be a time where he will need to face justice.
There is nothing wrong with tough love, and that includes the issuing of ultimatums. If he is unwilling to keep his marital vows, then he is no longer entitled to the home that you are providing. Marriage is an all or nothing package. You cannot chose which vows you keep and which ones you ignore. It is nowhere near sufficient to say, "At least I never cheated." That alone will not save him, or the effects of what he neglects from his duties. My wife gave me a final warning some years ago, and I was shaken to my core. I have never forgotten, and it was exactly what I needed, because I actually wasn't even listening the other times.
Note to everyone: this does not have to mean divorce, but there has to be some sort of standard somewhere. As for the OP, I am sorry to says this, but as much as you want to make it work, it will not work no matter what you do as long as he is willing to walk away. Also, you cannot hide this from the kids forever. If they don't know, it's time to tell them what's going on if they're past the age of reason. Nothing can straighten out a parent better than their own child pointing out their wrongdoings. I can vouch for that as well.
Don't give up, but don't put yourself in a position where you will break if things do not work out. You still have children who will need an explanation for what is happening so that their faith will not be shaken.