I had a problem like this with our twins. They got it into their head that Mass was a place to mess around. I thought at the time that it was because we had a "cry room", but I think it was because they were just very active kids. Sometimes, it takes "a body per baby" to keep things under control. I think this is why Mass is not an obligation for small children, and why the catechism specifically mentions the care of infants as a valid reason for missing Sunday Mass:
2181 The Sunday Eucharist is the foundation and confirmation of all Christian practice. For this reason the faithful are obliged to participate in the Eucharist on days of obligation, unless excused for a serious reason (for example, illness, the care of infants) or dispensed by their own pastor. Those who deliberately fail in this obligation commit a grave sin.
I have found that whatever you do, when the kids act up, it must absolutely NOT be entertaining. For instance, it would go better for you if the 2 year old realizes that if she acts up, there is no one EXCEPT Daddy who is going to take her outside, and even out there she is not going to get to get down and run around. Behaving well needs to be the thing that works out the most to her liking, 100% of the time.
I say this as the mother of altar servers who now do very well:
1) This too shall pass, but in the meantime, you need the nourishment of the entire Mass, not just Holy Communion. Getting to Mass without the kids used to be my sanity saver. Do not imagine that essentially missing it will be a good way to "suffer for the Lord". Only when it cannot be helped would I look at it that way. Fasting can be beneficial, but not starving yourself. There is no place in Scriptures, at any rate, that it is recommended to take a fast from the Lord.
2) Whatever you do, do not allow your children into any habit of behavior at Mass which is not reverent and appropriate. I left our kids home from church for probably a year in order to "reset" their expectations on that one. When they came back, it was a new ballgame. I was a little hesitant, but it worked well.
3) Either set the child up for success, or leave the child home. A child ought to be required to go to Mass and behave well for some stretch of time before he or she makes his or her First Holy Communion, but a child is not going to be spiritually handicapped by being denied the opportunity to be a toddler terror for two years. Having said that, a very small child needs something to occupy him or her. I liked those gizmos where you can draw with a stick on a string, then erase. Can't mark up the pews with those.
Consider doing this. First off, the Mass date idea is a great one. You want to be able to attend Mass with your spouse and without children too young to sit through Mass without acting up on a regular basis. But also consider (I come from a family of eight and a parish full of large families) leaving the younger children at home until the older ones are old enough to help mind them. In other words, when you don't have a "Mass date", split Masses, having one spouse at home with the kids while the other goes to Mass with the children who are old enough. Then the other spouse goes alone or with remaining older children.
Consider, for the kids who are old enough, making good behavior at Mass reason for a treat. First off, consider that kids who have been good get to go to Mass alone with you. That alone is a treat. Yes...*alone time with Mom! *Wow! When you have four brothers and sisters, believe me, that is gold. The same goes for alone time with Dad. When older kids habitually act like human beings on their own, I would even take them on the "Mass date." (Arrange a date night without them at another time.) I used to only allow our kids to have the after-Mass doughnut if they could tell me something that Father said during the homily. (For awhile, I had them write a little note in a notebook, one page for every Sunday, which I want to go back to. It is nice to look over the progression.)
Preparing for Mass: Get online and read the readings with the kids old enough to understand them with translation. Start reading the response only from the Responsorial Psalm. A priest taught me that this gives a key to the connection between the Gospel and the first reading. This works! Now, read the Gospel. If reading to your kids, read a line, explain it, read a line explain it, then talk about the whole thing with that Responsorial Psalm line. (This is a good time to bring in what the Gospel has to do with the liturgical season you're in, asking the kids to predict what color vestments for that week.) Only then do you go back and read the first reading, and then the response of the people--kind of a "ooo! I see how the Gospel is going to fulfill this! I can see this coming!!--that is in the Responsorial Psalm. Read the second reading last, noticing that sometimes it is chosen to go with the other readings and sometimes the Church is working her way from week to week through a particular epistle, instead. (That very often adds an insight that wouldn't come from choosing a reading with an obvious connection.)
If you've done that, not only will it not matter so much that you miss a line here and there during the Liturgy of the Word, but more importantly the kids will not get so bored with it. They'll be able to catch a snatch here or there and know where the readings are headed.
Also teach the older ones what different gestures and parts of the Mass mean. Stress that the Mass is, as the Church teaches, one single act of worship. All the parts work together, but the work goes better for those who understand how the Mass works.
Your goal here is not to physically have your kids at Mass, but to teach them to love the Mass as they grow, and to automatically participate well. If that means leaving them home until you can give them that, this is OK.