Given that the dogma of the Immaculate Conception teaches that Mary was preserved from Original Sin from the moment of her conception I think its safe to say that we inherit Original Sin from the moment of our conception.
Sacraments are the known and guaranteed ways of receiving grace and salvation. However, as St. Thomas Aquinas pointed out, God is not bound by sacraments and is always free to act as He chooses. Thus God could remove original sin from the souls of children who die in utero.
The Catechism of the Catholic Church states:
1261 As regards children who have died without Baptism, the Church can only entrust them to the mercy of God, as she does in her funeral rites for them. Indeed, the great mercy of God who desires that all men should be saved, and Jesus’ tenderness toward children which caused him to say: “Let the children come to me, do not hinder them,” allow us to hope that there is a way of salvation for children who have died without Baptism. All the more urgent is the Church’s call not to prevent little children coming to Christ through the gift of holy Baptism.
The International Theological Commission had to say in its address The hope of salvation for infants who die without being baptized:
The conclusion of this study is that there are theological and liturgical reasons to hope that infants who die without baptism may be saved and brought into eternal happiness, even if there is not an explicit teaching on this question found in Revelation.