The particular case you raise involves complexities that are not well answered in an internet forum.
Most of all, we will want to keep all of you in prayer, but I will mention some things.
While there are general requirements for novitiate and profession in institutes of conscrated life and societies of apostolic life, there are additional requirements that are imposed by the proper law of each. Should your mother pursue this, the director of novices or admissions would be in the best position to discuss the prospects of assuming vows following divorce and what would take place.
Rather than speculating about possible grounds of nullity or the prospects of a decree of nullity, these are things that would be directed to the tribunal via the parish priest if the marriage does in fact end in permanent separation. The determining error of canon 1099 about the sacramental dignity of marriage (or even as the closely related intention contra bonum sacramentalitas of canon 1101, § 2), mentioned above as a possibility, is difficult to establish, and the jurisprudence is complex. We do best to let tribunals investigate and assess the legal impact of the facts in marriage cases.
For purposes of general information though, Rome has permitted couples to remain married, dispensed them from the obligations of marriage without dissolving it, and then to enter religious life or ordained priesthood. Decrees of nullity were not involved.
The most celebrated case involved two Anglicans in the 1800’s in which a couple became Catholic. The man had been an Anglican priest. Subsequently, the husband was ordained as a Catholic priest and the wife entered the convent. Again, I mention this only as a point of general information (or Catholic trivia) and do not intend to apply it to your mother’s situation.
I should also note that a decree of nullity does not dispense a person from the obligations of marriage because it says they were never assumed in the first place. What does not exist cannot be dispensed from. However, canon law recognizes that certain obligations of natural law do arise even from invalid marriages and continue to bind after even definitive separation, namely things like any alimony, child support and education, etc.
So all of this can be complicated. I will say a prayer tonight for your discernment and the welfare of your family. God bless you.
Both the Church and the world need committed Catholics who will embrace the conscrated life and the evangelical counsels.