The Visitation history is complex, and varied from place to place, but they were hardly the only–or even the principal–secular sisters as they were enclosed so quickly (see, for instance, Pagliarini’s dissertation on them). Consider, in addition to Rapley, Marguerite Vacher’s history of the early Sisters of St. Joseph (translated into English by Patricia Byrne), and–very importantly–the essential 2-volume history of the Ursulines by Teresa Ledochowska (also translated into English). Elizabeth Makowski’s work, on Periculoso and other matters, is also foundational. There is an Ursuline canonist who wrote an excellent dissertation on this a few decades ago, but her name slips my mind at the moment.
The point is that the history has been well researched (and continues to be), and it is important not to generalize on the basis of a single conversation.