I suspect you’re right that advanced math, at least, was mostly taught in Latin. But that wasn’t a Church effort to keep people ignorant. We have to remember that for a very long time in Europe, there were dozens, if not hundreds of dialects. Even in Chaucer’s era, you couldn’t go far from London without encountering dialects or languages that were unintelligible to you. That’s one of the reasons Chaucer is (by scholars at least) credited with “creating” the English language as we know it today. He didn’t create it, of course, but his literature was very popular and spread the London dialect.
So, if one was from, say, Wessex and wanted to study in London, it was Latin or nothing, and a lot of people knew Latin. It was the “trade language” of the time, and a lot more people knew it then than do now, by far.
But one also needs to realize math and geometry were well understood by a lot of skilled people. Stone masons could not function without a very serious command of both, for example. One could not assemble stained glass windows that could last a thousand years without a pretty serious understanding of both chemistry and physics, both of which require advanced math. One wonders whether our modern glass skyscrapers will last 100 years without falling apart.
The medieval guilds didn’t exactly teach “math” in the same way, but they taught very advanced arithmetic to the apprentices. You couldn’t figure out how much in Dutch gulden to pay for amber in Lithuania to sell in Venice for Italian florins to buy salt herring in Norway to resell in Calais for English pounds without having a pretty good command of arithmetic, weights and measures and currency conversion. You couldn’t build a trading ship without understanding math, and wouldn’t know how much you could load on it without knowing at least your arithmetic. And the millers who could figure how much drop of how much water would turn a wheel and stones of “X” weight. They had to know their math to do that.
And while the guilds were exclusive, they were not class-based. Talent, not nobility, and certainly not Church sanction, was what counted. And even those outside the guilds needed to know some pretty significant arithmetic. Undoubtedly there were many peasants who counted on their fingers. But trade in medieval Europe was very widespread. There were English merchants in Kiev and Moscow, Italian merchants in Poland and Lithuania, French merchants in London and Flanders. Flemish merchants in Alexandria and Aleppo. People got around much more than we often think.
That’s one of the reasons why knowledge of Latin was so widespread, and so was arithmetic. More advanced math was pretty much the preserve of scholars and highly skilled artisans, of course. But all of that predated Martin Luther.