We believe also in the holy Church, that is, the Catholic Church. For heretics violate the faith itself by a false opinion about God; schismatics, however, withdraw from fraternal love by hostile separations, although they believe the same things we do. Consequently, neither heretics nor schismatics belong to the Catholic Church; not heretics, because the Church loves God; and not schismatics, because the Church loves neighbor" (Faith and the Creed10:21 [A.D. 393]).
“The apostle Paul said, ‘As for a man that is a heretic, after admonishing him once or twice, have nothing more to do with him’ [Titus 3:10]. But those who maintain their own opinion, however false and perverted, without obstinate ill will, especially those who have not originated the error of bold presumption, but have received it from parents who had been led astray and had lapsed . . . those who seek the truth with careful industry and are ready to be corrected when they have found it, are not to be rated among heretics” (Letters 43:1 [A.D. 412]).
We must hold to the Christian religion and to communication in her Church, which is catholic and which is called catholic not only by her own members but even by all her enemies. For when heretics or the adherents of schisms talk about her, not among themselves but with strangers, willy-nilly they call her nothing else but Catholic. For they will not be understood unless they distinguish her by this name which the whole world employs in her regard" (The True Religion 7:12 [A.D. 390]).
“If you should find someone who does not yet believe in the gospel, what would you [Mani] answer him when he says, ‘I do not believe’? Indeed, I would not believe in the gospel myself if the authority of the Catholic Church did not move me to do so” (ibid., 5:6).
“In the Catholic Church . . . a few spiritual men attain [wisdom] in this life, in such a way that . . . they know it without any doubting, while the rest of the multitude finds its greatest safety not in lively understanding but in the simplicity of believing. . . .There are many other things which most properly can keep me in her bosom. The unanimity of peoples and nations keeps me here. Her authority, inaugurated in miracles, nourished by hope, augmented by love, and confirmed by her age, keeps me here. The succession of priests, from the very see of the apostle Peter, to whom the Lord, after his resurrection, gave the charge of feeding his sheep [John 21:15–17], up to the present episcopate, keeps me here. And last, the very name Catholic, which, not without reason, belongs to this Church alone, in the face of so many heretics, so much so that, although all heretics want to be called ‘Catholic,’ when a stranger inquires where the Catholic Church meets, none of the heretics would dare to point out his own basilica or house” (Against the Letter of Mani Called “The Foundation” 4:5 [A.D. 397]).
“[On this matter of the Pelagians], two councils have already been sent to the Apostolic See, and from there rescripts too have come. The matter is at an end; would that the error too might be at an end!” (Sermons 131:10 [A.D. 411]).
“If the very order of episcopal succession is to be considered, how much more surely, truly, and safely do we number them from Peter himself, to whom, as to one representing the whole Church, the Lord said, ‘Upon this rock I will build my Church, and the gates of hell shall not conquer it.’ Peter was succeeded by Linus, Linus by Clement. … In this order of succession a Donatist bishop is not to be found” (Letters 53:1:2 [A.D. 412]).
You can further verify Augustine’s claims on the Church and on the Papacy.
The Bible: vatican.va/archive/bible/index.htm
List of Popes: newadvent.org/cathen/12272b.htm
Apostolic See: en.wikipedia.org/wiki/Holy_See