How long ago did you pull away from worldly things? Are you still into worldly things? Worldly influences? (an over abundance of worldly reminders from Friends, television watching, magazines etc.) I'm assuming the movement towards religious life is recent, like in the last 5 years?
You don't give your age here. I am assuming from what other posters have said that you are in your early 20's. If so, what you have said here sounds typical for your age.
Breaking away from the world for the first time is scary. I am assuming that you have given much prayer, thought and consideration to the truths of the faith, yes? Keep in mind, these trails are just that. Trials. No, you aren't gay, but many in society would/might probably deem you as being such given that most men (young ones especially) are deemed 'strange' by each other if they don't go girl chasing. Sorry to say, but you will have to learn how to deal with people who will assume wrong things about you.
Keep in mind too, that even lay catholic men have to pull away from this worldly way of thinking as well. Its a part of growing into a man... something you at this age are grappling with, yes? This is also a part of self control- you have to ditch what the world might think of you. I am assuming you are fearing that a bit now which is why you posted. There are probably many good men out there who have chosen to 'wait' until marriage that also get this sort of pressure.. so don't feel too alone.
Find some like minded friends who will support you and understand the beauty of celibacy.
You will be convincing when you will show forth the light of God. Pure. It shines through the eyes, and you will be able to then look any accuser straight in the eye. So work on that- being pure, being single minded for God. This may be a part of the reason why you feel imperfect/tarnished as you say? You may be pushing too fast in spiritual growth, trying to leap ahead.
If you don't find any consolation or attraction to marriage I don't think that is abnormal for your age either. I think you are trying to rush things or see into the future. I didn't find marriage itself (REAL marriage, not the fairy tale stuff) an attractive proposal until I was about 29- even though as a girl I always wanted to get married. (I STILL think its a daunting endeavor, though) I think part of your problem could be the culture right now and what it deems as normal. (Pssst... don't listen to what you see on your tv, its not reality.) You are trying to pull away and its hard to leave that fold. In due time with God's grace, you will be able.
Relax. Don't go so fast.. all in God's time!