How do I help my man's self-esteem in regard to his penis?

I'll try to be brief..... I am deeply in love. While we were getting to know each other, I thought I would be sexy and tell him I was into light domination. Once we dated for while longer, out of the blue he asked me about that. I, very stupidly, told him I liked being spanked and such. He freaked out because he is absolutely not into that. We were working through that when he, for some unknown reason, asked about my ex's penis size. I, very stupidly, said he was big. Now my boyfriend, the man I want to marry, is so anxious about having sex with me that he won't let me touch him or go down on him because he feels inadequate. I've tried and tried to reassure him but he doesn't believe anything I say. I am not sure why but he has serious self-esteem problems and the whole penis size thing is killing him. He says he's trying to work this out but it rips me apart to see him in pain like this. What can I do, what can I say, to make him understand that he has absolutely nothing to be concerned about and his penis is perfect? I love this man with my entire soul and I'm afraid of losing him because of my stupidity.....