How can I help him with his insecurity?

A while ago I told my husband (who was a virgin when we married) that one of me ex boyfriends was really big. He got really insecure and started holding back from me. It took a lot of work to get him "back". Anyway he started asking me detailed questions about my past boyfriends that he's never asked before. I asked him about all the women he's fallen in love with in the past and other stuff. His answers made me uncomfortable but it was not too bad. We were honest with each other and he seemed to be taking it all in stride. So I thought were were entering a new stage in our relationship where we are completely honest with each other no matter how sensitive the subject is. That all changed until he asked me a few days ago who out of the four guys I've been with (including him) is the smallest? The honest answer is him so I told him. This was a HUGE (pardon the pun) mistake! I thought he was going to collapse from shock. Since then he not only does not initiate like he used too, he rebuffs all of my advances. Does anyone, especially men, know any way to make help him get over this?