Yeah, you read that correctly! Here's how I did it.
You need regular cupcakes.
Some Ice cream.
A cupcake corer/plunger.
Core the cupcake and the ice cream.
Then you insert the ice cream into the cupcake.
Then you frost it, and enjoy.
Here is a picture of the inside. These were so good I highly recommend them. Imagine the possibilities.