Combine frosting ingredients and beat on medium speed until smooth and fluffy. (If necessary, add more powdered sugar to thicken the mixture, or more evaporated milk to thin the mixture.)
Place about 3/4 cup of frosting into two smalls bowls (3/4 cup per bowl.)
Mix a few drops of red and a few drops of green food coloring into their respective, separate containers, until you get the desired colors you want.
Use the remaining 1 1/2 cups of white frosting to frost the tops of the cupcakes.
Place the red and green food coloring into separate piping bags, or into two ziplock baggies with a small whole cut into the corners.
Decorate cupcakes with an ornament design.
Place the Reese's miniature cups inside the cupcake liners to complete the top of your ornament! You could also twist a paper clip into a little fish hook and stick that into the Reese's to make it even cuter!