Preheat oven to 375 degrees. Line a 9"x13" pan with parchment paper.
In a large bowl, beat together cake mix, butter, eggs and pineapple juice. Mix well.Set aside.
In a medium sauce pan over medium high heat melt butter and whisk in brown sugar and evaporated milk. Bring to a boil stirring consistently. Adjust heat to simmer and continue to stir for about 5-7 minuets being careful not to burn mixture. Remove from heat and whisk in vanilla.
Carefully pour hot caramel mixture into baking pan over parchment paper. Distribute pineapple rings over caramel sauce. Add cherries.
Pour cake batter over caramel and fruit.
Bake for about 45 minuets or until inserted toothpick comes out clean.
Once cake is finished baking, allow to sit for 3-5 minuets then remove cake from pan by flipping cake onto wired rack and carefully and slowly, pull back paper from caramel. to continue to cool. Failure to do this will allow the cake to keep baking, which will result to a dry cake.
Place cake onto desired dish and serve. Refrigerate leftovers.