In a large bowl, whisk together the pudding mix with the milk until smooth and well combined. Place in the fridge and let chill until firm.
Preheat oven to 375 degrees F.
Working with one pie crust at a time, place on a lightly floured work surface and roll out. Using a lightly floured rolling pin, roll each pie crust to make them slightly larger.
Trim the edges to create somewhat of a rectangle that’s the 9x10-inches in size. Then cut into six even rectangles. Place on a baking sheet in a single layer.
Repeat with another pie crust and place the rectangles on a separate baking sheet.
Place a small spoonful of pudding on the center of each, and then place one or two thin banana slices on top. Sprinkle with crushed up vanilla wafers. Brush the edges with a bit of beaten egg.
Repeat the rolling of the remaining pie crusts to create the tops of each tart. Place each piece on top of the filling and crimp the edges with a fork to seal. Poke one or two holes with a sharp knife on top of each tart to release steam while baking.
Brush each with egg wash and bake until golden brown, about 18 to 20 minutes. Remove from the oven and let cool completely.
In large bowl, whisk together the powdered sugar, vanilla extract, butter, milk and salt until smooth and spreadable in consistency. Divide glaze into four separate small bowls. Place 2-3 drops of each food coloring into the bowls with icing and whisk to combine. Spread the glazes on top of each cooled tart and top with sprinkles.