Preheat oven to 350 degrees. In a 9-by-13-inch baking dish, combine the sugar and water. Add the butter and place in the oven to melt the butter.
Meanwhile, place the flour in a mixing bowl. Use two knives or a pastry blender to cut in the shortening until the mixture resembles coarse crumbs. With a fork, stir in just enough milk or buttermilk until the dough leaves the side of the bowl.
Turn dough onto lightly floured surface and knead 2 or 3 times, then use a rolling pin to roll out to 1/4-inch thickness. Use a 3-inch cutter to cut 24 rounds. Place a piece of banana in the center of each biscuit, then pull the dough around the banana and pinch the ends to seal on top.
When the butter is melted, remove the pan from the oven and stir until the sugar completely dissolves.
Place the banana dumplings, sealed side down, in the syrup. Sprinkle with a little sugar and return to the oven.