Cut 3 of the lemons into one-eighth inch thick slices. Remove seeds and set aside. You will have about 30 lemon slices. Grate 1 teaspoon lemon peel from the remaining lemon. Set aside the grated peel; save the lemon for another use.
Heat 4 tablespoons of the butter in a 10-inch cast iron skillet or an ovenproof 10-inch saute pan until melted. Brush the sides of skillet with a little of the melted butter. Add the brown sugar, stir until it is moistened with the butter and spread it into an even layer. Arrange the lemon slices, slightly overlapping, to cover the bottom of the skillet. Set aside.
Heat the oven to 350 degrees. Combine the flour, baking powder and salt in a bowl and set aside.
Cut the remaining 6 tablespoons butter into a mixing bowl. Scrape the seeds from the vanilla bean with the point of a knife onto the butter. Using an electric mixer, beat the butter, scraping down the sides of the bowl, until creamy. Add the sugar and grated lemon peel and beat until light and fluffy. Beat in the eggs one at a time.
Add half the flour mixture and beat until blended. Add milk and beat until combined, then add the remaining flour mixture and beat until blended.
Spread the batter over the lemons in the skillet to cover evenly. Bake 30 to 35 minutes, or until the cake is golden and the center tests done. Let the cake stand 5 minutes, then invert the skillet onto a platter. To serve, slice into wedges with a sharp knife.