Melt butter in a pan and add brown sugar, stirring frequently.
Spray a 9x13 inch baking dish (important!) with cooking spray. Pour mixture on the bottom so it's about 1/8-inch thick. Lay the bread slices on top of it.
Mix together eggs, milk, cinnamon, and nutmeg. Pour egg mix over the bread. Cover tightly with plastic wrap and refrigerate overnight.
In the morning, preheat the oven to 350°F and uncover the baking dish. Bake for 30 minutes. You can serve with topping of your choice. It's already very sweet, so we eat it as is, but it's also good with fruit.
If you do 'breakfast for dinner', this can be made during the day anytime up to about 2 hours before the cooking time. This will give it time to soak up the egg mixture.