Place the baking rack in the center of the oven and heat to 350 degrees F.
Butter a shallow 1 1/2-quart baking dish (11 by 7 by 1 1/2 inches), or spray generously with cooking spray.
Cut the apples into 1/2-inch-thick slices. In a large bowl, mix together apples, lemon juice and cinnamon. Set aside.
Melt the 1/4 cup of butter and 1/4 cup brown sugar in a 10– to 12–inch heavy skillet over medium-high heat. Stir in the apple mixture using a wooden or heat-resistant spatula and cook over same medium-high heat, stirring frequently until the apples are softened and begin to turn golden brown, about 10 to 15 minutes. Stir in the walnuts or dried fruit and transfer the mixture to a large bowl. Set aside to cool slightly, about 10 minutes.
While the caramelized apple mixture is cooling, toast the bread on a baking sheet in the oven until crisp and the edges are just starting to color, about 6 to 8 minutes. Set aside on baking rack.
Once the caramelized apple mixture is slightly cooled, gently and evenly spread half into the buttered baking dish. Add the toasted bread to the remaining apples in the bowl and stir to combine; then spoon the mixture into the baking dish.
In a medium bowl combine the eggs, egg yolks, milk, remaining 1/4 cup of brown sugar, vanilla and nutmeg and whisk until the sugar is dissolved, about 2 minutes. Then pour evenly over the apple and bread mixture and let it sit for 10 minutes until the bread has absorbed the liquid.
Bake until puffed and browned on the top and the custard is set, about 45 to 55 minutes. Serve hot, warm or chilled.