Place first five ingredients in a small saucepot. Heat until warmed through, swirl the pan to coat the dried fruit, and set aside to soak.
Combine the brown sugar, cinnamon and nutmeg in a medium bowl. Using your fingers, crumble the butter into the mixture until it forms a slightly moist crumb topping.
Fill each pear with some of the mixture.
Place the pears in a 9-by-13-inch baking dish; pour the brandy mixture evenly over the pears. Cover with foil and bake for 30 minutes at 375°F
Uncover, gently flip the pears upside down—it's fine if some of the mixture falls out of the center. Bake for an additional 30 minutes. Transfer the pears to serving bowls; spoon the juices and fruit over.
For added flair, serve each pear with a scoop of vanilla frozen yogurt or sprinkle with lime zest!