This makes a delicious breakfast/brunch treat. Be sure to use canned pumpkin puree, not fresh, because even if pureed, fresh pumpkin can sometimes still have fibrous pulp. Note that cooking time also includes sitting and cooling time.
Butter a 10-inch springform pan. Wrap outside with foil. Warm milk, cream and vanilla in pan on medium. Remove from heat. Let sit 20 minutes. Pour into bowl. Heat oven to 325 degrees F. Whisk eggs and yolks. Whisk in sugar. Slowly stir into cooled milk mixture. Stir in pumpkin, ginger and salt.
Dunk bread into custard. Arrange slices in pan, overlapping layers. Slowly pour remainder of mixture into pan. Let sit 30 minutes.
Place pan inside roasting pan. Fill pan with hot tap water until it reaches about halfway up sides. Bake 90 minutes. During last 10-15 minutes top with 1/4 cup brown sugar, return to oven and raise temperature to 400 degrees F. After 10-15 minutes remove from oven and let cool for 20 minutes. Remove from pan. Garnish with whipped cream.