Place milk in a medium saucepan and heat to a simmer. Pour half over the tea bags in a medium bowl and let sit for 30 minutes. Strain, pressing all the liquid from the tea bags.
Stir powdered milk into milk mixture and keep warm over low heat.
Place egg yolks and sugar in a medium bowl. Using a hand mixer or whisk, beat until thick and pale yellow, the consistency of mayonnaise. While mixing, slowly add the hot milk and whisk until blended.
Stir the egg mixture back into the saucepan and add the tea-infused milk; increase the heat to medium. Stir the mixture constantly with a wooden spoon until thickened like a custard sauce and registers 180º on an instant-read thermometer.
Strain the custard through a fine mesh strainer into a medium bowl. Stir in cream, cover and refrigerate at least 6 hours before continuing.
Pour chilled mixture into ice cream freezer bowl land let mix until thickened, about 20-25 minutes, per manufacturer's instructions.