Puree the banana and ¼ cup of the milk in a food processor until smooth.
Combine the banana puree, the remaining 1¾ cups milk, ½ cup brown sugar, the cinnamon, cardamom, nutmeg, and salt in a medium-size heavy saucepan. Scald the milk mixture (heat nearly to a boil) over medium-high heat, stirring often, for 5 minutes.
While the milk is scalding, whisk together the egg yolks in a medium mixing bowl. Add the remaining ½ cup brown sugar and whisk until the eggs are light and fluffy. Whisking constantly, add a small amount of the hot milk to the egg mixture (to temper the eggs). Gradually whisk in the remaining hot milk.
Return the custard mixture to the pan and cook over medium-low heat, stirring often, until the custard is thick enough to coat the back of a spoon, about 5 minutes. Remove pan from heat and place in a bowl of ice water to quickly cool the custard. Let the custard cool, stirring often, for 5 minutes.
Meanwhile, combine cream and vanilla in a medium bowl. Stir in the custard mixture. Cover with plastic wrap and press wrap directly onto the surface of the custard. Refrigerate until completely chilled, at least 4 hours or overnight. The custard may be stored in the refrigerator for up to 3 days.
Freeze the chilled custard mixture in an ice cream maker according to the manufacturer’s instructions. Transfer ice cream to an airtight container and place in the freezer for 2 to 4 hours before serving.