Place the milk, scraped vanilla bean, cinnamon stick and lemon and orange zests in a saucepan and bring to the boil. Simmer for 5 minutes, then strain and set aside.
Whisk the egg yolds with the caster sugar in a bowl for about 5 minutes, until pale and creamy. Add the cornflour and mix well. Slowly add the warm milk mixture to the egg and whisk continuously. Return to the saucepan and cook over low to medium heat, stirring constantly, for 5-10 minutes, or until the mixture is thick and creamy. Do not boil as it will curdle. Pour into six 1 cup ramekins and refrigerate for 6 hours or overnight.
When ready to serve, sprinkle evenly with brown sugar and grill for 3 minutes or until it caramelises.