Warm half of the cream with the scraped vanilla bean in a small saucepan on medium heat for about 2 minutes. Keep aside.
Meanwhile, whisk together the egg yolks and the sugar in a bowl until the color becomes light and pale, about 2 minutes. Remove the vanilla pod and pour the warm cream over the egg mixture while constantly stirring. Reheat the mixture on low heat stirring constantly until it thickens and coats the back of a spoon, about 3 minutes. Remove from heat and let cool to room temperature.
Meanwhile, in a medium saucepan on medium heat, reduce by half the balsamic vinegar, about 10 minutes. Let it cool to room temperature.
Once your custard is cool, whip the remaining cream and fold it in with the custard mixture using a wooden spoon. Add the olive oil and fold in everything together. Drizzle 3/4 of the reduced balsamic vinegar in the cream mixture and use a fork to make swirl patterns. Pour the mixture in a stainless steel container and drizzle the leftover of the balsamic on top.
Put in the freezer until firm, a minimum of 4 hours.
Scoop into nice bowls and serve with fresh berries.