Preheat oven to 325°; butter an 8-cup pudding dish.
Put the raisins and Grand Marnier into a small saucepan; bring to a boil and simmer for a minute or so, then turn off the heat and leave the fruit to plump up in the liquor.
Make marmalade sandwiches with the brioche slices, using about 1 teaspoon marmalade in each.
Because the brioche is so buttery, don't spread any butter in the sandwiches or the pudding will be greasy.
Cut each sandwich into a triangle and put these triangles, one point side up, the next one pointy bit down and so on, in the buttered dish (*this is confusing to me, so I would say cut the sandwiches to fit the dish).
Sprinkle the raisins along with any remaining liquor in the pan.
In a bowl, whisk together the sugar, whole eggs, egg yolks, cream, and milk.
Pour over the brioche sandwiches and leave to stand for 15 minutes.
Sprinkle brown sugar over the top; bake for 45 minutes, by which time the custard should be at the point of just setting.