Bring cream, vanilla bean and 1/4 cup sugar to a low boil. Remove vanilla bean.
In a separate bowl, whisk together the egg yolks and remaining sugar. Whisk a little of the hot cream mixture into the egg mixture. Whisking constantly, pour the egg mixture into the hot cream mixture.
Place the chocolate chips in a large bowl.
Pour the custard over the chocolate chips; stir until chocolate has melted.
Layer the cake pieces, chocolate custard, toffee pieces in a greased 8-inch square cake pan. Repeat the layers as needed.
Bake at 350 degrees F until the center is firm, about 20 minutes.