Preheat the oven to 350°. Grease and flour a 9-by-13-inch baking dish. Using an electric mixer, beat the egg yolks with 1 cup sugar on low speed until combined, then increase to high and beat until fluffy, 4 to 5 minutes. Add 1/2 cup evaporated milk and the vanilla; beat until combined. Fold in the flour and baking powder.
Clean the beaters. In a medium bowl, beat the egg whites until foamy, about 30 seconds. Add the cream of tartar and beat until soft peaks form, about 1 minute. Beat in the remaining 1/4 cup sugar until stiff peaks form, 2 to 3 minutes.
Working in batches, fold the egg white mixture into the yolk mixture until just combined. Pour into the prepared baking dish. Bake until a toothpick inserted into the center comes out clean, about 30 minutes. Let cool for 45 minutes.
In a medium bowl, whisk together the remaining 1 1/2 cups evaporated milk, the sweetened condensed milk and 1 cup cream. Ladle over the cooled cake. Cover and chill until the liquid is absorbed, about 4 hours.
Beat the remaining 3 cups cream until soft peaks form, 3 to 4 minutes. Serve with the cake and fresh fruit.