Line a loaf pan with plastic wrap. Be sure to leave plenty of plastic around the edges so they can be folded over to cover the top. Put the almonds in the bottom of the prepared pan in an even layer. Make it pretty because this will become the top when serving. Put it aside.
Whip the cream with an electric beater until it forms medium peaks. Refrigerate until ready to use.
Bring a saucepan with about 1 inch of water to a boil over low heat. In a metal bowl, combine the 1 1/4 cups of sugar, egg yolks, lemon juice and zest and the pinch of salt. Whisk to combine. Put the metal bowl on the saucepan and reduce the heat under the saucepan so the water is just at a simmer. Whisk the egg mixture until it gets very light and fluffy, this will take about 4 to 5 minutes (it took me almost 20 minutes to do this). For an insurance policy, you can take the temperature of this mixture. It should be about 165 degrees F on an instant-read thermometer. Remove from heat and beat with an electric mixer until the mixture has doubled in volume and the bottom of the bowl is cool when touched (mine was still warm when the volume doubled).
Fold in the reserved whipped cream in increments (I did it in 4 increments). Do this gently but quickly. When it is a homogeneous mixture, pour it into the prepared loaf pan. Fold the plastic over the top and transfer to the freezer. Freeze at least 4 to 5 hours, but this can be done overnight or a couple days in advance.
Combine the oranges, mangoes, mint and remaining 1 tablespoon of sugar in a small bowl. Let the mixture macerate for about 1 hour before serving (I did this one day in advance and it's fine).
Remove the loaf pan from freezer and open the plastic. Turn the semifreddo out onto a serving platter and remove plastic. With a hot knife, slice the semifreddo into 1-inch slices and garnish with the orange/mango salad.