Arrange peach pieces in a single layer in a freezer-proof container. Cover; freeze until very cold but not frozen solid, about 1-2 hours.
In a blender, combine peach pieces, raspberries, yogurt, nectar and sugar (if using). Puree until very smooth, about 2 minutes. Pour into four tall glasses, dividing evenly. If desired, garnish with mint.