Melt butter in a medium saucepan over medium heat. Add leeks and cook, stirring ocasionally, for 5 minutes. Add potato, 1/8 teaspoons pepper and the nutmeg, and cook, stirring, 1 minute. Stir in cauliflower, 1/4 teaspoons salt, and 3 cups stock. Simmer, partially covered until cauliflower has softened, 12-25 minutes. Using a slotted spoon, remove 2 florets, and transfer to a cutting board; thinly slice lengthwise. Set aside for garnish.
Working in batches, purée mixture in blender, filling no more than halfway each time. Return to pan. Stir in buttermilk and remaining 1/2 cup stock. Sprinkle with 1/2 teaspoons salt and season with pepper. Serve hot or cold, garnished with cauliflower slices.