Peel the potatoes, quarter lengthwise, and slice into 1/2-in thick pieces.
Steam cauliflower 15 minutes. Transfer to a food processor, add the buttermilk and 1/4 cup of milk, and puree until very smooth, about 2 minutes.
Meanwhile, place potatoes in a saucepan, add cold salted water to cover potatoes about 2 inches, bring to a boil, reduce heat, and simmer until tender, about 15 minutes. Drain, return to the potatoes to the saucepan, and heat over medium heat for 1-2 minutes to dry. Mash with a potato masher.
Add the cauliflower puree, 1/4 cup milk, and a little bit of salt and pepper to the saucepan with the mashed potatoes. Stir to blend, and heat over medium-low heat, adding more milk as needed for a creamy consistency.