Rub oil over potatoes. Place directly on oven rack; bake until very tender, about 45 minutes. Transfer to rack; cool 10 minutes.
Using oven mitts, grasp 1 potato in hand; cut potatoes in half. Using spoon, scoop out potato, leaving 1/4-inch-thick shell. Repeat with remaining potatoes.
Transfer potato flesh to large bowl.
Place potato shells back on baking sheet and return hot oven while preparing filling so they will give a nice crispy shell.
Mash potatoes until smooth. Mix in cheese, then Buttermilk, butter, and chives, Fresh rosemary and Beacon Bits if desired, season with salt and pepper to taste.
Spoon about 3/4 of potato mixture into shells, dividing evenly. Transfer remaining potato filling to pastry bag fitted with large decorative tip. Pipe filling atop potatoes; place potatoes on baking sheet.
Position rack in center of oven and preheat to 375°F.
Bake potatoes until filling is heated through and tops brown, about 20 minutes.