Preheat oven to 375°; in a small bowl, combine the rosemary, ¼ teaspoon salt, and 1/8 teaspoon pepper.
Remove and discard the giblets from the hens; rinse hens with cold water; pat dry.
Remove skin, trim excess fat.
Work with 1 hen at a time: place 1 lemon piece in the cavity of the hen, tie ends of legs together with twine.
Lift wing tips up and over back; tuck under hen (repeat with remaining hen and lemon piece).
Rub hens with rosemary mixture.
Place hens, breast side up, on a broiler pan coated with cooking spray.
Toss potato with oil; sprinkle with remaining salt and pepper; arrange potatoes around hens.
Insert a meat thermometer into meaty part of a thigh, making sure not to touch bone.
Remove twine from lets; bake at 375° for 1 hour or until thermometer reads 180°.