Preheat oven to 350°F.
In a large mixing bowl, cream together butter and sugar. Mix in flour, baking powder and salt at a low speed, followed by the vanilla and milk. The dough should come together into a soft ball but it will be crumbly.
Take a tablespoon full of dough and flatten it into a disc about 1/4-inch thick. I found it was easiest doing this in the palm of my hand. Place on a parchment-lined baking sheet and repeat with remaining dough. Cookies will not spread too much, so you can squeeze them in more than you would for chocolate chip cookies.
Bake cookies for 9 minutes, until bottoms and the edges are lightly browned and cookies are set.
Immediately after removing cookies from the oven, use your thumb or a small spoon to make a depression in the center of each cookie. (I am still debating as to whether this is a necessary step.).
Cool for a few minutes on the baking sheet then transfer to a wire rack to cool completely.
In a small bowl, whisk together peanut butter, confectioners’ sugar, salt and vanilla. When the mixture has come together, heat it in the microwave for 30 to 45 seconds at a time, then stir. Heat until it is very, very soft. Let cool slightly and spoon a generous dome of the filling into each cookie’s “thumbprint”. Alternately you could use a piping bag but I did not find this necessary.
Chill cookies with filling for 20-30 minutes, or until the peanut butter is firm.
Melt the chocolate and butter in a small double boiler until melted.
Dip chilled cookies into chocolate with a fork, allowing theexcess to drip off, and place on a sheet of parchment paper to let the cookies set.