Preheat oven to 350 degrees F. In a large bowl, beat 2 cups butter with an electric mixer on medium to high speed for 30 seconds. Add granulated sugar, baking powder, baking soda, salt, cinnamon, and nutmeg. Beat until combined, scraping bowl occasionally. Add eggs and 2 teaspoons vanilla; beat until combined. Beat in pumpkin. Beat in as much of the flour as you can with the mixer. Using a wooden spoon, stir in any remaining flour.
Drop dough by heaping teaspoons 2 inches apart on an ungreased cookie sheet. Bake in the preheated oven for 10 to 12 minutes or until tops are set. Transfer cookies to a wire rack; let cool.
In a small saucepan, heat 1/2 cup butter and brown sugar until melted and smooth. Transfer to a medium bowl. Stir in milk and 1 teaspoon vanilla. Beat in powdered sugar until smooth. Spread frosting on cookies. If desired, sprinkle with additional cinnamon.