For the Cake: Cream together sugar and butter. Add eggs; beat on medium speed of mixer for 2 minutes. Mix dry ingredients together lightly with a fork. Add dry ingredients alternately with milk. Mix well. Stir in vanilla and nuts. Pour into a greased and floured 13x9-inch pan (or 2 8-inch round cake pans). Bake at 325 F for 45--50 minutes. Cool.
For the Frosting: Make frosting by melting butter in a medium saucepan. Add brown sugar; boil 2 minutes. Add milk; bring to a boil. Remove from heat; cool to lukewarm. Beat in sugar and vanilla. (You may add 1/2 cup chopped hickory nuts if desired.) Frost the cake and decorate with reserved hickory nuts.