Preheat the oven to 425 degrees F. Line a rimmed baking sheet with foil.
In a small bowl, whisk together the mayonnaise, lemon zest, paprika, celery seeds, salt, pepper, and cayenne. Season the chicken with additional salt and pepper, to taste. Arrange the chicken on the prepared baking sheet. Slather the mayonnaise mixture over the chicken. Bake until the juices run clear when the chicken is pricked with a fork, about 20 to 25 minutes. Transfer the chicken to a serving platter and serve hot.