Build a multi-level fire in your grill: Leaving one-quarter of the bottom free of coals, bank the coals in the remaining three-quarters of the grill so that they are three times as high on one side as on the other.
When the coals are all ignited and the temperature has died down to medium (you can hold your hand about 5 inches above the grill grid, over the area where the coals are deepest, for 4 to 5 seconds) you’re ready to cook.
Rub the mushrooms with the olive oil and then with the black pepper, then sprinkle generously with salt.
Put them on the grill and cook until they are tender and slightly seared, 6 to 8 minutes per side.
(To check for doneness, nick, peek, and cheat: Cut into one of the mushrooms and peek to be sure it looks moist all the way through, rather than dry at the center.).
While the mushrooms are cooking, whisk the dressing ingredients together in a medium bowl.
When the mushrooms are done, take them off the grill, cut them into thick slices, add to the bowl with the dressing, toss gently to coat, and serve warm.