Heat oil in a large saucepan or Dutch oven over medium high heat until very hot; add goat pieces and sear until golden brown on all sides, about 20 to 30 minutes; transfer meat to a plate.
Pour out most of the oil in saucepan, then add garlic, onion, carrot, and celery; cook, while stirring, until softened, about 3 to 5 minutes.
Add tomato paste and cook an additional minute.
Deglaze the pot with the red wine and vinegar (use a wooden spoon to loosen any brown bits that may have stuck to the bottom).
Continue cooking until most of the liquid has evaporated, about 20 minutes.
Mix in oregano, mustard, water, thyme, and rosemary.
Return goat meat to the pot, season with about 1 1/2 tablespoons salt and fresh pepper.
Bring to a boil, reduce heat to medium low, partially cover, and cook for 2 1/2 hours; skim off any scum that accumulates on the top.
Transfer goat meat and vegetables to a serving platter, discarding any cooked herbs.
Increase heat under pot and reduce pan juices, stir in roasted garlic puree, chopped dill, and mint.
Drizzle the pan juices over the goat and vegetables, then drizzle with extra virgin olive oil.
NOTE: To roast garlic, lop off the top of a head so that its cloves are exposed a bit. Drizzle with olive oil and wrap tightly in aluminum foil, then roast for about 40 minutes in a 400-degree oven. Squeeze the cloves into a small bowl and mash to form a puree.