Begin by preparing your fruit. Peel and pit the peaches. Cut into quarters and then cut each quarter in half. Next, peel, core, and halve the pears. Cut each pear half lengthwise into 1/4 inch slices. Transfer fruit to a large bowl; cover with plastic wrap and refrigerate.
Begin the marinade -- In a large, heavy saucepot combine together the wine, orange and lemon zest, sugar, cloves, nutmeg, cinnamon stick and ginger. Scrape in the seeds from the vanilla bean; add the bean. Give a good stir; over medium heat bring to a boil while stirring to dissolve the sugar. Once the marinade is boiling; reduce the heat to low and simmer for 5 minutes; add raisins during last two minutes.
Pour the warm marinade over the peaches and pears. Give a gentle toss to generoulsy coat. Allow to marinate at room temperature for 1 hour; stir occassionally.
Serve with plenty of the sweet juices, and top with a dollop of whipped cream or yogurt, and a sprinkling of walnuts.