Put a saucepan of water on to boil. Pare the zest of the orange and lemon, taking as little of the white pith as possible, then cut into long fine shreds. Drop into the boiling water and leave to simmer for 5 minutes to soften them and remove any bitterness. Drain.
Put the jelly, mustard and port into a saucepan and stir, then whisk, over a low heat until smoothly amalgamated. Bring up to the boil; add the blanched orange and lemon zest, the ginger and a little salt and freshly ground black pepper. Simmer gently for 5 minutes. Pour into a serving jug or bowl, and then leave to cool.