In a heatproof, nonreactive bowl, combine the mushrooms and horseradish. In a small nonreactive saucepan, combine the wine, vinegar, water, celery seeds, thyme and juniper berries and bring to a boil. Pour the hot brine over the mushrooms and let stand for about 2 hours. Drain the mushrooms, pressing out the brine and pick out the juniper berries and thyme sprigs.
The drained pickled mushrooms can be refrigerated overnight.