Preheat the oven to 300°. In a medium saucepan, cover the potatoes with hot tap water and bring to a boil. Cook over moderately high heat until tender, about 15 minutes. Drain, then return to the pot. Toss the potatoes with 1 tablespoon of the olive oil and season with salt and pepper.
Meanwhile, set the fish in a lightly oiled baking dish and season with salt and pepper. Bake for about 15 minutes, until the flesh just begins to flake. Put the walnuts on a pie plate and toast for 8 minutes, until fragrant. Coarsely chop the nuts.
In a bowl, mix the walnuts with the olives, capers, lemon zest, lemon juice and the remaining 2 tablespoons of olive oil; season with salt and pepper. Transfer the fish and potatoes to plates, spoon the walnut-olive relish on top and serve.