Preheat the oven to 400°. In a bowl, mix half of the lime zest, lime juice, garlic and chile. Stir in the olive oil. Make 3 shallow 2-inch slashes into the skin of each fish fillet. Spread the garlic-chile paste over the fish. Transfer the fish to a baking dish, skin side up; season with salt and pepper. Bake for 8 minutes, until the fish flakes easily.