Sliced Pecorino cheese and baguette toasts, for serving
Slice off the stems of the cherry peppers and discard. Scrape out the ribs and seeds. In a ceramic or glass bowl, cover the cherry peppers with the vinegar. Weight down the peppers with a plate and refrigerate overnight.
Drain the peppers; save the vinegar for another use. Stuff each pepper with 1 anchovy fillet and 2 capers. Pack the peppers into a glass jar. Add the fennel seeds and enough olive oil to cover the peppers completely; refrigerate. Serve with Pecorino cheese and toasts.
The jarred peppers can be refrigerated for up to 1 week.