Note: Any combination of olives can be used for this recipe. We used equal parts of picholine, nicoise, large green and large black olives, as well as kalamata. For the best flavor, we recommend buying the olives with the pits still inside.
To start, peel and lightly crush the garlic. Next, using a peeler, remove four 1" x 3" -inch pieces of zest from the lemon and the orange. Trim off any white pith. Break the rosemary down into smaller sprigs by pinching it off of the stem. Gather the remaining ingredients.
In a large pan, heat the oil and garlic over medium-low heat. Cook for about 3 minutes, just until the garlic begins to turn golden. Do not let it burn. Add the strips of zest, rosemary sprigs, bay leaves, red pepper flakes and allspice. Stir and let sizzle for about 2 minutes. Add the olives and toss to coat. Transfer to a bowl and let cool. Cover and refrigerate, stirring from time to time. The olives will keep for up to one week.
Before serving, place the mixture into a pan and reheat over low until warmed through, about 2 to 3 minutes. Place into a serving bowl and serve warm.