As immigration to Europe has increased, so has support for extremist parties. While many studies have examined the effect of immigration on election outcomes, few have probed the effect of asylum seekers - those fleeing strife and persecution - on voting, nor has there been much research on the mediating role of local economic conditions. Drawing on county level panel data from Germany, our study fills both gaps. We find that economic circumstances, as measured by the unemployment rate and the level of disposable income, condition voters' responses to the presence of asylum seekers, but the effects for parties on the far right and left diverge markedly. Under economic prosperity, immigration increases support on both sides of the political spectrum. As economic conditions worsen, however, the effect of asylum seekers on the vote share for the far right remains stable, but weakens for the left, eventually becoming negative. This divergence - which has not yet been reported in the literature - suggests that an influx of asylum seekers, particularly when coupled with an economic downturn, could tilt a political system rightwards. From a policy perspective, these results suggest that heterogeneity arising from local economic conditions has important implications for the regional allocation of asylum seekers.