How does exposure to refugees influence political behavior? We present evidence from Hungary, a country with widespread anti-immigration attitudes, that short term exposure during the 2015 refugee crisis predicts anti-refugee voting and sentiment. We code exposure to refugees at the settlement level using reports from state media, an independent online news site, and an online social media aggregator. Settlements through which refugees traveled showed significantly higher anti-refugee voting in a national referendum in 2016. The effect decreases sharply with distance. Using a difference-in-differences model, we find that the far-right opposition gained, while the governing right-wing party lost votes in these settlements in subsequent parliamentary elections. This suggests incumbents are punished by voters skeptical of immigration regardless of their policy position. Survey data supports this finding of a competition among right-wing parties, as individuals in exposed settlements are more fearful of immigrants and support restrictive policies only if they identify as right-wing.