Am I misunderstanding the situation? The person comes from a different ethnic background than the majority, has a more conservative look at immigration, happens to have an immigrant wife anyways.
Never really heard anyone get upset at the phrase. But maybe it's a disconnect between English and Dutch language in how it's perceived?
I mean I would apply the same phrase to Tomio Okamura in the Czech Republic. My aunt loves him and yet...