The only party that spoke against it was the extreme right, and any mention of anti-immgration caused a massive shitstorm by any other political party that refused to touch the subject.

Yeah, this is the problem right here. If none of the mainstream political parties are representing a particular opinion, large swathes of voters will go to the fringe parties that do represent how they feel. And this is how fringe far-right parties become the new mainstream.

It sucks to admit it, because on humanitarian grounds, I'm pretty in favour of accepting migrants, refugees, etc. But if such a large fraction of the population is against it, it simply can't happen, and compromises have to be made.