I doubt Gypsies and Jews were never counted as Swedes AFAIK, then why would they? They have a deeper different culture and appearance. An Icelander, a Finn and a Norwegian in Sweden is different from a Somali refugee or a MENA, many of them here since the 90s, your logic is ridiculous.. while the other three Nordic groups we share an ancient history with similar ethnic backgrounds..