I think it's more a fear of being hated, which in my opinion is the cause of a lot of racism that takes place in our day. There also are massive cultural differences that divide the people even if they were intent on getting along in the first place.
Hence the Ali G stereotype...a Pakistani trying to act Jamaican. (played by a Jew, should Muslims and blacks be offended by this??? haha Oh well I think it is funny as hell).