Comedians are almost without exception liberal. It's not a question of fearing Obama as them liking him. Can you think of a funny conservative comedian? I can't.
Also, in the interview I listened to, he was more concerned about jokes about Obama being edited and taken out of context (usually by Fox) than saying people were scared of being labelled racists.