True enough, it doesn't invalidate the principal of consent of the governed. What it does mean is that those who invoke said principle for themselves whilst refusing to grant it to others are pretty hypocritical, and since they don't really believe in it ( else they wouldn't own slaves ) aren't in a position to complain when they themselves are on the rough end of the equation.