Height is to women, what Beauty is to men. Men find beautiful women most physically attractive. Women find tall men most physically attractive. But there are other traits to consider for sexual attraction, for both men & women.
Women really don´t usually care about height all that much. It´s usually disillusioned shorter men who imagine this. As long as the man isn´t clearly smaller and shorter than the female it´s not of much importance really. Women are more interested in a man´s mental qualities like confidence and a sense of drive and direction. At least in more serious relationships.