It's a tad murky, but I'd say a person referred to as androgynous might sound more akin to hermaphroditic, possessing both male & female reproductive organs, though it can take on that meaning as well. Depends on context.
Androgyny would be clearer applied as an adjective to a certain personal trait - an androgynous appearance / look, hairstyle, name etc.
Besides, who decides what sort of traits get to define a gender? You might want to take such thoughts with a grain of salt, and come to terms with the basic truths.