I'm not sure where the line is either, and I think things are just going to get weirder with sex and gender related issues.
I know that I feel bad for Semenya that this detracts from her accomplishment. I don't think she's doing anything to deliberately cheat, and I think she thinks of herself as a woman. But when she was required to keep her testosterone down, she was nowhere near as competitive. It makes it pretty clear that it's the extra testosterone that is her gift and it's a gift that other female athletes can't supplement. (ETA: actually she was still competitive; what she wasn't was as dominant.)
If I were making the rules, I think that I'd say women competitors had to be XX and that any sex specific hormones had to be within a clearly defined range. This would be a bummer for the intersex and trangender folks, but it would be clear and fair for the other 99 percent of folks. If testosterone didn't confer such an athletic advantage, I don't know that I'd care very much.
If I knew any transgender athletes or intersex period in real life, maybe I'd feel different about it.