For what it's worth, I heard a version of this argument from a Human Terrain Team social scientist who happened to be female. She put on a hijab to go out and talk to Muslim women in Iraq. What most surprised her, she told me, was the way in which it stopped soldiers from treating her as a sexual being. It's a strange fact, since it's just a piece of cloth, but for whatever reason covering the hair and head somehow disconnected the sex drive even in young soldiers long deployed at war and forbidden other avenues of sexual relief.
So maybe there's something to it. Perhaps a symbol, under the right circumstances, doesn't have to mean what we ordinarily expect it to mean. Perhaps a lot has to do, as she says, with the choice of the person who wields it.
In any case, the social scientist I knew kind of liked the effect. She didn't wear it otherwise, probably because everyone would have thought it a bit weird. But she did like the effect it had on the young soldiers around her.