Parenthood has the power to transform how we see the world—and hear about its injustices—in immense ways. Upon having children of our own, it’s natural to carry the new weight of concerns for less privileged babies of far away places or to feel heartache when imagining our sons’ or daughters’ faces in place of those on the nightly news.

But while a deepened sense of empathy is one thing, parenthood should not be the baseline for a moral code.

This week, as dozens of women have come forward with sickening stories about the sexual offenses they suffered at the hands of Hollywood executive Harvey Weinstein, high-profile actors and actresses have shared welcomed statements of support for the victims.

There is just one troubling trend with some of the reactions: In multiple cases, the people—especially the men—referenced their own daughters as explanation for why they see the behavior as so repugnant.

Matt Damon speaking to Deadline about reports he helped kill an exposé on Weinstein in 2004: “Look, even before I was famous, I didn’t abide this kind of behavior. But now, as the father of four daughters, this is the kind of sexual predation that keeps me up at night. This is the great fear for all of us…”

Ben Affleck in a statement on Twitter (before apologizing for groping a young actress in 2003): “This is completely unacceptable and I find myself asking what I can do to make sure this doesn’t happen to others. We need to do better at protecting our sisters, friends, co-workers and daughters.”

Vanessa Carlton in a since-deleted Twitter plea to Gwyneth Paltrow to speak out against Weinstein (which Paltrow later did): “Big fan here, your silence on Harvey Weinstein is contrary to everything you stand for. Do it for your daughter.”

This narrative isn’t wholly surprising, given we’ve seen the “as the father of daughters” comment pulled out during many previous scandals.

But that doesn’t mean it’s a defense we should continue to let slide—because when we insinuate that only parents are capable of viewing women as fully formed people worthy of respect, then we’re doing a disservice to everyone.

We’re doing a disservice to women who shouldn’t have to check whether someone has procreated in order to determine if she can feel safe in that person’s presence.

We’re doing a disservice to men (the many, many men) who have always known what’s right and cared about women.

We’re doing a disservice to boys by suggesting they are off the hook when it comes to respecting women until the day they have children of their own.

And, yes, we’re doing a disservice to our daughters as they navigate this world that’s filled with plenty of child-free people. (There is little comfort to be found in, “I’m sorry you were abused by that person. He doesn’t have daughters, so he couldn’t have known better.”)

The parents of daughters or not, let’s start by holding everyone to the same standard of respect for women.