Actress Gabrielle Union’s new memoir provides an important reminder there is often no simple answer to the question, “When are you going to start a family?” For her, that road has been marked by “eight or nine miscarriages,” which some well-meaning people unintentionally made more painful by speculating about her family plans with husband Dwayne Wade.

As Union eloquently demonstrates in her book, We’re Going to Need More Wine, miscarriages and fertility struggles should never be sources of shame for women—but they are topics everyone else ought to be more aware of before asking assumptive questions like when it will be “your turn” to become a parent.

In excerpts from the forthcoming book published in People, Union, 44, shared she has been waiting for “her turn” for a long, difficult amount of time.

“For so many women, and not just women in the spotlight, people feel very entitled to know, ‘Do you want kids?'” Union tells People. “A lot of people, especially people that have fertility issues, just say ‘no’ because that’s a lot easier than being honest about whatever is actually going on. People mean so well, but they have no idea the harm or frustration it can cause.”

Union tells People she didn’t always want kids, but after marrying NBA star Dwyane Wade in 2014 and experiencing life as a stepmom to his boys, she changed her mind and the couple started trying.

“For three years, my body has been a prisoner of trying to get pregnant,” Union wrote in the memoir. “I’ve either been about to go into an IVF cycle, in the middle of an IVF cycle or coming out of an IVF cycle.”

The process itself is emotionally and physically taxing, as there are times during the IVF cycle where her body got so bloated she said she appeared pregnant. That led to even more questions that would be discouraging for anyone struggling with infertility.

Union tells People she hopes her book will help change the way we converse about planning pregnancies—and save other couples from questions that are well-meaning but actually open emotional wounds.

For anyone who is still wondering about her status, Union notes in her memoir that she and Wade are still pursuing IVF and “remain bursting with love and ready to do anything to meet the child we’ve both dreamed of.”

But until we hear an announcement that a baby is on the way—either from Union or from anyone we know personally—let this be a reminder that questions about others’ family plans are best left unasked.