I generally agree with you, but it is so complicated. I read a piece in The Nation a few years ago (written 2019) and whenever I see a question like this I have to dig it up. Sex workers in Spain applied to become a union (OTRAS, for short, full name basically means “the other women") and were approved in August 2018. Here are a few snippets:

After OTRAS was legalized, its two dozen or so members—who include women and men, both trans and cisgender—quickly found themselves engulfed in a national controversy. Prominent activists, academics, and media personalities swarmed social media under the hashtag #SoyAbolicionista (“I’m an Abolitionist”) to denounce what they saw as basic exploitation masquerading as the service economy. The union’s opponents argue that in a patriarchal society, women can’t be consenting parties in a paid sexual act born of financial necessity. They liken sex work to slavery, hence their name: “abolitionists.”

OTRAS calls this abolitionist opposition “the industry.” “They live really well off of their discussions, books, workshops, conferences, without ever including sex workers,” Necro says. “We’re not allowed to attend the feminist conventions.” OTRAS accuses “the industry” and the government—the two loudest arms of the abolitionist camp—of racism and classism, and is irked by their claims to feminism. “A government that refuses to guarantee the rights of the most vulnerable, poorest women with the highest number of immigrants? How is that feminist?” Borrell bristles. “We’re the feminists, the ones fighting for their rights.”

While advocates for legalization argue that it will make sex work safer, abolitionists counter that it could instead endanger women who, unlike the members of OTRAS, did not choose to enter the profession on their own. Abolitionists frame their anti-prostitution stance around the issue of human trafficking, specifically for prostitution. They argue that regulating sex work will simply allow traffickers to exploit women under legal cover.

“The trafficked women have no papers, so if police raid a club, the women have no choice but to say they’re there because they want to be,” says Rocío Nieto […] Once law enforcement is out of earshot, Nieto says, “none of the women tell you they want to be there. None of them tell you they want to do that work.”

A handful of smaller radical-left parties also back OTRAS, as well as one unlikely ally: the right-wing Ciudadanos party, known for its harsh anti-immigration stance, among other more traditionally conservative postures. “Experience shows us that when the State refuses to regulate, the mafias make the rules,” the party’s press corps wrote me in an e-mail.