There are three problems with this from a logical perspective.

(1) How do they know that they "are women"? (Or how do you, for that matter? How do I know that I'm not?) We each presumably can know who we are, but why would that mean that we also can use the label in a coherent way with how others use it? What does it mean to say that a label is applied correctly (or not)? Or are you actually saying that "woman" has no content beyond the declaration that you are (or are not) one?

(2) Does this not apply to people with phantom limb syndrome--should we say "we have to trust that they still have the limb"? If someone has bodily integrity dysphoria, do we agree that the offending limb(s) really aren't "part of them"? Why or why not?

(3) Isn't your declaration circular? What is a "trans woman" aside from someone who doesn't, by some other criteria, seem obviously to be a woman but who says she's a woman? Aren't you actually saying, "We have to trust anyone who says they're a trans woman that they are a woman?"

From a pragmatic perspective, I don't think this is what usually happens. Usually there's some measure of psychological distress, a long-standing discrepancy in self-identification (or desire for identification), etc. etc.. There's a somewhat consistent phenomenon that we put a label to (precisely because of the consistency), and it's also possible for people to use the wrong label and for something else to be going on; psychologists try to catch these cases before recommending gender-affirming care. One can fully support the idea that (at least some) trans women are women (in some meaningful sense) without needing to also conclude that the terms are all arbitrary matters of self-declaration.