Manhood is an accomplishment, an internal need and quest, for both cis and trans men. When we acknowledge this, the necessity and intelligibilty of questions like "But how do you know?" fades away. Seeing this will help dismantle one of the structuring cultural approaches to transgender people: mistrust.
We are in a society that screams that it embraces equality. But we still quietly and subtly embrace the concept of "Adam then Eve" not the equal idea of "Adam and Eve." And we would never even consider giving "Eve" first billing in the couple's title. Who's ever heard of "Eve and Adam"? No one, because when it matters, the man comes first.