And the reason this works, if it works, is because of reciprocity.

The mechanism of action of the "white gaze" is the stereotype held by black people (or other non-white people; and perhaps not without justification) about the expectations of white people.

So the question is: how do I avoid having people's stereotypes about me affect them so negatively?

If you are an individual to them, they can replace their stereotype of you with who you are as an individual.

And a good way to encourage them do that is to see them as an individual, keeping in mind that they might have experiences of a nature that is unfamiliar to you. This will tend to encourage them to, reciprocally, view you as an individual. (And assuming you're not actually confirming their stereotypes of you.)

Once properly formulated (as above), though, this isn't the only answer. There are other ways to subvert stereotyping.