There is a phase of growth in which our brains aren't fully developed to grasp certain things. The peekaboo effect, for instance. We all know that children think we've disappeared and appeared when we play with them. The same goes for when your kids stand right in front of you to watch television. They actually don't realize that they're blocking you. They think you're seeing with them. With their eyes. That's how they experience the world. It revolves around them and their senses. Literally.
So don't be mean to your kids. They're not being rude or selfish. They don't know any better. I believe that stage ends around 5-6. Children start to understand that others have needs and experience a range of emotions just like they do. They start to empathize with others.
Now, apply that same concept to privilege. When you are privileged (male, white, both, either/or), you are oblivious to the experiences of others (women, POC, gay, religious, non-religious). You think that others can just stop thinking or talking about discrimination and it'll go away. That because you don't experience it, it doesn't exist.
Some people can't wish discrimination away when the world is constantly reminding them that they're different via a a number of things: from church, the board of ed, hiring practices, housing, health, etc. And just because some of those folks got incredibly lucky and were born and coasted through life without being persecuted (even though they're part of a minority), it doesn't mean that discrimination doesn't exist. One, because they've been privileged. And two, because one experience doesn't equal a trend.
Privilege, no matter how sweet, freeing, liberating it is, gives us blind spots -- just like the blind spots a child has up to a certain age. So don't be so quick to dismiss the concerns of women, POC, gay, religious, non-religious... because you don't see or aren't experiencing certain things. Privilege atrophies the heart and mind, especially if you don't acknowledge or relinquish at least some of it.
Open your mind and your heart. You will see that others hurt when being mistreated. However big or small that slight may be, it still hurts. And not everyone has the privilege to dismiss or suppress those obstacles, those hurts.
There are other people outside of your being. Acknowledge them. Respect them.