It has almost become a truism in our culture that we need to have high self-esteem in order to be happy and healthy. Psychologists have conducted thousands of studies touting the benefits of self-esteem. Teachers are encouraged to give all their students gold stars so that each one can feel proud and special. We are told to think positively of ourselves at all costs, like in Stuart Smalley's book of positive affirmations: "I'm good enough, I'm smart enough, and doggone it, people like me!" But as research is now starting to demonstrate, the need to continually evaluate ourselves positively comes at a high price.
The main problem is that having high self-esteem requires feeling special and above average. To be called average is considered an insult in our culture. ("How did you like my performance last night?" "It was average." Ouch!) Of course, it's logically impossible for every human being on the planet to be above average at the same time. So we develop what's known as a "self-enhancement bias," which refers to the tendency to think of ourselves as superior to others on a variety of dimensions. Studies have shown that most people feel they're friendlier, more popular, funnier, nicer, more trustworthy, wiser and more intelligent than others. Ironically, most people also think they're above average in the ability to view themselves objectively! The result of wearing these rose-colored glasses isn't so pretty.
This need to feel superior results in a process of social comparison in which we continually try to puff ourselves up and put others down (just think of the film Mean Girls and you'll understand what I'm talking about). Bullies generally have high self-esteem, for instance, since picking on people weaker than themselves is an easy way to boost self-image.
One of the most insidious consequences of the self-esteem movement over the last couple of decades is the narcissism epidemic. Jean Twenge, author of Generation Me, examined the narcissism levels of over 15,000 U.S. college students between 1987 and 2006. During that 20-year period, narcissism scores went through the roof, with 65 percent of modern-day students scoring higher in narcissism than previous generations. Not coincidentally, students' average self-esteem levels rose by an even greater margin over the same period.
At the same time that we try to see ourselves as better than others, we also tend to eviscerate ourselves with self-criticism when we don't meet our high standards. As soon as our feelings of superiority slip -- as they inevitably will -- our sense of worthiness takes a nose dive. We swing wildly between overly inflated and overly deflated self-esteem, an emotional roller coaster ride whose end result is often insecurity, anxiety and depression.
So what's the alternative? How do we feel good about ourselves without needing to feel better than others and thus falling into the narcissism/self-loathing trap? One answer is to develop self-compassion.
Self-compassion involves being kind to ourselves when life goes awry or we notice something about ourselves we don't like, rather than being cold or harshly self-critical. It recognizes that the human condition is imperfect, so that we feel connected to others when we fail or suffer rather than feeling separate or isolated. It also involves mindfulness -- the recognition and non-judgmental acceptance of painful emotions as they arise in the present moment. Rather than suppressing our pain or else making it into an exaggerated personal soap opera, we see ourselves and our situation clearly.
Self-compassion doesn't demand that we evaluate ourselves positively or that we see ourselves as better than others. Rather, the positive emotions of self-compassion kick in exactly when self-esteem falls down; when we don't meet our expectations or fail in some way. This means that the sense of intrinsic self-worth inherent in self-compassion is highly stable. It is constantly available to provide us with care and support in times of need. My research and that of my colleagues has shown that self-compassion offers the same benefits as high self-esteem, such as less anxiety and depression and greater happiness. However, it is not associated with the downsides of self-esteem such as narcissism, social comparison or ego-defensiveness.
Instead of endlessly chasing self-esteem as if it were the pot of gold at the end of the rainbow, therefore, I would argue that we should encourage the development of self-compassion. That way, whether we're on top of the world or at the bottom of the heap, we can embrace ourselves with a sense a kindness, connectedness and emotional balance. We can provide the emotional safety needed to see ourselves clearly and make whatever changes are necessary to address our suffering. We can learn to feel good about ourselves not because we're special and above average, but because we're human beings intrinsically worthy of respect.
Why not try it? If you want to learn more about self-compassion, order my new book titled "Self-Compassion: Stop Beating Yourself Up and Leave Insecurity Behind," or test your own self-compassion level, go to self-compassion.org.
HuffPost Lifestyle is a daily newsletter that will make you happier and healthier — one email at a time. Learn more