THE BLOG
03/04/2013 05:19 pm ET | Updated May 04, 2013

At the Billionaire's House

In the late '80s, when I was a few years out of college, for the first time in my life I stayed overnight at a private property that could legitimately be called an estate.

These days the word "estate" seems to get applied to every development home worth more than half a million dollars. I'm not talking about that kind of estate. I'm talking acreage -- close to 100 acres within easy driving distance of a big city. I'm talking infrastructure -- multiple houses and outbuildings, driveways, a greenhouse. I'm talking mansion -- a residence with a servants' quarters.

On the bedside of the guest room where I slept that night (it should go without saying that the house had more than one guest room), there happened to be a copy of Forbes magazine featuring that year's Forbes 400. I began flipping through. Yep. The extended family that owned the house was on that list.

This blew my mind. I had not been raised without privilege myself. My father grew up poor, but he did well enough over the years to send me to a private university without loans. We had a live-in housekeeper. He put my younger sister through private school and private college. We belonged to a country club. When I moved to New York, he purchased me an apartment, and I only had to pay the maintenance.

But we lived in a ranch house that had been constructed as part of a vast development. Our house sat on a fraction of an acre. My father took an ordinary commuter train into New York. You didn't read his name in the papers or see it on any list. There was never talk of family money.

So the visit to this estate lay far outside my experience. It had a driveway that was a quarter mile long. Some of the people who worked there didn't sleep in the spare room in the basement, as our housekeeper had, but occupied entire houses of their own on the property as perks.

My boyhood home had an address. This estate had a name.

That night, before going to sleep, I wandered into the bathroom that a member of the owning family used. I expected to see toothbrushes made of gold or silver, a shaving kit worthy of a lord. Instead I found a tube of Crest and an Atra razor. The mega-rich may be different from you and me, but mostly they use the same toiletries.

Years passed. I married well and I did well. Well enough to live better than my father had by some measures. But times change. We've never had a live-in housekeeper, for example. My wife cooks our dinner most nights. Long ago, the family that owned the estate where I'd visited dropped off the list of the nation's wealthiest.

In 1982, when the Forbes 400 began, one needed "only" $75 million to make it onto the list. Today, everyone on the list is a billionaire.

Since that visit to the estate, I myself have had the privilege of meeting four of these individuals. I use the word "privilege" not to suggest these guys are better than you or me, only to say that I presume meeting a billionaire remains a rare event for most people.

All four of the billionaires I've met are self-made. One founded a private industrial company. He lived near our house an hour from New York, and we got invited to a couple of parties at his modernist estate. His staff served great wine, I have to tell you, but it was often ill-matched to the food.

Another of the billionaires I've met is Michael Bloomberg, who of course founded a media company. At a charitable event some years ago, I shook hands with him and we had a brief chat.

When I was a literary agent, I represented Howard Schultz of Starbucks, who became a billionaire a decade after we did business together. We spent some quality hours with one another, and I flew on the corporate jet, but that wasn't social time. I never got invited to his house.

The last billionaire I met is most relevant to the never-ending debate in America about the fairness of our financial markets. He possesses a substantial Wall Street fortune, amassed by trading securities. This billionaire may or may not use Crest toothpaste and Atra razors. I never asked so I'll never know. I do know he wears boxers, because he was famous among household employees for walking around his mansion dishabille.

For about a year, I was invited occasionally into a tennis game at his estate. Never mind how that came about. Suffice to say that I was not there as a friend. I more resembled an appurtenance, maybe like a kitchen wine fridge or a flat-screen television.

It is the nature of tennis that playing with people more skilled than you tends to elevate your own game. The billionaire wasn't a bad player, but he'd never been a great one, and he was older than me by thirty years when we met. Nevertheless, it was the job of his social secretary, who always made the arrangements, to find three players of greater skill to participate in the billionaire's recreation. And, not incidentally, to lift up his game.

The billionaire never called me. Only the secretary did. But when I got that call, I have to admit, I dropped everything to get there. There was usually a pro on the court and the other player was guaranteed to be high-level. Plus, the billionaire himself made up with sheer competitiveness what he lacked in skill.

And, of course, there was the fact that playing tennis at the house of a billionaire gave one the opportunity to say he played tennis at the house of a billionaire.

Check that. Having thought this over, I must hasten to add that to say I played at the billionaire's "house" would be an overstatement, as he never invited any of us to set foot inside the mansion. We played at the estate, and this estate was something of a compound, almost like a high-class resort where everything remained pristine due to an exceedingly high staff-to-guest ratio. There was a barn and fields, a skating rink, and a giant indoor pool worthy of a luxury condominium or the setting of a Roman bacchanalia. I was never invited to use any of those facilities, either.

Between sets, we sometimes did chat, but only perfunctorily. It was around the time of the post 9/11 invasions, and the billionaire had some strong opinions about that. We disagreed, but that didn't matter. The billionaire made it clear, without quite coming out and saying so, that I wasn't there for my opinions on anything. I was there to fill out a foursome.

Despite this disrespect, I found it hard to resist the siren call of the billionaire's compound. As I said, the quality of play was usually pretty good. And once you received that call, you knew the game was guaranteed to go forward. We played on the outdoor tennis court when weather permitted, on the indoor tennis court at other times. If a squall came through mid-set, you just went inside to finish. The fact that these courts existed at the billionaire's house and you were playing with the billionaire necessitated zero accommodation to the needs of others. The billionaire wanted what he got, and he got what he wanted.

This went on for about a year. Then, one day, I had a bad outing. Maybe I was tired or unfocused or I just wasn't seeing the ball well that day. I missed some easy shots, shanked a few balls. I wasn't holding up my end of the bargain, my responsibility to help elevate the level of the billionaire's play. It may have been coincidence, but I never got invited back after that.

Years later, I met a woman who had worked as a stock trader for the billionaire at a time when he only had hundreds of millions. She said he used to call from all over the world and ask her to execute this or that trade from the computer at her desk. At the time she was only an order taker. Like at the McDonald's drive-thru but with better compensation.

That's what these people did. They pressed buttons on a trading terminal and money came out. Sometimes a little bit of money. Sometimes a lot. If it was easy, of course, we'd all be doing this. For one reason or another, it is hard for most of us but easy for the billionaire. When he pursued riches in this fashion enough money came out of that terminal to buy all the toys and houses and buildings and cars he ever wanted. Enough to buy all the people he ever wanted, too.

When this woman told me that she'd worked for the billionaire, it took my mind back to the most extensive chat I'd ever had with him between sets. I told him I'd recently read a book that might interest him, and since I'd annotated my copy I could type up the excerpts and share them. He scoffed at that, saying he'd already read the book himself. It went without saying that he didn't care what I found interesting inside its pages.

The billionaire didn't need my notes because he had his own opinions. He'd acted on those opinions, sometimes ruthlessly, and the result was evident in his Forbes 400 ranking and what you saw all around when you stood beside his tennis court.

Unlike Howard Schultz or Michael Bloomberg or the guy with the privately held industrial fortune, however, all this man had ever done was press buttons. Unlike those other billionaires whom I'd met, he hadn't built anything but his own fortune and, perhaps, the fortunes of very few others.

Where the other billionaires I'd met employed thousands of people, to my knowledge this one employed mere dozens. Yet, despite having built little more than a business that pressed the right buttons -- albeit exceedingly well -- he was the most arrogant of the four billionaires. Or maybe he was the most arrogant exactly because he had managed to amass his fortune without having to build anything of substance. In a business owner's mind, after all, workers are overhead. Ordinary people got to share in this billionaire's good fortune only a little bit and only at his sufferance. If you didn't help him elevate his game, you ended up on the wrong side of the trade.

When I think of what's happened on Wall Street in the past few years, I think of this billionaire and others like him and the buttons they push. I think that a lot of people in Las Vegas and Brooklyn and Miami and Phoenix and Detroit and across our great land ended up on the wrong side of a lot of trades in the past decade. And most of those people will never get to set foot on any billionaire's tennis court.