Friday night, Jon Stewart and Stephen Colbert gave a candid talk to a crowd in Montclair, NJ at a fundraiser for the Montclair Film Festival. Sharilyn Johnson of Third Beat reports that Stewart admitted that after taking over "The Daily Show" from previous host Craig Kilborn in 1999, Stewart wanted to bail on the job.

“What I did not realize is, a lot of the people who worked there were assholes,” Stewart said. He ultimately decided to stick around for the job although he said that it took him about two and a half years to get comfortable. But his decision to stay came after much resistance from Kilborn's previous staff, many of whom Stewart inherited.

One of those staffers was a young actor named Stephen Colbert, whose first interaction with Stewart was marked by the type of friendly competition the two men would see throughout their careers. When Stewart was first hired as the "Daily Show" host, Comedy Central held a press conference to make the announcement, and Colbert, as a mock reporter for "The Daily Show," covered the event. He deadpanned to Stewart: "Does this announcement have any effect on the prospects of me getting the hosting job?”

At the Montclair event, Colbert fleshed out the relationship that he and Stewart have developed over the years. Kristin Wald of Baristanet, who also covered the event, wrote that Colbert compared his and Stewart's relationship to that of musicians Elvis Costello and Bruce Springsteen. Early in his career, Costello tried desperately to emulate Springsteen, but in failing to mimic the New Jersey rocker, Costello developed his own personal style.

“I tried to be like Jon Stewart,” Colbert explained. “And by trying to be him, I found myself.”

Beyond the origin stories of "The Daily Show" and "The Colbert Report" as we know them today, Stewart and Colbert shared a few other anecdotes from behind the scenes at the show. Stewart said that the worst guest he ever had was, surprisingly, British actor Hugh Grant.

“And we’ve had dictators on the show," Stewart quipped. He remembered Grant as being especially unpleasant to deal with, and promised he never would return.

Another guest unlikely to return is Yusuf Islam, a.k.a. Cat Stevens. The singer-songwriter appeared on Stewart's "Rally to Restore Sanity" in 2010, and his inclusion set off the ire of Salman Rushdie, a friend of "The Daily Show" who was the target of a fatwa that Mr. Islam endorsed in the '80s.

After Stewart learned that Rushdie was offended, he reached out to Islam to find out how he really felt -- and the conversation did not leave Stewart confident that Islam was truly remorseful.

"Death for free speech is a deal breaker,” Stewart said.

To read the full recap of the event, click over to the Third Beat.

Also on HuffPost:

Loading Slideshow...
  • He Hangs Out With The Muppets

  • He Debated Bill O'Reilly For Over An Hour

    And we'd like to think that he won.

  • And Let's Not Forget: He Also Sat On Papa Bear's Lap

  • He Helps Promote Local Businesses

    Yes, even strip clubs.

  • Even Cats Love HIm

    To be fair, his guest that night was a bag of Meow Mix.

  • He Doesn't Forget His Stand-Up Roots

  • He Saved Us From Craig Kilborn

    What if we were still watching "The Daily Show with Craig Kilborn"?

  • Stephen Colbert

    Stephen Colbert was already a correspondent on "The Daily Show" when Jon took over, but the chemistry between the two is what really allowed Colbert to shine and eventually break off and host his own show. Here's a clip from Jon's first episode of "The Daily Show" in January 1999, which also featured a segment with Colbert (at around 4:59).

  • He Gave Us The Rally To Restore Sanity

  • He Made Us Cry After 9/11

  • He Can Handle An Awkward Interview

    Backstage at "The Daily Show," Zach Galifianakis gives Jon Stewart <a href="http://www.funnyordie.com/videos/5325b03b52/between-two-ferns-with-zach-galifianakis-jon-stewart">the "Between Two Ferns" treatment</a>.

  • His Biannual 'Night Of Too Many Stars' Autism Education Benefit Show

  • He Was Kind Of A Badass In The 90s

    Before he was informing twentysomethings about political and media blunders, he was making crude jokes in a leather jacket (and smoking in non-smoking areas) at colleges like in this video.

  • He Seems Like A Great Dad

  • He Has A Great Rapport With The First Lady

  • He's Not Bad With President Obama, Either

  • He Hires Amazing People

  • Even Republicans Like Him

    Some even think he's the <a href="http://www.huffingtonpost.com/2012/08/12/paul-ryan-jon-stewart-is-_n_1770895.html">funniest person in America.</a>

  • He Told The 'Crossfire' Hosts That Their Show Was Terrible

    Remember this?

  • He Stood Up For 9/11 First Responders

  • He Has Serious Chats With Bono

  • He Interviewed George Carlin

    Two great minds came together in 1997.

  • He Hangs Out With Brian Williams, Al Roker and Danny DeVito

  • This Airport Security Bit From The 2002 Grammy Awards

    Now that's commitment to a bit.

  • He Seems Like A Really Fun Uncle

  • His Correspondents Go On To Do Great Things

    See: Stephen Colbert, Steve Carell, Rob Corddry, et al.

  • He Doesn't Get Caught Up In 'Late Night Wars'

  • He's An Emmy-Magnet

    He's won ten in a row!

  • He Raised Money For Hurricane Relief

  • He Wore An American Flag Jacket... For America

  • He Hangs Out With Pirates

  • Fictional Jon Stewart Took Over For Larry Sanders

  • His Amazing Glenn Beck Impression

    All he needs are a chalkboard and a pair of glasses.

  • Let's Just Stop And Appreciate How Cool This Photo Is

  • He's Loyal To His Friends, But Expects Them To Behave

    See: Weiner, Anthony.

  • Even Grandma and Grandpa Like Him

  • This Interview With Baby Jon Stewart And Baby Conan O'Brien

  • His Takedown Of Jim Cramer

  • His Reputation Is Impeccable

    Many consider him to be the <a href="http://www.reuters.com/article/2010/10/27/us-jonstewart-idUSTRE69Q07R20101027">most trusted man in America</a>

  • He Hosted 'SNL'

    In 2002.

  • He Even Beats Jay Leno's Ratings Sometimes

    At least in the demographics that matter.

  • His Stereotypical Jersey Character

    <a href="http://www.huffingtonpost.com/2011/10/18/jon-stewart-no-disrespect-supercut-video_n_1018184.html">No disrespect.</a>

  • He Has A Heart

    He was brought to tears by Jodi DiPiazza and Katy Perry at this year's Night of Too Many Stars.

  • He Played Soccer In College

    At the College of William & Mary (And no, that's not him in the picture).

  • He Was In 'Half Baked'

  • His Epic Fight With Conan O'Brien And Stephen Colbert

  • America: The Book Is Amazing

  • His Love For Don Rickles At The Comedy Awards

  • He Was In The Underrated 'Death To Smoochy'

  • Let's Face It, He's A Dreamboat