It's time to play our favorite game: Guess the man who wronged Taylor Swift.

It's a pastime that is sweeping the nation, as Swift's latest album "Red" has given us 16 new tracks to analyze. Previously we deduced that "We Are Never Ever Getting Back Together" is likely (almost definitely) about Jake Gyllenhaal -- and though the two dated for only a few months in 2010, it looks like Swift got some good material out of it.

An insider reveals to Us Weekly that the song "All Too Well" is also about Gyllenhaal -- to which all we can say is "duh."

Anyone who followed Swift and Gyllenhaal's short-lived romance, which included Park Slope coffee dates while visiting the actor's sister, Maggie Gyllenhaal, knows that "All Too Well" is about their time together.

In the song Swift sings about the scarf she left at his sister's house, which of course, has to be the same scarf both were photographed wearing during their relationship (that same scarf makes an appearance in the video for "We Are Never Ever Getting Back Together"). Swift seems pretty obsessed with that scarf, judging by her lyrics:

"Now you mail back my things and I walk home alone/ But you keep my old scarf from that very first week/ Cause it reminds you of innocence and it smells like me/ You can't get rid of it, cause you remember it all too well."

The 22-year-old songstress recently told "Good Morning America" that "All Too Well" was the hardest song on the album to write because it took her a long time to filter through everything she wanted to say. "It started out being a 10-minute song, which you can't put on an album. I had to filter it down to a story that could work in the form of a song," she explained.

Swift is known for writing about her exes, but it reportedly hasn't scared off One Direction's Harry Styles, who she may have already written a song about. Swift is rumored to have had a fling with the British boy band member and penned "I Knew You Were Trouble" about him. If true, that hasn't stopped him from going after Swift again, as the two are reportedly currently seeing each other.

Loading Slideshow...
  • Joe Jonas

    Taylor Swift started dating Joe Jonas back in summer 2008, and that relationship fell apart a few months later. Jonas reportedly broke up with Swift in a<a href="http://www.justjared.com/2008/11/06/joe-jonas-taylor-swift-phone-call-dump/"> 27-second phone call</a> and then started <a href="http://www.hollyscoop.com/joe-jonas/joe-jonas-and-camilla-belle-dating.html">dating actress Camilla Belle</a>. Swift was heartbroken and <a href="http://www.mtv.com/news/articles/1655326/Taylor-Swift-Dating-History-Joe-Jonas-Jake-Gyllenhaal.jhtml">penned the song "Forever and Always" </a>about Jonas, in which she called him a "scared little boy." She also wrote the song "Better Than Revenge," which is <a href="http://voices.yahoo.com/taylor-swift-she-mocking-actress-camilla-belle-7038864.html">believed to be about Belle stealing Jonas away from her. </a>

  • Taylor Lautner

    Taylor Swift met then 17-year-old Taylor Lautner on set for the film "Valentine's Day," and dated from August 2009 until November of that year. It's believed that the song "Back To Decemeber" is <a href="http://www.mtv.com/news/articles/1649822/taylor-swift-back-december-lyrics-apology.jhtml">Swift's apology to Lautner for their breakup</a>.

  • Lucas Till

    Taylor Swift dated her "You Belong With Me" video co-star Lucas Till for a short time in 2009. <a href="http://www.thehollywoodgossip.com/2009/12/lucas-till-i-dated-taylor-swift/">After they broke up, the actor told MTV</a>: "We dated for a little bit. But, there was no friction because we were too nice," he said. "Most relationships work out cause you get along and then you don't, and then you make up and its passionate, and with us I really just liked her as a friend. That’s the only reason that didn't work out."

  • John Mayer

    Taylor Swift dated notorious ladies' man John Mayer from November 2009 to February 2010. The breakup left Swift heartbroken and she penned the song "Dear John" about it. Some have speculated that Swift lost her virginity to Mayer, as the brief relationship really did a number on her. Earlier this year, Mayer <a href="http://www.huffingtonpost.com/2012/06/06/john-mayer-taylor-swift-dear-john-cheap-humiliating_n_1573416.html">admitted that he felt "humiliated"</a> when he heard the song, but Swift refused to own up to writing about him. She told Glamour magazine that it was "presumptuous" of him to think she'd write about him. Despite her protests, we all know "Dear John" is about Mayer, and it's <a href="http://www.huffingtonpost.com/2012/10/09/taylor-swift-i-knew-you-were-trouble-song-likely-about-john-mayer_n_1951492.html">likely "I Knew You Were Trouble" is as well. </a>

  • Cory Monteith

    Taylor Swift was rumored to have dated "Glee" star Cory Monteith for a few months in 2010. Both played coy and said they <a href="http://content.usatoday.com/communities/entertainment/post/2010/05/is-taylor-swift-dating-glee-star-cory-monteith/1">were just good friends,</a> but it looked like they may have been friends with benefits, if not actually a couple.

  • Toby Hemingway

    Taylor Swift and Toby Hemingway filmed a music video for Swift's song "Mine" together, and they <a href="http://www.telegraph.co.uk/news/celebritynews/7968407/Taylor-Swift-inseparable-from-British-actor-Toby-Hemingway.html">reportedly took the romance offscreen for a short time in 2010. </a>

  • Jake Gyllenhaal

    In October 2010, <a href="http://www.huffingtonpost.com/2010/10/25/taylor-swift-jake-gyllenh_n_773466.html">Taylor Swift's romance with Jake Gyllenhaal</a> came as a surprise to most people. The brief relationship suspiciously played out in the public eye as the couple were often photographed -- as seen here strolling in Brooklyn on a coffee date. The odd pairing had many wondering if the relationship was a publicity move, but it seems Swift really had her heart broken. The couple broke up in late Decemeber 2011,<a href="tp://www.usmagazine.com/celebrity-body/news/taylor-and-jake-why-they-split-201141#ixzz2AKsPn9wy"> with Us Weekly reporting </a>that the then-30-year-old actor wasn't into her anymore and he could feel the 10-year age difference. It wasn't quite over as they were spotted having dinner together in January 2011, but called it quits again shortly after. It's believed that Swift's song "We Are Never Ever Getting Back Together," as well as a number of other songs on her new album, "Red," <a href="http://www.huffingtonpost.com/2012/09/05/taylor-swifts-we-are-never-ever-song-jake-gyllenhaal_n_1858318.html">were written about Gyllenhaal.</a>

  • Zac Efron

    Taylor Swift and Zac Efron were rumored to be dating while they were promoting their movie "The Lorax" in February 2011. Both stars <a href="http://www.accesshollywood.com/taylor-swift-and-zac-efron-address-dating-rumors_article_60944">denied they were together</a>, but judging from <a href="http://www.huffingtonpost.com/2012/10/25/taylor-swifts-ex-boyfriends-ellen-video_n_2015960.html?utm_hp_ref=celebrity">Ellen DeGeneres' recent decree that the were a couple,</a> we're going to go out on a limb and say they totally, maybe dated.

  • Garrett Hedlund

    Taylor Swift and Garrett Hedlund were spotted having dinner together in April 2011, which <a href="http://www.huffingtonpost.com/2011/04/06/taylor-swift-garrett-hedlund-date_n_845418.html">sparked rumors the two were dating. </a> "It was their first time going out," a source told Us Weekly. "They've been emailing for a while, and she hinted they should get together."

  • Eddie Redmayne

    Taylor Swift dated actor Eddie Redmayne when she was auditioning to play Eponine in a film adaptation of "Les Miserables" in October 2011. "They hung out in New York City with the movie's execs," <a href="http://www.usmagazine.com/celebrity-news/news/why-taylor-swifts-secret-romance-with-eddie-redmayne-fizzled-201292">a source told Us Weekly</a> "And Taylor developed feelings for him fast." The relationship didn't last long, and the couple broke things off in February 2012, because Eddie reportedly wasn't interested in doing the whole long-distance thing.

  • Harry Styles

    Recently there have been rumors that Taylor Swift dated One Direction's Mick Jagger look-alike, Harry Styles. <a href="http://perezhilton.com/2012-10-19-taylor-swift-secretly-dated-one-direction-harry-styles-cheated-dumped-romance#.UImJGml25Og">She reportedly dated him before she started dating Conor Kennedy in summer 2012. </a>

  • Conor Kennedy

    Taylor Swift is OBSESSED with the Kennedy family, so naturally she decided it was a good idea to start a relationship with 18-year-old Conor Kennedy. The couple were first spotted together in July at his family's compound in Hyannis Port, and Swift reportedly went to far as to buy a home close by. Swift <a href="http://www.huffingtonpost.com/2012/10/10/taylor-swift-conor-kennedy-grown-man-defends-self-rolling-stone_n_1954553.html">defended herself against critics of the relationship</a> who couldn't understand what a 22-year-old woman was doing with a high school student. The relationship quickly came to an end, however. <a href="http://www.huffingtonpost.com/2012/10/25/taylor-swift-conor-kennedy-split-after-three-months-of-dating_n_2017743.html?utm_hp_ref=celebrity">It was reported on Oct. 25 </a>that the couple called it quits and hadn't seen each other in a month. At least Swift was able to get some good material out of the relationship, as she penned "Starlight" about the Ethel Kennedy, and it's believed that "Everything Has Changed" was written about Conor.

Related on HuffPost: