Libor manipulation cost Fannie Mae and Freddie Mac more than $3 billion, according to an estimate by a government watchdog, who recommends the government-owned mortgage giants sue the big banks.

That estimate and legal advice were made in a private report by Steve Linick, the inspector general for the Federal Housing Finance Agency, the regulator for Fannie and Freddie, which were taken over by the U.S. government during the financial crisis. The Wall Street Journal first reported the watchdog's analysis on Wednesday.

The report, a copy of which was obtained by the Huffington Post, was in a memorandum prepared by the FHFA watchdog's staff and delivered to FHFA Acting Director Edward DeMarco on Nov. 2. The FHFA did not immediately respond to a request for comment. According to the WSJ, the FHFA has told Linick in a recent letter that it was "exploring potential legal options."

“We conducted a preliminary analysis of potential Libor-related losses at Fannie and Freddie and shared that with FHFA, recommending that they conduct a thorough review of the issue," Kristine Belisle, a spokeswoman for the FHFA inspector general, said in an email. "FHFA agreed to study the matter further.”

More than a dozen banks in the U.S. and Europe are under investigation for allegedly manipulating a key short-term interest rate known as Libor, which influences borrowing costs throughout the global economy. Swiss bank UBS on Wednesday agreed to pay $1.5 billion to settle charges that its traders manipulated Libor over several years. The bank's Japanese unit pleaded guilty to a crime -- a rarity for a bank -- and two of its former traders have also been hit with federal criminal charges. Earlier this year U.K. bank Barclays Capital agreed to pay about $450 million to settle Libor charges.

One of the early defenses raised in the Libor scandal was that interest rates were often manipulated lower during the crisis, a boon to borrowers. What was the harm in that, the defenders asked? But Barclays and UBS traders -- and likely traders at many other banks -- also manipulated interest rates higher, to help bolster profits in derivatives trades.

Higher Libor rates might have hit Fannie and Freddie with higher borrowing costs, but the inspector general's analysis only covers the period during and after the financial crisis, in which banks were mainly manipulating Libor lower to hide their own troubled finances.

According to the inspector general's report, Fannie and Freddie may have been cheated out of interest-rate payments on floating-rate bonds and interest-rate swaps by the lower Libor rates. Lower Libor rates during that period may have helped Fannie and Freddie in other ways, offsetting some of those losses, according to the report. But that was apparently not enough to help Fannie and Freddie avoid losses.

Libor-setting banks have already been sued by their trading partners and customers alleging billions of dollars in losses. And Wall Street analysts have tried to estimate just how big the legal liability could be for the banks. But Linick's analysis marks the first semi-official estimate of actual damages caused by the Libor scandal.

Of course, $3 billion is not a great deal of money, in the grand scheme of things. JPMorgan Chase's "London Whale" trading debacle cost it twice that much.

But it's not nothing. It will be interesting to see if the U.S. government, which has avowed a fear of prosecuting banks and bankers, but has been more willing to extract large fines, will try to get that $3 billion back for the taxpayers who own Fannie and Freddie.

This post has been updated with additional details throughout.

Also on HuffPost:

Loading Slideshow...
  • Workers are not reaping the gains of their extra productivity.

    Worker productivity grew 11 times more quickly than worker pay between 1979 and 2011: While <a href="" target="_blank">worker productivity rose 69 percent</a>, median hourly compensation rose just 6.5 percent, according to the Economic Policy Institute. [Chart credit: <a href="" target="_hplink">Economic Policy Institute</a>]

  • CEO pay has skyrocketed.

    Maybe it's time to consider your CEO's massive pay package as a cut out of your own paycheck. <a href="" target="_hplink">CEO pay is more than 200 times</a> that of a typical worker, up from 30 times that of a typical worker in the late 1970s, according to the Economic Policy Institute.

  • There aren't enough jobs.

    At its current rate of job creation, the U.S. will not return to its pre-recession unemployment rate of around 5 percent before 2020, according to the Economic Policy Institute.

  • Job growth was slow even before the recession.

    From the Economic Policy Institute: "The business cycle from 2000-2007 is the weakest full business cycle on record for job creation, due to the fact that demand was insufficient to drive overall GDP gains that were robust enough to generate strong job growth." It appears that the middle class squeeze has hurt job creation and economic growth.

  • We are poorer than we could be.

    Households in the middle fifth of income distribution would have been making $18,897 more per year as of 2007 if their incomes had grown as quickly as overall average incomes between 1979 and 2007, according to the Economic Policy Institute. (The sizable income growth for top earners since 1979 skewed the overall average.)

  • The rich have captured most income growth.

    The top one percent captured 60 percent of total income growth between 1979 and 2007, while the bottom 90 percent was left with just 9 percent of the total, according to the Economic Policy Institute. Moreover, the top one percent's incomes rose 241 percent, in contrast to 11 percent growth for the bottom fifth and 19 percent growth for the middle fifth. [Chart credit: <a href="" target="_hplink">Economic Policy Institute</a>]

  • Wages have grown more quickly for the rich.

    Wages for the top one percent spiked 131 percent between 1979 and 2010, while wages for the bottom 90 percent of workers rose just 15 percent over that same period, according to the Economic Policy Institute. [Chart credit: <a href="" target="_hplink">Economic Policy Institute</a>]

  • The poorest Americans are earning less than in 1979.

    Americans in the bottom tenth of the wage distribution earned less last year than the lowest earners did in 1979, accounting for inflation, according to the Economic Policy Institute. Meanwhile, the real wages of the median worker rose only 6 percent between 1979 and 2011.

  • The American Dream is eroding.

    "Families headed by early baby boomers (born between 1945-1954) are the last generation (on average) to achieve higher living standards than the one that preceded them," the Economic Policy Institute says. Among families with incomes below $28,000 in 1994, less than 1 percent made it to the top fifth of incomes 10 years later, according to the Economic Policy Institute.

  • This has been a lost decade.

    On average, hourly pay has not grown at all since 2002 for workers with a college degree or with only a high school degree, according to the Economic Policy Institute. Wages have not grown for college graduates in nearly every occupation, and college graduates in the 70th income percentile or lower have had stagnant or falling wages since 2000. [Chart credit: <a href="" target="_hplink">Economic Policy Institute</a>]