WASHINGTON -- Former Internal Revenue Service Commissioner Douglas Shulman is coming under further scrutiny over the number of times he visited the White House while the IRS targeted conservative groups for additional review, with the Daily Caller reporting Thursday that Shulman visited the White House 157 times over the four-year period.

Rep. Jim Jordan (R-Ohio) confronted Shulman over the White House visits during a May 22 Oversight & Government Reform Committee hearing. Jordan declared that Shulman -- who was appointed by President George W. Bush -- had visited the White House 118 times during the period of time that the IRS' Cincinnati office was targeting conservative groups for further review of their tax exemption applications, according to the publicly disclosed White House logs.

But on Thursday, the Daily Caller reported that Shulman had visited the White House 157 times between January 2009 and January 2013, according to the White House logs -- more than the top Cabinet officials in President Barack Obama's White House. The Daily Caller suggested that the high number of Shulman's visits "strongly suggests coordination by White House officials in the campaign against the president’s political opponents."

The problem with this assertion, and the touting of Shulman's visits to the White House based on the publicly available visitor logs, is that it lacks context and relies on cursory reading of the logs.

In fact, according to visitor logs, Shulman only visited the White House -- the physical building -- 39 times when Obama was in office. The vast majority of Shulman's visits listed in the visitor logs, 115 of them, are for meetings at the Old Executive Office Building, adjacent to the West Wing, which houses offices for White House staff.

And who was Shulman visiting? The officials Shulman met with the most were Sarah Fenn -- 54 visits -- and Nancy Ann DeParle -- 40 visits. DeParle and Fenn were in charge of implementing the Affordable Care Act and Shulman's IRS had just been given broad new authority under the new health care law. Ten of Shulman's visits with DeParle took place at the White House, making up a quarter of his total White House visits.

DeParle was Shulman's most visited official at the White House followed by Deputy Director of the National Economic Council Jason Furman. Furman's research includes extensive work on both taxes and health care. He also wrote blog posts and did interviews defending the health care reform law. Shulman met with Furman nine times at the White House.

Shulman also had five meetings at the White House with Domestic Policy Council Director Cecilia Munoz, and three with her predecessor, Melody Barnes. Other White House visits by Shulman included single meetings with Cabinet Secretary Chris Lu, Lu's executive assistant Chad Maisel, Chelsea Kammerer of the Office of Intergovernmental Affairs and Office of Public Engagement and Council of Economic Advisors Chairman Alan Krueger. Shulman also entered through the visitors office twice for events held on the South Lawn.

Obama appears six times in Shulman's meetings, although the nature of all of those meetings are not directly known. One of those meetings is for a daily presidential briefing on the economy and another, which includes first lady Michelle Obama, is likely the White House's Hanukkah reception and dinner. The other meetings include one each in the Situation Room, the South Lawn, the State Floor and, generally, in the West Wing. These, like most meetings listed in the White House visitor logs, involve more than one person meeting with the president.

Shulman also met with other White House officials, mostly those involved in economic, regulatory and health care policymaking, in the Old Executive Office Building. These included former Office of Management and Budget director Peter Orzsag, former Office of Information and Regulatory Affairs director Cass Sunstein, health policy advisor Ezekiel Emanuel, former Council of Economic Advisors Chairman Austan Goolsbee, former deputy director of OMB and current deputy White House Chief of Staff Rob Nabors and former OMB deputy director Jeffrey Zients.

But overall, the vast majority of Shulman's visits recorded in the White House visitor logs were to the Old Executive Office Building and with officials involved in implementing the new health care law -- indicating that the main conspiracy unearthed here is one to implement a law passed by Congress.

This covers the point implicit in Jordan's display of the visitor logs and made explicit by the Daily Caller: that Shulman's number of White House visits "strongly suggests coordination by White House officials in the campaign against the president’s political opponents." The other point made by the Daily Caller -- that Shulman visited the White House more times than other Cabinet officials -- is also suspect.

The first reason is plainly explained by the Daily Caller in its article: "The visitor logs do not give a complete picture of White House access. Some high-level officials get cleared for access and do not have to sign in during visits."

High-level Cabinet officials routinely do not need to make an appointment through the visitor logs to gain access to the White House and, thus, the numbers are far from a complete picture. For example, former Secretary of State Hillary Clinton logged only 25 meetings in the visitor logs during her tenure, but, according to The Washington Post, actually visited the White House 681 times during that period.

The IRS commissioner, on the other hand, is not a high-level official (the commissioner is not even in the Cabinet) and because of this will record a higher number of visitor log entries simply by default. And, as stated above, the vast majority of these were not even to go to the White House. (Also, many logged visits are never attended, although record keeping on this is difficult to parse from the publicly released data.)

The visitor logs show that the top visitor to the White House itself was Attorney General Eric Holder, with 54 recorded visits. Some of these are duplicate entries, as the visitor logs show rescheduled appointments alongside the original schedule. The second biggest visitor is Penny Pritzker, Obama's second term Commerce Secretary nominee who was not in his Cabinet when the logs were recorded, with 48 visits. Shulman ranks third with 39 visits, a majority of which were with officials working on health care and economic policy.

Even though the logs are incomplete for nearly all of Obama's Cabinet, one could extrapolate influence by the number of meetings at the White House with the president. In this regard, Shulman ranks at the bottom. (It should be noted that there are some duplicate meetings counted in the logs and some separate meetings, like a state dinner followed by a dessert reception, that could be condensed into one meeting and, as previously stated, top level Cabinet officials certainly met with the president more than listed in the logs.)

According to the logs, Holder had 47 meetings with the president, Health and Human Services Secretary Kathleen Sebelius held 34, former Treasury Secretary Timothy Geithner had 31, then-senator and current Secretary of State John Kerry had 26, both Homeland Security Secretary Janet Napolitano and Transportation Secretary Ray LaHood had 25 scheduled meetings with the president, and on down the line. Shulman's six logged appointments with the president ranks him with Office of Management and Budget official and soon-to-be acting IRS Commissioner Danny Werfel, well below the number of meetings by Obama's current (including nominees) and former Cabinet.

In fact, Pritzker, Department of Justice official and Labor Secretary nominee Thomas Perez and Transportation Secretary nominee Anthony Foxx all had more appointments with the president than Shulman during a time when they were not in the Cabinet, according to the logs.

A scandal that could be extrapolated, just from simply looking at these numbers, is that Pritzker -- a fundraiser who bundled more than $500,000 in cash for Obama's campaigns -- may have more access to the White House and the president than the IRS commissioner and numerous other government officials.

Also on HuffPost:

Loading Slideshow...
  • President Barack Obama

    "This is pretty straightforward," Obama said at a <a href="http://www.huffingtonpost.com/2013/05/13/obama-irs-scandal_n_3266577.html" target="_blank">press conference</a>. "If, in fact, IRS personnel engaged in the kind of practices that have been reported on and were intentionally targeting conservative groups, then that is outrageous, and there is no place for it, and they have to be held fully accountable, because the IRS as an independent agency requires absolute integrity and people have to have confidence that they are applying the laws in a non-partisan way. You should feel that way regardless of party." (AP Photo/Jacquelyn Martin)

  • Sen. Marco Rubio (R-Fla.)

    Rubio <a href="http://www.rubio.senate.gov/public/index.cfm/press-releases?ID=bc8ce2a9-4e95-4792-8744-501d0c1b63b3" target="_blank">penned a letter</a> to Treasury Secretary Jack Lew demanding the IRS commissioner's resignation. The letter begins: "Recent revelations about the Internal Revenue Service’s selective and deliberate targeting of conservative organizations are outrageous and seriously concerning. This years-long abuse of government power is an assault on the free speech rights of all Americans. This direct assault on our Constitution further justifies the American people’s distrust in government and its ability to properly implement our laws." (AP Photo/J. Scott Applewhite)

  • House Speaker John Boehner (R-Ohio)

    "The admission by the Obama administration that the Internal Revenue Service targeted political opponents echoes some of the most shameful abuses of government power in 20th-century American history. Today, we are left with serious questions: who is ultimately responsible for this travesty? What actions will the Obama administration take to hold them accountable? And have other federal agencies used government powers to attack Americans for partisan reasons? House Republicans have made oversight of federal agencies a top priority on behalf of the American people, and I applaud the work that members such as Charles Boustany, Darrell Issa and Jim Jordan have done to bring this issue to light. I also strongly support Sen. McConnell’s call for a transparent, government-wide review to ensure similar practices are not happening elsewhere in the federal bureaucracy," Boehner said in a <a href="http://www.speaker.gov/press-release/speaker-boehner-statement-irs-targeting-conservative-groups" target="_blank">statement</a>. (AP Photo/Cliff Owen)

  • Rep. Darrell Issa (R-Calif.)

  • Rep. Michele Bachmann (R-Minn.)

  • Sen. Harry Reid (D-Nev.)

  • Sen. Chuck Grassley (R-Iowa)

  • Rep. Vern Buchanan (R-Fla.)

    Buchanan also <a href="http://buchanan.house.gov/index.php?option=com_content&view=article&id=4787:buchanan-to-treasury-secretary-the-nations-trust-in-government-was-betrayed&catid=1:latest-news" target="_blank">wrote a letter</a> calling for the IRS commissioner's resignation. His letter reads: "On March 22, 2012, as a member of the House Ways and Means Committee which oversees the IRS, we heard then-Commissioner Douglas Shulman clearly state that the IRS did not engage in the practices of which it is now accused saying "there is absolutely no targeting." Yet, less than a year earlier, Commissioner Shulman's own deputy, Lois Lerner, was made aware that such malpractice had indeed occurred. It became evident that groups with "tea party" or "patriot" in their names were extremely vulnerable to auditing harassment. Even nonprofit organizations that sought to educate Americans about the U.S. Constitution were unfairly singled out for scrutiny. The nation's trust in government was betrayed by this unconscionable behavior. On behalf of my constituents, your immediate response is not only warranted but essential to clearing up a matter that would have our founding fathers rolling in their graves." (AP Photo/Steve Nesius, File)

  • Sen. Jerry Moran (R-Kan.)

  • Sen. Dean Heller (R-Nev.)

  • Sen. Jeff Flake (R-Ariz.)

  • Rep. Joe Barton (R-Texas)

  • Rep. Erik Paulsen (R-Minn.)

    "It is unconscionable that the IRS deliberately targeted individuals based on their political beliefs. Absolutely no one should come under extra scrutiny from the IRS because of their political affiliation. It’s simply un-American," Paulsen said in a <a href="http://paulsen.house.gov/press-releases/paulsen-statement-on-fridays-hearing-examining-irs-targeting-conservative-groups/" target="_blank">statement</a>. (Photo by Daniel Acker/Bloomberg via Getty Images)

  • Rep. Kenny Marchant (R-Texas)

    “I have long been concerned with reports that the IRS has unfairly targeted some political groups over others – a charge that they have repeatedly denied. In March 2012, I sent IRS Commissioner Doug Shulman a letter demanding an explanation of this unacceptable behavior. Now, more than a year later, the IRS has admitted to what we have long suspected – it was targeting tea party groups. The IRS’s actions are unacceptable, and I commend Chairman Dave Camp and Oversight Subcommittee Chairman Charles Boustany for moving forward with a full investigation. We will continue to work to ensure there are protections in place so no American, regardless of political affiliation, has their right to free speech threatened by the IRS," Marchant said in a <a href="http://marchant.house.gov/news/documentsingle.aspx?DocumentID=333635" target="_blank">statement</a>. (Photo By Chris Maddaloni/CQ Roll Call)

  • Rep. Jason Chaffetz (R-Utah)

  • Rep. Ed Markey (D-Mass.)

  • Rep. Roy Blunt (R-Mo.)

  • Rep. Bob Goodlatte (R-Va.)

  • Rep. Tom Price (R-Ga.)

  • Sen. Mark Udall (D-Colo.)

    <i>CORRECTION: An earlier version of this slideshow incorrectly identified Udall as a Republican.</i>

  • Rep. Jack Kingston (R-Ga.)

  • Rep. Aaron Schock (R-Ill.)

  • Sen. John Cornyn (R-Texas)

    “Today’s revelation that the IRS targeted average Americans using taxpayer dollars solely for disagreeing with them politically is completely unacceptable from this Administration. “Partisan politics have consistently characterized this White House, and the Administration must take immediate disciplinary action and ensure American citizens are not subject to this type of Orwellian persecution again," Cornyn said in a <a href="http://www.cornyn.senate.gov/public/index.cfm?p=NewsReleases&ContentRecord_id=ebc9edeb-f748-45cf-a6b3-0143c6cb41c0" target="_blank">statement</a>. (AP Photo/J. Scott Applewhite)