WASHINGTON -- A U.S. congressman who visited garment factories in Bangladesh earlier this week had harsh words for the U.S. clothing brands that have declined to sign on to a legally binding safety agreement for the country's factories.

"I've seen what this industry has driven, the abuses visited upon these workers all over the world," Rep. George Miller (D-Calif.) said on a call with reporters Friday. "If Walmart and Gap want to stand alongside collapsing factories and burning factories and women jumping out of buildings, I guess that's their choice. But I don't think they should be allowed to do business in Bangladesh."

Miller spent Memorial Day weekend in the small country that's suddenly become the third largest exporter of garments in the world. Having visited with workers, manufacturers and government officials, Miller said they all seemed to agree they are at a "defining moment" for the country to implement much-needed workplace protections after not only the Rana Plaza building collapse last month that claimed more than 1,100 lives, but the Tazreen Fashions factory fire that killed at least 112 people in November.

In Miller's estimation, that sense of urgency hasn't reached most of the U.S. retailers that source their clothes from Bangladesh.

While a number of American brands have pledged to help improve safety in overseas factories, only Abercrombie & Fitch, Sean John Clothing and PVH, owner of the Tommy Hilfiger and Calvin Klein labels, have agreed to sign on to the Accord on Fire and Building Safety in Bangladesh, which will legally require companies to help underwrite safety improvements at dangerous facilities. So far, that accord has 40 signatories, the majority of them European.

With U.S. companies apparently worried about legal liabilities in the accord, Walmart and Gap jointly announced Thursday that they would be developing their own independent safety plan for factories in Bangladesh. The companies said they will spend the coming 30 days collaborating with other companies and trade groups to develop "a single, unified action plan" to address the root causes of the workplace disasters that have drawn worldwide attention.

Miller, however, argued that non-binding self-enforcement by U.S. firms has done little to curtail a string of factory catastrophes in Bangladesh and elsewhere.

"They're off on their own again," Miller said. "They have historically tried to do self-regulation. The fact is, they've led this race to the bottom over many years that's led to this dangerous and deadly situation. We'll wait to see what they come up with, but it's disappointing we can't have a unified effort around a transparent, enforceable and resourced effort."

"What good would this agreement be if it wasn't binding?" Miller added of the fire and safety agreement. "If it's not enforceable, not binding, then it's just what we have today: We have statements."

Walmart spokesman Kevin Gardner said in response that the company has taken "a number of actions that meet or exceed other factory safety proposals."

"These include strengthening safety standards for factories, a zero-tolerance policy for unauthorized subcontracting, increased transparency, and requiring that in-depth safety audits and remediations be made to every factory directly producing product for us in Bangladesh, reflected in the cost of the goods that we buy," Gardner said.

Bill Chandler, Gap's vice president of global corporate affairs, said in a statement Friday that the company hopes the program it develops with Walmart and others will bring "long-lasting change for the garment industry in Bangladesh."

"We believe the American alliance can be a powerful path forward to achieve lasting change in Bangladesh, and will build upon the work that is already underway. We will focus our energies on participating in a way that helps this effort be successful."

Although hardly any U.S. companies have joined the binding agreement, Walmart and Gap in particular have been pressured by labor groups and U.S. consumers, given their stature in the U.S. market and their implication in earlier tragedies. (A Walmart supplier had been sourcing clothes at Tazreen without authorization, the company later said, and Gap had clothes supplied from the Hameem factory in Bangladesh where more than 20 workers died in 2010.)

Given the Bangladesh government's failure to adequately monitor the country's buildings, some labor groups are now trying to pressure the White House to revoke Bangladesh's preferential trade status. Some State Department officials, however, are concerned that such a move could damage diplomatic ties with the country, according to The New York Times.

Miller said Friday that if Bangladesh cannot quickly improve its record on labor rights and safety, it doesn't deserve the breaks it currently receives from the U.S. on tariffs under the World Trade Organization's Generalized System of Preferences. U.S. officials are now considering revoking that status.

"When we're giving [trade] preferences, we clearly cannot support a system that does not go to the future," Miller said. "I hope the administration would recognize this is a matter of international human rights."

While noting that Bangladesh has "dragged its feet on labor protections" and allowed dangerous conditions to persist, Miller argued that the buying practices of Western brands and retailers were one of the main contributors to the tragedies, as manufacturers are pressured to cut costs wherever possible.

"I think there's universal recognition that these problems begin with the system as it's designed by the major retailers and brands," Miller said. "They designed this system to put pressure on manufacturers for the lowest possible price ... I think it's very clear that whether you're Walmart or H&M or the Gap or [PVH] or any other brand, you have to make a decision now whether you want to have blood on your labels."

Also on HuffPost:

Loading Slideshow...
  • Walmart

    Walmart declined to join the Bangladesh safety accord, instead creating its <a href="http://www.huffingtonpost.com/2013/05/14/walmart-bangladesh-factory_n_3275756.html" target="_blank">own safety program to address factory working conditions</a> in the country. The program will help more than <a href="http://news.walmart.com/news-archive/2013/05/14/walmart-to-raise-inspection-standards-provide-full-transparency-safety-conditions-factories-bangladesh-supply-chain" target="_blank">17,000 workers in 34 factories in Bangladesh</a>, according to "new principles [that] actually go further than the accord," Walmart said in a statement.

  • Gap

    Retailer Gap will sign the safety accord if rules on <a href="http://www.huffingtonpost.com/2013/05/14/gap-bangladesh-safety-accord-decline_n_3271406.html?utm_hp_ref=business" target="_blank">"how disputes are resolved" are changed</a>, the company said in a statement. Gap says it does business with <a href="http://www.gapinc.com/content/gapinc/html/csr/bangladesh.html" target="_blank">78 factories in Bangladesh.</a>

  • Macy's

    Macy's has reportedly declined to sign onto the safety accord, instead electing to develop <a href="http://www.safety-reporter.com/articleview/17996-north-american-retailers-to-develop-bangladesh-safety-accord" target="_blank">its own safety standards for factories in Bangladesh, along with other retailers</a>, Canadian Safety Reporter reports.

  • Sears/Kmart

    Sears has not signed the safety accord despite its "ongoing efforts to work collaboratively with other brands and retailers to improve working conditions in Bangladesh," the company wrote in an email to The Huffington Post. Instead, Sears says the company is working with retail trade associations to create an <a href="http://articles.chicagotribune.com/2013-05-15/business/ct-biz-0516-sears-bangladesh-20130516_1_safety-accord-industriall-building-safety-agreement" target="_blank">"alternate proposal"</a> for improving factory safety.

  • JCPenney

    JCPenney has said it's working with the North American Bangladesh Worker Safety Working Group to improve conditions in <a href="http://www.businessweek.com/articles/2013-05-15/bangladesh-safety-accord-is-too-binding-for-american-retailers" target="_blank">Bangladesh, but the retailer has not signed the Accord on Fire and Building Safety in Bangladesh</a>, Businessweek reports. "Moving forward, we’ll strengthen our audit requirements related to structural and electrical inspections, and continue enforcing a policy that prohibits the use of factories located within multi-use buildings," JCPenney wrote in an email to The Huffington Post.

  • VF Corp

    VF Corp, which owns brands such as The North Face, Wrangler jeans and Vans, has not signed the accord. It wrote in an email to The Huffington Post that the company "has a long history of working in Bangladesh and intends to maintain operations in the country ... We are currently evaluating all agreements and programs aimed at addressing safety standards to determine the best opportunity to create effective change [for Bangladesh garment workers]."

  • Target

    Target has declined to join the safety accord, instead <a href="http://pressroom.target.com/news/targets-position-on-bangladesh" target="_blank">highlighting its involvement with the North American Bangladesh Worker Safety Working Group</a>, a third-party organization that will endorse its own <a href="http://www.washingtonpost.com/blogs/wonkblog/wp/2013/05/16/u-s-retailers-arent-signing-a-new-safety-accord-for-bangladesh-heres-why/" target="_blank">“broad set of [safety] proposals,”</a> according to The Washington Post.

  • Kohl's

    Despite <a href="http://www.calvert.com/newsArticle.html?article=20447&source=RSS%3A+All+Calvert+News" target="_blank">facing pressure to join</a>, Kohl's has yet to sign onto the safety accord, according to the Worker Rights Consortium.

  • Cato Fashions

    Despite receiving pressure to sign the accord from labor and consumer groups, <a href="http://www.nytimes.com/2013/05/14/business/global/hm-agrees-to-bangladesh-safety-plan.html?_r=0" target="_blank">Cato Fashions is yet to get on board with the agreement</a>, The New York Times reports.

  • Carter's

    OshKosh B'Gosh parent company Carter's has yet to sign the Bangladesh safety accord, says the Worker Rights Consortium.

  • Nordstrom

    In the wake of the Rana Plaza factory collapse in Bangladesh, Nordstrom reviewed its operations, affirming it <a href="http://seattletimes.com/html/businesstechnology/2020985594_nordstrommeetingxml.html" target="_blank">produces apparel at three factories in the country</a>. While saying it will work with the American Apparel and Footwear Association to improve worker conditions, Nordstrom has not signed on to the Bangladesh safety accord, the Seattle Times reports.

  • American Eagle Outfitters

    American Eagle is among retailers cited by the Worker Rights Consortium who have not yet signed on to the Bangladesh safety agreement.

  • The Children's Place

    Kids' retailer The Children's Place is reportedly still evaluating <a href="http://www.usatoday.com/story/money/business/2013/05/13/hm-sign-bangladesh-labor-agreement/2154991/" target="_blank">whether it will sign onto the safety accord</a>, USAToday reports.

  • Foot Locker

    Despite <a href="http://www.bloomberg.com/news/2012-12-05/wal-mart-nixed-paying-bangladesh-suppliers-to-fight-fire.html" target="_blank">links to Bangladesh manufacturing</a>, Foot Locker has yet to sign the Accord on Fire and Building Safety in Bangladesh, according to the Worker Rights Consortium.