Retailer Men's Wearhouse unveiled plans to close its Golden Brand manufacturing facility after announcing a profit nosedive in the company's fourth quarter.