Men's apparel retailer Men's Wearhouse has said former chairman George Zimmer was terminated from his position because he wanted to gain control and take the company private.