The buyer of UK suit and formalwear specialist Moss Bros has had its proposal to pull out of the sale rejected by the Takeover Panel.