Apparel maker Perry Ellis International said it is "extremely disappointed" with its fourth-quarter and full-year results as it cut its outlook on poor weather and a challenging retail environment.