Specialty apparel chain New York & Co swung to a loss during the first quarter, weighed down by weaker comparisons on last year.