Specialist apparel company Carter's fell short of expectations with its fourth quarter results, blaming wholesale markdowns and slow-moving inventories.