Golf apparel maker Sport-Haley Inc has swung to a fourth quarter loss after Wal-Mart stopped selling its Top-Flite branded apparel line.