Activewear maker Delta Apparel Inc has swung to a third quarter loss, blaming a combination of restructuring costs, higher raw material, energy and transportation costs coupled with lower sales.