Australia-based fashion company Pacific Brands swung into loss over the financial year as it saw sales decline and was hit by a series of impairment and restructuring costs.