Pat Handley, the chairman of apparel and footwear maker Pacific Brands, is to retire after seven years in the post.