Larry Johnson, president of apparel producer Oxford Industries' Oxford Slacks division, is to retire at the end of January.