Double-digit sales growth and expanding margins helped Delta Apparel to record a strong increase in first quarter profit.