Hong Kong-listed casual fashion brand Esprit has blamed the challenging operating environment for its decline in first-half sales and earnings.