Plus-size menswear retailer Casual Male Retail Group Inc has suffered a first quarter net loss of $5.1 million, widened from a loss of $2.8m in the same period of fiscal 2003.