Casual Male Retail Group Inc on Thursday said it plans to speed up the closure of Levi's and Dockers Outlet stores and transfer the operations of its remaining Candie's outlet stores to Candie's Inc.