Weavers of the iconic Harris Tweed fabric have voiced their concern over plans to limit production to just five patterns and restrict its use to men's jackets.