Footwear retailer and wholesaler Brown Shoe Company has swung to a fourth quarter loss after being hit by lower sales and restructuring charges in what it described as "one of the most challenging quarters" in its 130-year history.