Men's wear company Björn Borg recorded a 3% fall in full-year net profit after sales dropped 6% in the fourth quarter.