At the FOSDEM (Free and Open Source Software Developers Meeting) conference in Brussels, two openSuse developers presented their research on usability of KDE desktops.
Their testing methods consists of interviews, questionnaires, screen recordings plus precise video documentation of a group of 10 people trying to accomplish 12 tasks using various desktop systems (KDE classic, KDE reloaded and Windows Vista). Besides looking for 'succes rate' and 'accomplishment time', they also measured 'the hedonic aspect' which I thought was pretty interesting. The AttrakDiff standard (which is in itself a proprietary method ;-)) measures 'pleasure' i.e. the joy of using a system. Instead of asking: 'Did the interface do what you expected', it tries to find out whether it was an interesting experience. Which of course could also mean, that the system did the opposite of what you thought it would do.