Remember, you are testing the software or the website, not the user
Make this clear to the participants. They will point out things they don’t like. Expect them to do things you didn’t plan for. After all, that is why you are taking the time to get their input. Realize that they may hate features that you designed. If they perceive any rejection or evaluation of their opinion, you will have lost them for the rest of the session.
Expect to go through several iterations of testing and fixing
If you test early (with paper drawings and users’ imagination), making iterations will be easier. Track the problems at each stage as well as the changes made. Some changes will have unexpected consequences. Keep good records. The earlier you discover an issue (good or bad), the easier it is to make the software better. This means that the development team needs to be on board before you begin. At least one developer should be involved in the testing.
Determine the makeup of your desired user group
Are they experienced with "X"? Do they all come from a certain background or work environment? Try to have a mix of testers with many that are similar and some that are different from your expected users. Keep track of the characteristics of the users so that later you can make correlations like, “The testers over fifty had trouble reading the blue on grey text." If you can make your site work for everyone, you will have a better chance for larger acceptance. And, if you have a few important users, you can make sure that they quickly find what they need.
Create your tasks for the testing sessions
A test scenario should explore the user experience rather than the features of the software. Have a set script that doesn’t change between participants. Don’t guide the user. You can explain the scenario, but don’t help them with the software. The words you use for the scenario should not be clues to named features in the software. Start with general questions, then transition into greater detail. Spend more time watching and taking notes than talking. You want to find out what the user does when you are not there.
Make the user comfortable before the test begins
You want the experience to be as easy as when they go to the website from their home. Don’t have more than one or two observers in the room. It is hard to be natural with people watching over your shoulder.
Keep the testing short
The interaction with the software should be no longer than about twenty minutes. Most people get tired after that and their feedback becomes less informative.
Let the user do the work
After setting up whatever starting scenario you want, don’t touch the mouse or keyboard. The point of the testing is to see what a user might do. You won’t be there to help the users after the site goes live, so now is your chance to figure out how to make the software help them.
Encourage the user to describe what they are thinking at each step
It is not enough to know that they clicked a certain location. You need to know why they thought that location would work. Maybe they clicked on the search box because it was in the upper right rather than because it had the word “Search” beside it. By knowing that, as you change the layout, you will know what to keep (the location) rather than accidentally breaking a functionality that you didn’t understand.
Review the recording and make notes while the session is fresh
Editing the recordings is usually for reports and presentations rather than evaluating the software. Clips can be extracted to give examples of issues for people that weren’t present in the testing session. Remember to preserve people’s privacy and treat them with respect
Review your notes and the recordings again right after the entire round of testing is finished
Make a list of things the users did. Include what they liked and didn’t like. Count the number of people, their user group and number of times each thing came up in testing. This lets you focus on the issues that will have the greatest impact.
After you have a list of issues, talk to the software developers to find out what changes could possibly fix each issue
Find out which are easy or hard to implement and how well the change might affect each issue. Listen to the developers, listen to the users and then balance the needs of the users with the difficulty of development.
Here are a few useful tips for usability testing.