DH@UVA
The University of Virginia's new Digital Humanities program had a website that needed to be tested to be sure it was aligning with users' goals and needs.
Usability Testing
Design Process
It was important to gain an understanding of the stakeholders' goals and define them in order to create our study and research problems they were experiencing.
-
1
Understand
We conducted stakeholder interviews to gauge their understanding of the problems they were facing and the goals they wanted to achieve with the tests.
-
2
Define
We had to define the problem we were addressing before defining our objectives of the study and the users.
-
3
Ideate
This is where we created the usability tests.
-
4
Validate
We recruited participants, conducting the tests, then consolidated the findings and provided recommendations to the stakeholders.
1. Understand
Understanding the Stakeholder’s Purpose Through Stakeholder Interviews
Stakeholder Interviews
Asking the rights questions allowed us to narrow down the main functions of the site and get a feel for why they wanted the usability study in the first place.
These sorts of questions allowed us to narrow it down to the cause: students were asking faculty about the requirements and the application process rather than going to the website to find out for themselves.
Understanding our client’s purpose for the usability tests was the first step in creating the tasks and it required us to dig deeper than our client’s answer of “I just want it to be better”.
2. Define
Learning to define and narrow down the focus of the study makes it easier to create actionable results.
The Problem
Students are asking faculty about the requirements and the application process rather than going to the website to find out for themselves.
Research Objectives
Getting a grip on the underlying cause allowed us to create a set of goals or objectives for the study which set the foundation for the tasks to follow.
Identifying the Users
This can be quite difficult for a site that hasn’t officially been developed yet. They had a wide range of users that this certificate program could apply to so narrowing it down was essential. We asked who this program would be most appealing to in its early stages and who most likely would sign up and go through the application process? We were then able to reduce the user types for this test down to this:
Graduate students at UVA who are interested in DH & the Grad Certificate in particular.
3. Ideate
As a student, I helped create tasks that would be relevant for a student when trying to learn more about a program and the process of applying.
Creating the Test
The test was sent to the stakeholders to confirm that it aligned with their goals and to receive any feedback before testing.
When creating the tasks, it was important to not only see how a participant completes a task, but also their thought process behind completing the task. The questions with the grey highlight are tasks that the participants were requested to complete that were given on slips of paper. The non highlighted tasks were questions that we asked them.
4. Validate
This is where we recruited the participants, conducted the test, and came up with recommendations.
Participant Recruitment
Since our users were such a niche group, we could not use our email list to find participants and do gorilla testing. I realized that being able to utilize other methods of recruiting participants, as opposed to what you are used to, is something you may face when conducting research.
After asking the stakeholders for a list of all graduate students who noted an interest in the program, we recruited 5 participants from various graduate degree programs. When you have more than 5 participants for a study you start to see diminishing returns so that is why we selected this number.
Usability Testing
Challenge: the participants had varying knowledge of the program and they knew answers to questions despite not locating the information on the screen.
This obviously doesn’t assess the web page so we had the participant explain out loud their thought processes for completing the task as if they had never visited the site before.
Findings and Recommendations
New DH@UVA Website
Our recommendations were given to the stakeholders in which they were able to improve their site as they pleased.
What I Learned
-
Dig deep into the stakeholders in order to determine the main issues with their site and the causes
-
Having a successful stakeholder meeting makes it easy to create tasks and compare them to your goals
-
Sometimes you have to look into other methods to recruit participants from what you’re used to.
-
When conducting a test, understanding the participants thought processes is crucial for analyzing the site for improvement.
-
If possible, get the stakeholders involved in the testing to see how necessary the changes are