Discussion Guide - Hi-Fi Usability test

Discussion Guide / Usability test HI-FI

Research Objectives

  • Have users connect with other fans and artist before the show

  • Have the user experience the concert with the intersection available

  • Have the user connect with fans after the show.

Evaluative test:

I’d like to begin by thanking you for making time to speak with us. Your feedback is valuable, and will help us determine if our website functions as intended. Just to confirm, we’d like to keep this session to 30 minutes. Does that still work for you?

Great. If you need a break or to stop at any time, please let me know.

During this session, I’ll start by asking you questions around your work. Later on, I’ll ask you to share your screen and accomplish a few tasks using the website we’re evaluating.

Please be aware that there are no wrong answers. In fact, this is probably the one place today where you don’t have to worry about making mistakes! As you go about using the app, I’ll ask you to think aloud as much as possible: to describe what you’re looking at and what you’re trying to do. This will be a big help.

Also, please don’t worry that you’re going to hurt our feelings. We’re doing this to improve the site, so we need to hear your honest reactions.

If you have any questions as we go along, just ask. I may not be able to answer them right away, since we’re interested in how people do when they don’t have someone sitting next to them to help. But if you still have any questions when we’re done I’ll try to answer them then.

With your permission, I’d like to record this call. The recording will only be used to help us figure out how to improve the app, and it won’t be seen by anyone except those with a need-to-know. Recording this call also helps me, because I don’t have to take as many notes! Do I have your permission to record this call?

Finally, I want to confirm that you’ve received and had a chance to review the participant agreement? Did you have any questions? If you have signed the agreement, please send me a copy. Otherwise, do you agree to the terms of the participant agreement?

Great. Do you have any questions for me at this time?

Warm up

Before we start to navigate on the app , I'd like to ask you a couple of questions:

  1. Tell me about something you can find on virtual live show apps that you enjoy?

  2. Why do you enjoy this?

Setup screen sharing

OK, great. We’re done with the questions, and we can start looking at things. The first thing I’d like you to do is share your screen. You can do that by clicking “Share screen” in the bottom-middle of our video chat. Then select the screen and click on “share”. Before you share, make sure to close anything you don’t want recorded.

Great. Next I’d like you to click on the link that I posted via chat. To start, please look at this page and tell me what you make of it: What are your first impressions? What is this app for? You can scroll if you want to, but don’t click on anything just yet.

Thanks. Now I’m going to ask you to respond to the following few scenarios. I’m going to read each scenario out loud, and I’ll also share the text of the scenario via chat.

“ You bought a ticket from a new app for a virtual live show by your favorite singer. To access a valuable part of the site you had to wait for the arrival of a wristband. Finally, it arrived!”

Task 1: This first part of the questions is about a pre show phase. " With your wristband in your hands you have to scan it to get the first access" .

  • Could you show me how you would scan it?

  • Could you show me how you would interact with other fans pre show?

  • Could you show me how you would interact with the artist pre show?

  • Could you show me where you can find a place to schedule foods and drinks?

Questions after task 1:

  • Could you tell me how you felt during this process?

  • Is any part of this process unnecessary for you? If yes, can you explain what part and why?

Task 2: Now we are in the second part that is about the day of the show, or to be more precise is actually the time that the show is happening. "The big day has just arrived and finally you are going to watch the virtual live show of one of the artists that you love.

  • Could you show me how would you access the show?

  • Could you show me what kind of interactions would you experience during the show?

Questions after task 2:

  • Could you tell me how you felt during this process?

  • Could you tell me what would be one thing that you like most? why?

  • Could you tell me what would be one thing that you like least? why?

  • Could you tell me who will be playing at 5:00pm?

Task 3: The third and last part is all about past shows. "The show has passed and now you feel the need to continue sharing your experience with other fans".

  • Could you show me where would you find your memories after the show or past shows?

Questions after task 3:

  • Could you tell me how you felt during this whole process?

  • Is this experience what you expected? If not what experience have you expected?

  • Is it the app that you expected? If not why?

  • How do you think you can improve it?

  • Can you look at the colors and fonts and tell me what you think about?

Great, we’re finished with the bulk of the test.

Would you be open to speaking with us again in the future?

Do you have any questions for me, now that we’re done?

Give them their incentive, or remind them it will be sent to them

Hi-Fi Usability Report

Executive Summary

The purpose of this test is to see how the user would access a concert using the new and improved Live from Home app. This test helps us define more paint points we missed in the previous gray scale test since there is a lot more detail in this Hi-Fi prototype.

Key Findings

1. Access Concert with wrist band

Users need a form to have a festival experience during virtual festivals in the real world

2. Navigation Needs work

Users thought it would be better to have a different form of navigation (Add home screen)

3. Users liked how the layout worked

Users felt the layout of the app was great it just needed better copy to better follow it

Methodology

Outreach & Screening

The user testing was done through zoom. They all took it on a computer. A total of 3 users were tested on the Hi-Fi Prototype.

Participant Profiles

Number of Users - 3

Date of Test - Nov. 20, 2020

Age of Users - 20’s - 30’s

Testing Setup, Moderation & Post-Task Questions

Goal of the user test (Hi-Fi)

  1. Have users connect with other fans and artist before the show

  2. Have the user experience the concert with the intersection available

  3. Have the user connect with fans after the show.

Task One

This first part of the questions is about a pre show phase. " With your wristband in your hands you have to scan it to get the first access" .

Task Two

Now we are in the second part that is about the day of the show, or to be more precise is actually the time that the show is happening. "The big day has just arrived and finally you are going to watch the virtual live show of one of the artists that you love.

Task Three

Task 3: The third and last part is all about past shows. "The show has passed and now you feel the need to continue sharing your experience with other fans".

Results

Task One

User is trying to access the concert with the ‘wrist band’ they have ‘at home’

What’s working What’s not working

They understand that proceeding allows them into the ‘waiting room’. Having to wait for the show after accessing it with wristband

Copy is good Users aren’t sure exactly what activate means

Task Two

Users are trying to get to watch the virtual show and  are able to play around with the ‘waiting  room’ screen.

What’s working What’s not working

Having different activities before the show ‘fest friends’ ‘chat with artist’ etc The fact that the countdown is so far in advance

Size of font

Task Three

User is at the concert is checking what works well with the concert viewing experience.

What’s working What’s not working

User can add comments There is no where for memories

User can put reactions Not necessary to have reactions at all time

Additional Takeaways

What’s Working What’s not working

The color work well together The Copy needs better work 

The layout of the app is well done Navigation needs to be more clear

User isn’t always sure where to click

Recommendations & Actions

  1. Need to have a better way to show users what is clickable

  2. Need to change font size

  3. Needs to specify better the navigation’s icons.