The Pratt Libraries A-Z Databases is a research tool that provides students at Pratt Institute access to a wide range of databases. The website offers functions to browse, search, and filter its database collection to find content related to students’ research interest. This project uses remote moderated user testing to find usability issues on the beta version of the website and provide possible solutions
Usability Testing
User Research
UX Design
Content Strategy
Figma
FigJam
Google Workspace
Springshare A-Z List
Marko Stanisic
Indrani Thool
DURATION6 weeks
A-Z Databases is an additional research tool offered to Pratt Institute students to facilitate academic research. Basically a database of databases, it allows students to browse, search, and filter through a large list of databases to find one that suits their needs. We started with a promising kick-off meeting with the client to define the scope of the study.
Pratt Libraries had received feedback that students haven’t been able to effectively utilize the A-Z Database functions. They had found that the website had low engagement, low retention of student research, and reduced returning users. While A-Z Databases are a powerful tool for academic research, it was not being utilized to its full research capabilities.
Pratt Libraries had just developed a beta version of this website when they approached our team. The client wanted to test how intuitive and navigable the website was, especially for students unfamiliar with this type of interface (Novice Users). Additionally, the client wondered if there was a way to increase student engagement with this website. Through preliminary conversations with our client, we identified the areas that needed review in the usability study.
By conducting research about A-Z Databases’ user behavior, collecting data, and analyzing recurring patterns, we found the following pain points for user experience. These insights were crucial in helping us create appropriate solutions.
100%
of the novice users misunderstood or were confused about the purpose & function of the A-Z Databases website & its features
87%
of the users were overwhelmed with design of the search & filter features & unsure how to proceed
50%
of the users thought there was not enough information/options to narrow their search to find relevant databases
90%
of the users acknowledged that the A-Z Database website offers access to helpful databases & resources
After identifying usability issues from client conversations and user research, we set goals improve the UI/UX of the website to increase search efficiency, reduce cognitive load, and enhance general usability, thus improving user engagement, exploration, and experience on the Pratt Libraries A-Z Databases website for research purposes. Our process is outlined below:
STEP 01: STRATEGIZE
- Understand all client expectations
- Define scope of usability test
- Develop a targeted, task-based test aligned with research goals
- Recruit participants
- Conduct moderated user interviews over Zoom
- Monitor and record task responses
- Develop solutions for identified problems
- Create high-fidelity prototypes for demonstration
- Communicate process, findings, and recommendations to client
- Compile and categorize participant responses
- Identify, record, and prioritize common issues
- Evaluate the System Usability Score
After conducting research, observed issues were identified, recorded, and prioritized. A total of 26 usability problems were identified with the interface. We prioritized issues based on two main factors: the severity of the problem and the frequency of occurrence among participants. These major issues were then selected for further examination.
To address these usability problems, we developed a series of recommendations outlined below. Each of the recommendations addresses multiple of the 26 usability problems we found. They are designed to provide clear guidance for the Pratt Libraries team, enabling them to effectively solve the identified issues and improve the overall user experience of their database website.
To achieve our set goals by the end of our design process, we created high-fidelity prototypes to demonstrate our proposed solutions to problems identified through the research process.
01. PROVIDE ADDITIONAL CONTEXT
Adding description and context for the function of A-Z Databases was key in providing clarification and alleviating confusion in users.
Providing rationales for curated database lists to increase trust between users and the interface.
02. OPTIMIZE SEARCH & FILTERS
Creating a streamlined search experience by separating A-Z search unit from Pratt’s search unit, adding context to each search category, and making it more comprehensive in a glance.
Additionally, we added checkboxes to indicate selected options, categorized inputs within filter for efficiency, and added a Clear Filter option when filters are applied.
03. PRIORITIZE ACCESS TO RESOURCES
We decided to reorganize content on the website and move the Subject Resources feature to the top of the page for high visibility, since users thought it was the most helpful feature. Additionally, we reimplemented the list of Subject Guides for easy access.
This solution also included reducing the content on the side panel to a more manageable cognitive load by employing an expansion/contraction feature for curated lists.
Implement a tagging system to provide more information about each database, allowing users to effectively evaluate relevant databases.
Moreover, database tags would enable users to retrace their database searches to relevant topics
05. CONTINUOUS UPDATE OF SEARCH RESULTS Users expected search results to dynamically update when applying or removing filters without the need for further clicks.
This will provide users with instantaneous and dynamic feedback as results update and make the search process more efficient.
To evaluate the usability of the Pratt Libraries A-Z Databases website, our team of four usability experts conducted Moderated Remote User Testing, which is widely considered the gold standard in usability research. Moderated testing allows for the highest assurance in identifying usability issues within a digital interface compared to other methods, such as cognitive walkthroughs and heuristic evaluations. Despite being the most time-consuming and costly method, the results of moderated user testing are invaluable for successfully restructuring any digital interface.
The process involved defining the scope of the usability test after the kickoff meeting. The team analyzed the website and streamed a scenario-based script for moderated usability testing. Overall insights were gathered, and issues were prioritized based on frequency, task completion, and success metrics. Throughout, we identified user behavior and brainstormed on developing design solutions that would best assist students in ease of navigation. The tasks used to test the interface are outlined below.
We recruited and conducted interviews with Pratt Institute Graduate students for this study. All participants shared a high level of comfort with digital technologies and had prior experience using library catalogs, with four self-identifying as seasoned users. Additionally, they were all familiar with the Pratt Libraries website. This user group was selected to ensure that the feedback and insights gathered during testing would be relevant to the core audience of the Pratt Libraries database website.
Half of the participants recruited were from the Information Experience Design Program and the other half were from the Library and Information Science Program. Because of this division, two user groups became apparent within our study:
NOVICE USERS
- DESIGN STUDENTS
- UNFAMILIAR WITH DATABASE WEBSITES
- EASILY OVERWHELMED BY THE COMPLEX & CONTENT HEAVY INFRASTRUCTURES
- NEEDS TIME & HELP TO ENGAGE & INTERACT WITH WEBSITE FEATURES
EXPERT USERS
- LIBRARY SCIENCE STUDENTS
- HAS INTIMATE KNOWLEDGE & PERSPECTIVE OF ACADEMIC RESEARCH FROM A RESOURCE MANAGEMENT POV
- FREQUENT USER OF COMPLEX & CONTENT-HEAVY INFRASTRUCTURES
- EXPERIENCED WITH LIBRARY & RESEARCH WEBSITES
Insights from the participant interviews were summarized, and most users thought the A-Z Databases was a beneficial resource. Most of the novice users weren’t aware of its functionality but, after completing the tasks, thought they would revisit the A-Z Databases.
Both user groups finished the tasks but incorporated alternative ways since they were sometimes confused. Overall, users thought the website was clean and easy, but some changes would help make the database website’s functions efficient and successful.
- EXPERT USER
- NOVICE USER
- EXPERT USER
- NOVICE USER
We gathered data during the post-test with the System Usability Scale (SUS), which led us to draw conclusions about the website's efficiency, intuitiveness, ease, and satisfaction. Interestingly, the novice users had a low score of 57.5%, while the expert users had a fairly high rating of 80.6%. The average rating of 69.1% is borderline between needing improvement and good.
This hypothesizes that the website needs to change more to cater to the needs of new/novice users first and then implement modifications suggested by expert users.
Based on the System Usability Scale Scores, we found that while novice users had many usability issues with the A-Z Databases interface, the expert user had less trouble navigating it. Therefore, while our main solutions geared to make the interface more usable for all users, there were some higher-level problems identified by the expert user that we provide solutions to below. These are low-priority usability solutions, but we believe they could be implemented in the future to further enhance the interface.
ADDITIONAL KEYWORDS/TAGS IN THE BACKENDWe believe adding additional tags to each database on the webpage's backend would help students generate their desired search results. Therefore, more keywords/tags should be added to databases on the backend to help users broaden their search results.
REIMPLEMENTING THE LIVE CHAT OPTIONWe recommend reimplementing the live chat option since users find it more convenient for consultation than emailing someone, which would be a deterrent.
OPTION FOR DARK MODE ON THE WEBSITE
We believe this could be implemented in the future to allow users to customize settings according to their preferences and make the most of the resource.
By implementing our proposed solutions into high-fidelity mock-ups, we created the final prototype seen below as a reference for Pratt Libraries to visualize the recommended changes.
Our findings and recommendations were met with positive feedback when presented to Pratt Libraries. The clients were inquisitive about the process with which we came our conclusions, and seemed excited to implement the changes we proposed.
They also mentioned that while some of our recommendations were things they had never thought of before, there were also recommendations that Pratt Libraries had considered implementing, but never had valid rationales to support them. Our study provided statistical support and validation to these ideas, and encouraged implementation. Moving forward, Pratt Libraries hopes to hire a UX Librarian and discuss potential implementations to the A-Z Databases website.
This project provided a unique view into user groups and how different their interactions with an interface can be. I learned how to make full use of the System Usability Scale, which helped us analyze and communicate the significant difference between Novice and Expert Users, and their relationships to the interface.
Synthesis & Communication
Synthesizing a huge project into a report and presentation for the client was difficult, especially since we had a very detailed process and so many findings. However, the process of creating those final deliverables allowed me to reflect upon our project, and understand how to identify and prioritize significantly impactful findings that would be beneficial to communicate to our clients.
Confidence
Our team worked end-to-end on this project: developing strategy, conducting user testing, data collection, analysis, prototyping, and communication. This process helped me assess and validate the skills I bring to the table as a collaborator and an individual contributor.
As part of the project, our team wrote a usability
report that details our strategy, process, findings,
and recommendations thoroughly, as a reference
for Pratt Libraries.
VIEW REPORT
At the end of our usability study, we presented our process and findings to Pratt Libraries, and held an open session for brainstorming or answering any questions they had.
VIEW PRESENTATION