Udemy
Building an Internal Tool to Manage Taxonomy
Role: Designed the full experience, including journey mapping, wireframing, prototyping, and working closely with engineering to ensure quality.
Team: Taxonomist, PM, Engineering
*for the purposes of confidentiality, all data has been replaced with placeholder information
The Challenge:
Build a tool to manage Udemy’s structured data
In early 2018, we began introducing taxonomy, or “structured data”, to Udemy by creating topics. All courses uploaded onto the site would be tagged with a main topic, and our in-house taxonomy team would review them 1-by-1 and adjust or approve as needed. At the time, we had a small taxonomy team, led by Marni, who was in charge of tracking the differences between 2000 different topics, managing the topic canon, and also going through a backlog of 7,500+ published courses that still didn’t have topics.
The taxonomy team was getting crushed by inefficiencies from her current workflow, and needed a more specialized tool that would help her accomplish the minimum tasks. After numerous conversations with the executive team, our team decided to build a new taxonomy management tool in-house to allow for more specific features and needs for our team.
The Goal
Help the taxonomy team do their work more efficiently.
Additional jobs-to-be-done:
Manage our evolving taxonomy & ontology
Find and tag courses with different “nodes” (i.e. topics, categories, subcategories, occupations, etc) quickly
Keep the topic canon up-to-date
The Approach
Fast, Cheap, and Engineering Forward
The bulk of the work fell on the shoulders of the engineering team, who were building the entire system from scratch. Because of this, we knew that we had to build out the tool in different stages, with engineering setting the pace.
Phase #1 had two objectives:
Show all courses that need to be tagged with a topic
Manage the list of all current topics.
We had 3 months to build out all the features for Phase 1, and leverage as many existing components as we could.
Discovery
Building out a feature list
I worked closely with Marni over multiple sessions to understand more about the problems she was facing and her needs. The first step was to understand how courses are tagged at Udemy — a massive manual effort, where taxonomists go through each course individually to determine the appropriate topic for each course.
We went through her entire workflow, and created a journey map to understand her specific pain points, surface open questions and identify opportunities to streamline.
We also did competitive research on existing taxonomy tools — including PoolParty, Synaptica, and Stanford’s Protoge Tool — as we were creating our preliminary list of user stories. From here, we met with engineering and product to review the journey map and a list of prioritized user stories for Phase #1.
Establishing Design Principles
Throughout the project, there were frequent conflicts between engineering’s and product’s visions of the tool. Product wanted to add more features to take advantage of the data; engineering wanted to optimize for performance by keeping the tool as lightweight as possible. Although having both would be ideal, we still had limited resources and frequent discussions were slowing us down.
To bring us back into alignment, I led a session with the full team to establish design principles and rank them in terms of priority. We referred back to this list any time there were conflicts about which features to build.
Process
Creating the Architecture
Udemy has two separate branches of structured data:
Categories/Subcategories (i.e. Business, Development, Languages)
Topics (C++, Herbalism, Instagram Marketing, Calligraphy)
Every course was tagged with a category/subcategory and also topics. I initially thought that topics could easily ladder up into categories, but courses with topics such as “SQL” could easily fit into any of the categories “Business”, “Data Science” or “Marketing”. It was dependent course-by-course.
Our earliest challenge was to create the architecture for the tool. Marni and I experimented with different ways of organizing information before settling on using 4 major sections, based on differences in uses. “Course Classification” is for tagging courses with different nodes, “Nodes” is primarily for managing different types of tags, and the final two — “Relationships” and “View” were placeholders for future features as the system became more sophisticated.
Building a Page Structure
Fundamentally, we knew each page had similar actions:
Looking through a list of topics, categories, courses, etc
Take more actions on details of each list item (i.e. description, published status, SEO tags, localization, etc.)
Search and filter within the list— for courses this meant any keywords in their titles
Marni wanted to be able to switch quickly between viewing the full list, to skipping from individual list item to list item, and viewing details on each item. We prototyped a panel that would slide in and out that contained all metadata details — this would allow Marni to be able to zoom in and out with what she needed. Our main points of inspiration were Square Dashboard, Amplitude, and Shopify.
Modal Details
Within each modal, we needed to figure out how to organize all the pieces of metadata. The goals of each modal were slightly different.
For classification, Marni needed to quickly scan basic information about a course to be able to determine which topics should belong. Based on her normal workflow, we worked together to determine her To reduce time, we pulled all basic information from an instructor’s landing page into the modal so that Marni would have all the information she needed in front of her.
For managing topics, the goal was slightly different — once a topic was created, this information served mainly as reference. Additional information would be populated automatically. We again used tabs to separate and prioritize information.
Searching within Filters
The Classification page needed two levels of search: one at the course level, and a second within filters. For example, Marni wanted to be able to search for all courses with “Photography” in their titles, and then filter the results by including or excluding certain topics — e.g. keep all results with the topic “Landscape Photography”, but don’t show all results that have “Night-time Photography”.
At the time, we had a relatively short list of filters that could be applied, so we borrowed the filter pattern from our marketplace product, and focused on the topic filter.
Explore the Prototype
Outcome
The structured data tool was launched officially in August 2018! As our taxonomy team has grown, we’ve continued listening to their feedback and tackling more problems as they have arisen. As structured data has grown in influence, multiple teams unexpectedly have descended on the tool as a way to quickly search through our catalog and see the different metadata attached to each course.
Some later projects:
Bulk edit — Assigning the same topic to multiple courses
Testing automatic-tagging algorithms — Evaluating similarities between taxonomy team’s topics and algorithmically generated topics
Permissions (in progress!)— Ensuring the right people have the right power.
Expanding filters (in progress!) — As we add more users with different needs, the current structure of filters has gotten unwieldy.