Introduction to Multi-Class Text Classification
We live in a time where Artificial Intelligence can help us automate many tasks. One example is using embeddings to easily classify large amounts of text.
Simply put, embedding is a way to represent real-world things, like pictures or text, in a form that machine learning can use. Here, we turn conversations into insights, and those insights are shown as infographics.
In this article, we’ll explore insights from conversational text, focusing on subreddit posts and comments about hotels and related topics.
We will classify around 5,000 conversations about hotels and hospitality topics from various subreddits into four categories: Solution Requests, Pain & Anger, Advice Requests, and Money Talk. Using OpenAI embeddings and the GPT-4o-mini model, we’ll organize these conversations into their respective classes.
Here is the overall chapter based on the visualization and analysis we’ve working on.
- Distribution of Each Classes
- Distribution of Overlapping Classes
- Special Insights on Overlapping Classes
- Conclusion
Before diving in, let’s explore what classification is and how we can use it to organize conversations.
Classes of Conversation
Conversations are more than just words—they reveal emotions, needs, and opportunities. By sorting them into categories, we can uncover patterns and insights to make better decisions.
Before that, what are these classifications exactly? Here are a few explanations of each classification.
- Solution Requests: These are posts or comments where individuals actively seek tools, services, or specific solutions to address their challenges or achieve a goal.
Example:
Post: How much can I expect to pay for a hotel in Spain? I don't want to go to big cities or touristy areas because I'm going to improve my Spanish (currently upper intermediate so I don't want to go to a language school).
Response 1: You know they have websites with an accurate, up-to-date answer to this question, right? Just go to hostelz.com or hostelbookers.com so you don't end up with inaccurate guesses.
This conversation is a solution request that talks about finding the cost of hostels in non-touristy areas of Spain to improve Spanish language skills. The focus is on obtaining actionable information about hostel prices through reliable sources. - Pain & Anger: This category includes messages where people vent their frustrations, express dissatisfaction, or share struggles and pain points related to a situation.
Example:
Post: Cracks me up that u rate a hotel badly because of a check-in All that matters is the cleanliness of the room Not check-in not breakfast not neighbors JUST if the room has mold or bugs or not.
Response 1: OP I will disagree it all matters; how the guest are treated, rather we show value to them and recognize their value to us and so forth. If hotels do not want bad reviews they should not do bad things. The best hotels tend to have the best teams and provide the best most exceptional service and they generally rewarded for it, from the ownership to the team members from my experience.
This conversation highlights frustration with poor hotel service, including how guests are treated and the quality of their experience. The response stresses that every part of a guest's stay—service, recognition, and care—matters, as bad experiences often result in fair negative reviews. - Advice Requests: These are instances where individuals ask for guidance, suggestions, or resources to help them navigate a particular issue or make informed decisions.
Example:
Post: Fairly self explanatory question, I'm planning on booking a hotel in Japan for skiing this winter, however I turn 18 just a few weeks before departure and therefore won't be able to book the room with my own card.I'd have my own card with me to cover damages, and our last names etc obviously line up, but I won't have the card which was used to pay for the room with me to show at check-in. Would they deny me?
Response 1: Different hotels have different procedures, even within the same brand. You should contact the hotel directly. However, the simplest way to go about it is booking with your own card, since you will be of age at check-in.
In this conversation, the poster asks if not having the card used for booking might cause problems during hotel check-in, especially since they’ll have just turned 18 before the trip. They also seek advice on handling this situation smoothly. - Money Talk: Posts or discussions centered on financial matters, such as spending habits, budgeting, saving strategies, or the cost of goods and services.
Example:
Post: Is it true that japan has become cheap? And if so how cheap has it got ?
Response 1: Currently about 150 year to 1 US dollar. Normally it is closer to 110 yen to 1 dollar. So with prices being relatively static your currency exchanged goes farther.
In this conversation, the money talk focuses on the exchange rate between the Japanese yen and the US dollar. It highlights how the yen's depreciation makes traveling to Japan more affordable due to favorable currency rates.
Now, based on these examples, we want to see the distribution of each classes individually. Here is the classification distribution for the hotel subreddit conversation, visualized by pie chart.
We can see how dominating Solution Requests, Advice Requests, and Pain & Anger are on the conversation classification. But remember that each conversation can have multiple classes at the same time, so we are going to analyze the distribution of multi-label classification.
Visualizing Overlapping Classes in Conversation
A conversation usually contains more than one classification. In many cases, a single discussion can encompass multiple themes or intentions, making it essential to analyze and categorize it through a multi-label approach. This multi-label classification create a deeper context to be analyzed.
Let’s check on an example. Let’s say we have a conversation in the following form.
Title: A common line from the election cycle is $5 Gas. It was $3.19 by my house. What's it where you live?
Post: A little more context. I'm doing an East Coast road trip from Harrisburg to Atlanta to Orlando next week. Trying to find the most cost-effective places to fill up.
Response 1: I paid $2.80 the other day at the Costco in Charlotte, NC. If you're passing through, it's worth checking out—it usually has some of the cheapest prices around.
With the help of OpenAI embedding and model, we can classify these conversations into 1536-dimensional vectors and end with simple classification as follows:
Classification: Solution Requests, Money Talk.
This visualization, powered by LLMs and sentiment analysis, makes it easier to identify trends in recent conversations. Here is a visualization of multiple classes using a bar chart.
As we can see on the infographics, the top five of the infographics can be described as follows.
- The highest bar in this infographic is “Pain & Anger”, indicating that the conversation revolves around people expressing frustrations, dissatisfaction, or challenges they are facing, related to hotel problems.
- The second-highest bar in this infographic is “Advice Requests, Solution Requests”, indicating that the conversation focuses on the interaction between poster and commenter, seeking both guidance and specific solutions to address their needs or challenges.
- The third-highest bar in this infographic is “Advice Requests, Pain & Anger,” indicating that the conversation involves people expressing frustrations while also seeking advice or guidance to address their issues related to hotel problems.
- The fourth-highest bar in this infographic is “Advice Requests.” This is a more generalized form of other combinations, and the conversation focuses on posters seeking guidance or recommendations without necessarily expressing frustrations or discussing specific solutions.
- The fifth-highest bar in this infographic is “Money Talk, Solution Requests,” indicating that the conversation revolves around posters and commenters discussing financial aspects, such as costs or budgeting, while also seeking practical solutions to their concerns.
While there are other bars we need to check, they often have similar issues but with more dense in terms of specific nuances. These can be generalized based on the patterns observed in the first five categories, which we are going to discuss in the next part.
Advice Requests vs. Solution Requests: Navigating the Path from Inquiry to Answer
As we can see on the previous bar chart, one of the highest bars in the bar chart is conversations about Advice Requests and Solution Requests. One of the things we want to check is how the path of asking a question by the poster to an answer by the commenter.
Before we start with the path, we want to see how the connection between advice requests and solution requests. This Venn Diagram shows the connection between the two classes in our conversation data. Note that these Venn Diagrams count each of the classes. For example, “Advice Requests, Money Talk, and Solution Requests” are considered as the intersection between both.
We can see that we have Advice Requests dominating over Solution Requests in terms of count. But more importantly, we can see that most of the conversation has Advice Requests and Solution Requests as their classification, which brings us to the next point.
In the conversation, we can determine how the process of a conversation moves from a poster query to a commenter's answer. It can start as an Advice Request and end as a Solution Request. For example, A user reaches out to a customer support team asking, "What’s the best plan for someone who needs high-speed internet but is also affordable?" (Advice Request). As the conversation progresses and options are discussed, the user narrows it down with, "Can you tell me how to switch to Plan X and set it up?" (Solution Request).
We want to see how most conversation starts and end, either from Advice Requests or Solution Requests. This stacked bar explains how the distribution of the start and end of the conversation, considering the Advice Requests and Solution Requests.
From the stacked bar, we can see that.
- The highest path is a conversation started with Advice Requests and ends with Solution Requests.
Example:
Post: I'm only 20 so I can't rent a car. Apparently, you have to have a car so I guess something like Uber would be my only option but maybe someone has a better idea.
Response 1: Uber worked well for me. There might be public transportation but I didn't use it.
Response 2: Thanks! There isn't public transportation unless Google is lying.
Response 3: There is public transportation in the San Juan metro area, buses, trains, and even boats. Where in Puerto Rico will you'll be staying at?
This conversation starts as Advice Requests because the original post seeks general guidance and ideas for getting around without a rental car. The poster is open to suggestions, hoping someone might have a better solution than relying on Uber.
As the conversation progresses, Response 3 transitions the discussion into a Solution Request by providing specific details about public transportation options in Puerto Rico (buses, trains, and boats) and asking for more context about the poster's location. This shift narrows the focus from general advice to identifying and proposing actionable, location-specific solutions.
2. The second-highest path is a conversation that starts with solution requests and ends with advice requests.
Example:
Post: I have the option to pay $2,000 and fly direct from Seoul to Seattle. (11 hour flight) or pay $300 and fly Seoul-Qatar-Seattle with a 3-hour layover in Qatar. (30 hours including layover) This is also a standby ticket and I don't get to choose my seat.
Would the 30-hour flight be worth it?
Response 1: As a female I'd avoid Qatar airport like the plague after reading a story about all females being taken off a plane to check to see who had recently given birth to account for baby that was left in a bathroom.
This conversation starts as a Solution Request because the original post presents two specific travel options and explicitly asks for a judgment or recommendation on which option is better. The focus is on solving the dilemma of whether the cost savings of the longer, less convenient route outweigh the direct flight's convenience and higher cost.
However, Response 1 shifts the conversation into the realm of Advice Requests by introducing a broader concern about safety and comfort, particularly for women, at Qatar airport. This response isn't directly about choosing between the two flight options but instead offers a personal perspective based on external factors (airport safety), prompting the poster to seek further advice or reconsider their criteria for making the decision.
From these examples, we can create many ideas for the hospitality industry.
For instance, travelers debating between comfort and cost, or safety and convenience, reflect common challenges faced during trips. Hotels can use these insights to create tailored packages—such as offering shuttle services for convenience or ensuring accommodations prioritize safety and comfort for all guests.
Additionally, when customers share personal experiences, like concerns about specific destinations or transportation options, hotels can collaborate with local businesses to address these gaps, offering added value to travelers. These conversations not only highlight areas for improvement but also open doors for innovation, helping hotels position themselves as thoughtful and responsive to guest needs.
Conversations like these, where people weigh their options, seek advice, and share experiences, provide a goldmine of insights for businesses, especially in the hospitality industry. For hotels, understanding customer dilemmas and decision-making processes helps identify pain points and unmet needs.
Overlapping Categories: Advice, Solution, and Pain & Anger
Conversations often carry multiple layers of meaning, with overlaps between categories like Advice Requests, Solution Requests, and Pain & Anger. For instance, a user frustrated with a recurring issue might simultaneously seek advice on avoiding it in the future (Advice Request) and demand an immediate fix (Solution Request), all while expressing frustration (Pain & Anger).
First, we need to see the distribution between these classes. We can see it through Venn Diagram.
As we can see here, Pain & Anger doesn’t intercept that much with the other two classes. Here are some examples:
Post: I have an upcoming trip abroad and have booked one night through expedia with option to pay at property. How does this work if i need to cancel? I can no longer make it out and missed the cancellation window. The property would charge the whole booking for cancellations or no shows. Who collects the fee, the hotel or expedia? If they don't have a valid cc on file and i choose not to pay the cancellation, what would happen? Is there a chance the bill would be sent to collections in the US or other potential consequences? 've contacted the hotel to explain my situation and they were unwilling to make an exception to the cancellation fee, though we are weeks out. I know still my fault but just wondering the above. Edit: fully prepared to be banned from expedia and hotel. Wondering if there would be other repercussions beyond that.
Response 1: If the credit card you booked with is invalid, the hotel will notify Expedia. That's pretty much the end of it, no collections. Happens all the time.
Response 2: I'm not sure about the collections aspect, we typically send no-shows to collections if we can't charge the card on file. So far, collections have always been successful in recovering the full amount for the stay.
This conversation contains these classifications.
- Pain & Anger: The poster expresses frustration about missing the cancellation window and the lack of hotel flexibility.
- Advice Requests: The poster asks for guidance on who collects the fee and potential repercussions.
- Solution Requests: The responses provide actionable insights about invalid credit cards and collection processes.
This example the value of multi-class classification in capturing the full scope of user intent. Recognizing overlapping categories helps teams respond with empathy (acknowledge Pain & Anger), provide immediate resolutions (offer a Solution), and suggest preventive measures (share Advice). This approach not only resolves the immediate issue but also builds trust and improves the overall user experience.
Financial Conversations: The Role of Money Talk
Conversations about money often reveal more than just spending habits—they reflect the challenges, opportunities, and emotions tied to financial decisions. By analyzing the “Money Talk” category, we can see how it intersects with other themes like Pain & Anger, where frustrations about costs are shared, or Solution Requests, where people actively seek ways to save or make better financial choices.
The accompanying infographic, a Venn Diagram, visualizes these overlaps. It highlights how financial discussions blend with other categories, uncovering the deeper context of financial challenges or opportunities within the market.
The Venn Diagram reveals that the overlap between "Money Talk" and Solution Requests, is larger than Money Talk on its own. This indicates that financial discussions often extend beyond simple spending topics, encompassing ideas about solving financial challenges, sharing frustrations, or exploring opportunities. This overlap could serve as a valuable source of insights, offering innovative ideas and perspectives on managing money effectively or addressing financial concerns in creative ways.
Here are examples of these classifications:
Post: Okay, so this is my first time staying at a hotel. I booked the king suite at choice hotels back in March. I used uplift, to pay for the hotel room over time, and the hotel room got confirmed.
My question is this. Since uplift is considered third party, and my room was booked through it, so I just need to bring my credit card for them to charge the 100 usd security deposit, or will they try to charge me for the full hotel stay, plus the security deposit?
Sorry if this sounds confusing.
Response 1: I'm not personally familiar with uplift, but if it is a buy-over-time, and they have already charged your card, then correct, you only need to bring your CC for the $100/whatever security deposit.
Response 2: This may be obvious but also bring ID. 've seen posts about people being surprised by needing ID.
Response 3: To add further, a physical copy. I've been told some places accept pictures of IDs, but some properties, it's policy to have a physical ID on you in order to check in. I know the property I'm at, very few exceptions are made.
Here is the conversation classification.
Money Talk: The poster discusses payment methods, mentioning Uplift as a buy-over-time option, and asks about additional charges such as the $100 security deposit. The focus on financial aspects makes it relevant to this category.
Solution Requests: The poster seeks clarification on check-in requirements and payment processes at hotels, which is addressed by responses providing detailed steps, such as bringing a credit card, and ID, and ensuring it is a physical copy.
Conversations like these provide valuable insights into customer concerns and expectations, particularly for first-time guests. By understanding the questions and uncertainties travelers face regarding payments, deposits, and check-in requirements, hotels can improve their communication and services.
For example, guests frequently inquire about security deposits and payment processes, indicating a need for clear, upfront information. Hotels can use this knowledge to refine their booking confirmations and pre-arrival emails, explicitly detailing payment expectations, required documents, and policies for smoother check-ins. By paying attention to these types of conversations, businesses can better anticipate customer needs and provide a more seamless and satisfying experience.
Conclusion: Visualizing Insights Leads to a Powerful Tool
Visualizing the classification of conversations into categories like Solution Requests, Pain & Anger, and Advice Requests provides a powerful tool for businesses—especially in the hotel industry. To better illustrate this, let’s take a look at the stacked bar chart below, which breaks down emotional engagement into positive, negative, and neutral sentiments.
This visualization highlights key areas of concern, satisfaction, and neutrality, offering actionable insights. For instance, the prevalence of negative sentiment in Pain & Anger emphasizes the need to address recurring frustrations, while the balance of positive sentiment in Solution Requests shows the value of resolving customer issues effectively.
For businesses, this visualization offers actionable insights. Negative sentiment in Pain & Anger highlights areas requiring immediate attention, such as resolving common complaints or frustrations. Positive sentiment in Solution Requests demonstrates the impact of effectively addressing customer needs, which can be leveraged to enhance services, improve processes, and build stronger customer relationships. Neutral discussions, on the other hand, provide opportunities for proactive engagement and innovation.
For people in general, particularly travelers, this kind of conversation helps identify common issues like poor service or unclean rooms, allowing them to make more informed decisions when choosing hotels. Positive feedback inspires finding reliable services, while neutral sentiments highlight areas that might benefit from deeper research before booking.