With the proliferation of opinionated content on the web, this particular application of natural language processing – Sentiment Analysis (Opinion Mining) is used to determine the sentiment of the given speech or text is gaining more popularity. In layman’s term sentiment analysis aims at identifying and classifying spoken words or written text as positive, negative or neutral.
What is Sentiment Analysis?
In the field of Natural language processing, sentiment analysis also known as opinion mining is task of classifying a given text as positive, negative or neutral based on the opinion or subjectivity of the text. Sentiment analysis can be considered a sub filed of text classification which involves classifying a text in to predefined categories. Sentiment analysis can be used to evaluate the attitude of the people about a topic, comments of Facebook posts, tweets, product review, political agenda, review sites, etc. Once important characteristics of sentiments and opinions is that unlike factual information, they are subjective. Examining the opinion of single individual reflects the subjective view point of that given individual which doesn’t hold much value. Thus sentiment analysis involves the analysis of opinions from a collective group of people to get some kind of summary of the opinions. Such opinion summary may be in the form of structured summary of short text summary which usually include the generalized opinions about different entities and have a quantitative perspective.
Since sentiment analysis mainly involves analyzing the opinions expressed in the text, it is important to discuss the different types of opinions. Broadly opinions can be categorized as regular opinion and comparative opinion.
- Regular opinion, often referenced to simply as an opinion in literature has two main sub-types known as direct opinion and indirect opinion. Any opinions which expresses a views bluntly to an entity or object can be categorized as direct opinion. For e.g. “The iPhone takes pictures of great quality”. An indirect opinion forms an opinion on an entity or object indirectly and usually references to an external entity for forming such opinion. For e.g. “After sitting in chair for ten hours, I suffered from severe back pain” explains the reason for the “back pain” and indirectly gives a negative opinion about sitting in chair.
- Comparative opinion expresses the preference of the opinion holder towards an entity after a comparative analysis of similarities and differences between that particular entity with other entities. Comparative or superlative form of an adjective is usually used to express a comparative opinion. “iPhones have better performance than android phones” is an example of a sentence with comparative opinion. Based on how the opinions are expressed, opinion can also be classified as implicit opinion and explicit opinion. Explicit opinion can give either a regular or comparative opinion and usually involves a subjective statement. For e.g. “iPhone is the best” and “iPhones have better performance than android phones”. Implicit opinion also known as implied opinion makes use of objective statement to imply a regular or comparative opinion. For e.g. “I bought an Android phone a week ago, it is already starting to lag every now and then”.
Applications of Sentiment Analysis
Opinions are an integral part of all human activities as they influence our behavior and play a major role in the decision making process. More often than not we tend to care about other people’s opinions when making a decision. In ancient days, people used to ask friends and families when the seek for opinions. Similarly, every time businesses and organizations needed some opinions, they conducted surveys, opinion polls, and focus groups. In fact, collecting public opinions and consumer view point has been a business of its own for marketing, public relations, and political campaign companies.
Today with the burgeoning increase in the use of web and increased accessibility of internet in many parts of the world, the amount of content generated in web such as review, forum discussions, micro-blogs, twitter, blogs, posts in social networking sites is growing at an explosive rate. Today, people seldom ask their friends or family for an opinion, they leverage the reviews and opinions available in the web. For businesses and organizations as well, they seldom need to conduct surveys and opinion polls in order to gather public opinions because they could gather such opinions from publicly available sources in internet such as review forums or micro blogs and social networking sites. However, collecting and monitoring opinionated content on the web and extracting concrete usable information from such content is still an arduous task. Because of the diverse nature of the sites and huge volume of opinionated text, it is nearly impossible for an average human reader to extract and summarize the opinions from such content available in the web. Thus, there is a necessity of automated sentiment analysis system.
Sentiment analysis can be used for information extraction by discarding the subjective information in the text and only leveraging the objective content. It can also help recognize opinion oriented questions and seek out in answering such question. In addition, sentiment analysis accounts for multiple viewpoints thus creating a summarization of the opinions. It can also be used form “flame” detection, bias identification in news, filtering inappropriate content for ad placement, filtering swear words and unsuitable video based on comments, etc. Applications of sentiment analysis can be noticed in every possible domain, from businesses and services, consumer products, healthcare, financial services to social events and political elections.
- Application of sentiment analysis in Business Intelligence – Businesses these days can hardly overlook the application of sentiment analysis. If leveraged properly, the information obtained from sentiment analysis can result in complete revitalization of brands. Based on the summarized opinion review obtained after sentiment analysis, businesses can estimate their customer retention rate, adjust to the present market situation and make plans to address the dissatisfaction of the customer. Sentiment analysis enables business to be more dynamic by helping them make immediate decisions with automated insights. Concept testing is very important in business to roll out any major change or introduce the next big idea. With sentiment analysis, business can easily roll out any new ideas such as new product, campaign, new logo, etc. for concept testing and analyze the sentiments attached to it. Since sentiment analysis can be applied to any piece of text, it is prudent for businesses to perform sentiment analysis on publicly available opinionated texts of their competitors as well. This enables businesses to understand what their competitors are doing better and come up with a plan to poach customers from their competitors. Sentiment analysis also gives insights on current customer trends enabling business to implement better strategies and gain a leading edge over their competitors. Having a good product or services is not always enough for business to build up their reputation and attract customers. Online marketing, social campaigning, content marketing and customer support plays and important role in making a business popular and retaining customers. Sentiment analysis takes in to account all these factors and quantifies the perception of the current and potential customers. With the knowledge of negative sentiments, business can make the required changes in their branding and marketing strategy to appeal more customers and enabling them to make a quick transition.
- Application of Sentiment Analysis in Politics – Sentiment analysis have been widely used by political campaigners to analyze trends and identify ideological bias. In political campaigns, gauging reactions of group of people and figuring out the suitable message or agenda for a specific target group of people is quintessential, sentiment analysis can be leveraged to do all those above mentioned tasks. Similarly, sentiment analysis helps political parties gain an insight of general public opinions on their policy and evaluate the public or voter’s opinion.
- Application of Sentiment Analysis in Sociology – In Sociology, idea propagation through groups is an important concept. Identifying opinions and reactions to ideas play an important role in deciding whether to adopt any new ideas or not. Sentiment analysis can be used to detect mass reactions to any new concept or ideas.
In a nutshell, human being are subjective creates. Opinions and sentiment hold greater importance to human beings and being able to interact with people on that level has some major significance to the information systems.
Challenges in Sentiment Analysis
Although sentiment analysis has lots of applications, it is challenging to build a state of the art sentiment detection tools because of many barriers. Sentiment analysis deals with the opinions expressed by people, and people use many complex ways to communicate their views. Also the context in which the opinion has been expressed also plays a significant role in determining the sentiment. For e.g. “My whole seller does an exceptional job when it comes to charging more money for the goods”. Taking the complexity and context of the expressed opinion in to consideration while performing sentiment analysis is very challenging. In addition, sentiment ambiguity and sarcasm are also very difficult to take in to account while performing sentiment analysis and may result in labeling the sentences in to incorrect group. Sometimes, a sentence with positive or negative words doesn’t necessarily mean it expresses such sentiment. For e.g. “can you recommend an awesome phone to buy?”. Although, the sentence has positive word “awesome”, it doesn’t express a positive sentiment. Similarly, some sentences can express sentiments although there are no sentiment words in the expressed sentence. For e.g. “This software makes use of lot of computer memory.”, holds negative sentiment although it doesn’t contain any sentiment words. Sarcasm is another factor which can alter the whole sentiment of the expressed sentence. For e.g. “Sure, I have plenty of time to proof read your article today when my thesis is due for submission tomorrow.”. Furthermore, the language used to express the opinion itself is a challenge for doing a sentiment analysis. It is very challenging to account for all the slang, dialects, and language variations.
Sentiment words also known as opinion words are without a doubt most important identifiers of sentiments. Positive sentiment words such as awesome, mind blowing, good, amazing, etc. and negative sentiment words such as horrible, bad, awful, etc. are usually used to indicate positive or negative sentiments. In addition to individual sentiment words, phrases and idioms can also be used as sentiment identifiers and play significant role in sentiment analysis. Sentiment lexicon also known as opinion lexicon is a list of such words, phrases and idioms. Many researchers have used different techniques to compile and make use of such sentiment lexicons for the task of sentiment analysis and are useful in many cases, however, they also have few limitations which are listed below:
- The orientation of sentiment word may depend on the application domain. For example, the word “suck” usually indicates negative sentiment, e.g., “This phone sucks” but can reflect positive orientation when used in different domain, e.g., “This vacuum tube sucks air really quick”.
- Sentiment words in sarcastic sentences are very hard to account for. Sarcasms are mostly prevalent in political discussions compared to product reviews making political opinions hard to work with.
- A sentence may not necessarily express sentiment although it contains a sentiment words. Interrogative and conditional sentences are two examples of such sentences. For e.g. “Can you tell me if Samsung Note 7 is good phone?”. Although, the sentence contains the word “good”, it doesn’t reflect positive sentiment.
Finally, it is important to address the fact that the underlying problem of sentiment analysis is Natural Language Processing (NLP). Sentiment analysis closely relates with the major elements of NLP such as negation handling, segregating word disambiguation, resolution of conference, etc. Although sentiment analysis is a very narrower NLP problem, it is very difficult one because of the major elements of NLP mentioned above which are tied with the sentiment analysis. Those problems do not itself have a concrete solution in NLP to utilize in the sentiment analysis tasks.
Challenges in Social Media Sentiment Analysis
With the advent of social networking sites and mobile technologies, the amount of opinionated content on the web is on the rise. People from all over the globe can express their views and opinions freely in the web. People seldom have to worry about the consequences of posting their thoughts on the web and they may even use anonymity to express their opinion. Even though, these opinions of people extremely valuable for social media analysts and sentiment analysis, it comes with a price. Regardless to say that when people don’t have to fear about the consequence of their opinion on the internet, it is very likely that people with malicious agendas can take advantage of such freedom. People can fake their opinion and post biased reviews on the internet to belittle any person, products, organization, etc. Not only people some organizations and commercial companies also do not shy away from posting fake reviews, forum discussions and online posts to spread negative rumors about their competitors to gain a competitive edge in the market. These individuals and organizations posting fake reviews and opinions are known as opinion spammers and this tendency of posting biased views is known as opinion spamming.
Opinion spammers and opinion spamming is a major problem when studying social media sentiment analysis. Opinion spammers tends to skew the results of the sentiment analysis and show biased results, hence it is of utmost importance to detect any spamming activates. Only trusted sources and reviews can be taken in to consideration to mitigate the effect of opinion spammers and opinion spamming when conducting a social media sentiment analysis.