Measuring Sentiment

by Morgan Siem on January 8, 2010

in Uncategorized

Measuring Sentiment

Measuring Sentiment

I just read a good article from threeminds called There’s No Silver Bullet For The Big, Bad Social Media Measurement Wolf. One of the interesting messages in the article stated that:

“If marketers’ business objective is to influence brand perception, then conversation sentiment – rather than conversation volume – is what should be monitored and measured.”

I agree that sentiment is one of the more valuable metrics to monitor when measuring success. However, it’s easier said than done. The article goes on to state that, “a listening platform with sentiment analysis would be an important tool in a marketer’s social media tool belt.” My response, after trying several of these tools out, is that they’re highly inaccurate and should not be strongly relied upon…at least not yet. Radian6’s sentiment analysis is fairly new (even in social media years). They say that as more people use the tool and correct for inaccurate analyses, the better the tool will become. Ok, I believe that.

The problem with these tools, though, is that they misread sarcasm. Imagine the auto-sentiment ranking on the following: “Pages are taking 10 minutes to load today. Thanks a lot [insert name of internet provider]!”

To make up for this, there is (with most tools) the option to manually set sentiment. FiltrBox is one example of a tool that allows you to manually set sentiment as negative, neutral or positive. This can work if you are monitoring a brand that brings in a low volume of posts. If that’s the case, then absolutely measure sentiment rather than volume.

However, if you have volume, it becomes very difficult to manually set sentiment because the posts are coming in as quickly as you can read them. For this reason, I disagree that sentiment should be measured rather than volume. Measure sentiment in the ways that you can. Take volume with a grain of salt. Understand that both play a role in your ROI and are important metrics.

The article goes on to make a good point that “the number of fans is important but only when coupled with the measurable influence of those advocates.” Yes.

What I’m getting at is that there are a lot of important metrics to measure.

What are the most important metrics to measure for YOUR business and why? What do you currently measure? What do you want to learn how to do a better job of measuring?

Similar Posts

No related posts.

{ 7 comments… read them below or add one }

Michael Hubbard Michael Hubbard January 8, 2010 at 8:28 pm

I like this post a lot – I think every marketing software would love to have some form of artificial intelligence that continues ranking items like sentiment, but we all know there are flaws in it, as well as fears of too much data tracking going on.

That being said, the reason I’m most intrigued by this post and ThreeMinds’ post is that it is completely against the age-old PR adage of any press is good press. Doesn’t that apply to social media as well? If you have thousands of people weighing in both for and against you, one of the best things is that they’re talking about you – correct? Yes, I know, nobody ever wants bad press, but it does provide an opportunity for the company to address and correct the issue at hand and show all of those people that they are listening.

So for my money, I’m still ok with just tracking the number of mentions, and letting the user sentiments identify additional opportunities… But speak away – good or bad is what I say! Just do it a lot :)

Reply

David Requa January 9, 2010 at 4:00 pm

I agree with you about the challenges of measuring sentiment. I also see why marketers are have such a tough time trying to establish a framework for measuring social media ROI. Unlike traditional marketing, which is at least somewhat quantifiable in terms of money in money out (financial capital), the very nature of social media is about creating social capital, knowledge, information, opportunities, contacts, trust, confidence, friendship, good deeds, and goodwill. All of which will be nearly impossible to quantify.

Further, I think marketers run the risk of cramping their social media style if too much emphasis is put on ROI. So, as much as I’m fascinated by the attempts to do so, I try not to get too caught up in the idea of quantifying ROI in favor of focusing on the attributes of successful campaigns.

Thanks for the article. I enjoyed it.

Reply

Dwight Moore January 9, 2010 at 5:05 pm

Nice short piece on measurement, and several interesting comments. I agree that for low volume, manual settings may be feasible. I would disagree with your assessment on large volume. The rationale is that this is largely a statistical measurement, and in large volume, these exception cases would be low relative to the overall sample. Sentiment is a directional concept, not a specific measure using a fine units of measurement like today’s temperature (at least not in the near term). One has to keep in mind what and why it is being measured.

I do agree with the commenter (Mr. Requa) on ROI. There is a huge focus on ROI which tends to discourage taking smart risks. On the other hand, the reason for focus on an ROI is to be good stewards of the funds. Knowing when and where to direct money (capital or expense) is on of the key roles for leadership, and the old adage “you can’t manage what you can’t measure” applies. In these times and this environment, the pendulum has swung to the conservative side to be very focused on measurement.

Reply

Brian McDonald January 11, 2010 at 2:02 pm

Morgan,

Nice review. I’ve had the same challenge measuring sentiment. When you have a small number of comments/tweets to review, manual sentiment setting is OK but what do you do when you are a national brand like Budweiser, GE or the NFL?

I don’t think artificial intelligence will ever get to the level to understand human emotion, humor, slang, etc. But measuring sentiment is crucial. At the end of the day the boss wants to know did they love it or hate it and what affect did it have on our brand (i.e. revenue). With all chatter, there is great real time feedback but how can you manage it, consume and react to the important posts and reactions about your brand?

2010 will be the year of ROI in social media and interactive marketing. One interesting thought I’ve had recently is that with interactive marketing we have a great opportunity to direct customers to a landing page and try and influence them. The ability to track this is much strong than TV, radio and print ads that do not have as such a strong correlation from impression to action.

Reply

Morgan Siem January 11, 2010 at 2:35 pm

Great comments! Thank you!

Michael – I’d say, careful what you wish for. I think it depends on the type of negative press and the nature of the product. Negative press can be ok or beneficial when it’s negative press about the personal life of one of the employees or something like that. The flying rumors can help put the brand name out there in the public eye. However, negative press could devastate a brand if the brand’s No. 1 product is something like a car seat for a toddler, and suddenly information comes out that your car seats are actually harming children. I just don’t think you ever want press that says you hurt babies :)

David – While I agree that too much emphasis on measuring ROI can cramp your social media style, I do believe that it’s important to have measurable goals and benchmarks for success. I actually think that the success of a social media strategy can be MORE easily measured than the success of traditional marketing since most things can be tracked online.

Dwight – You make a good point that with higher volume, the exceptional cases may become negligible. I also like your point that too much focus on ROI can discourage taking risks. With social media, I’ve seen that the risk takers see the greatest ROI. Measurement should encourage taking risks because you can see with each risk what worked and what didn’t.

Brian – You’re right. Just because it’s not easy to measure doesn’t mean it’s not important. Here are some ways I’ve learned to work with what’s available to measure sentiment:
•What words are appearing and disappearing from the conversation cloud? Are they negative words (hate, broken, problem, bad) or positive (love, great, thanks, cool)?
•When there’s a spike in conversation, what caused the spike? Did one particular post go viral? Was it good news or bad news?
•And there’s still manual or auto-sentiment ranking. Take it with a grain of salt.

Reply

Michael Hubbard michaelhubbard January 11, 2010 at 2:57 pm

I’m not a PR major, but it has been proven time and time again, that all PR is good (give or take :) )… Here’s an example for you. Let’s say I’m flying off to Miami, and I’m checking for the lowest rates. I see AirTran has the best rates. I think to myself, hmm… I remember something about them, just can’t remember what – so I book the flight because it’s a really good deal. In mid-flight, it then occurs to me… Oh yeah, AirTran used to be Value Jet!?! If you “Google” Value Jet of 1996, you’ll remember why they changed their name. This is no disrespect meant to those that died in that crash – this is merely proving a point that sometimes we remember things, and we don’t exactly remember why. Unlike most, I don’t assume the worst – therefore, even bad PR makes me remember a name, but why I remember it gets fuzzy as the years go on.

Reply

Michael Hubbard michaelhubbard January 11, 2010 at 2:57 pm

Quick point of clarification… I don’t actually encourage bad PR… I’m just going back to my original point that I don’t believe tracking sentiment is all it’s cracked up to be.

Reply

Leave a Comment

Want to add a picture to your comments here on Media Two Point {OH}? Upload your picture at Gravatar to make it happen.

Your comment may be nofollow free.

Previous post:

Next post: