Rush Limbaugh has 33% more buzz online than Jay Leno according to the Vitrue Social Media Index. Social game company Zynga has tweets by posters with 3 times as many followers as their competitor Playfish according to the data over the last 30 days from Radian6. But it doesn’t really mean anything unless you know what percentage of that buzz or those tweets are positive or negative.
Sentiment is the guiding light that helps marketers and their organization know what to do with the mountain of social media that is growing exponentially through Facebook and Twitter. We need to understand our current positive to negative ratios, drill down to uncover what’s driving it, then put programs in place to mitigate the negative and foster growth of the positive. Just like we determine the ROI on capital improvements, with an accurate measurement of positive and negative sentiment we can measure the return on investing in programs to improve consumer sentiment about our brands.
So the vision thing is great, but the reality of going through thousands of posts and tweets and determining sentiment is the biggest impediment facing marketers and their organizations to moving forward. The industry average in being able to automate this process of categorizing the sentiment of user posts is about 60%, based on a conversation with Radian6.
Radian6 is a great workflow tool, but today they only offer users the ability to hand code the sentiment of each post. Their goal is to introduce automated sentiment attribution into the Radian6 dashboard this summer to get to 70 or 75% accuracy.
Crimson Hexagon has shown through research by co-founder and Harvard Professor Gary King that its approach to automatically categorizing the percentage of posts in blogs is higher than hand-coding or strictly counting the number of words. “Crimson Hexagon doesn’t count words, which can mislead; it amplifies human judgment to give the percentage in each category accurately,” noted King in a recent tweet. One of King’s colleagues noted that Crimson Hexagon is close to 80-88% accuracy for positives and negatives using their approach.
Very few marketers and organizations will spend the resources to hand-code responses (in fact King’s research suggests that one shouldn’t and typically see diminishing returns after 500). We will more and more rely on automated tools to do this work for us. Thus when the time comes to pick vendors, agencies and tools to help us measure sentiment, it is critical for us to understand the underlying data, methodology and resulting accuracy rates.
To date Crimson Hexagon’s methodology seems to provide the most promise. What other tools are you using to identify positive and negative sentiment of your brand and what is their underlying methodology?
A trending topic means squat unless you measure whether it’s positive or negative. All day long people commented that the social media buzz for Adam Lambert, predicted he’ll beat out Kris Allen tonight in the American Idol final. Adam was constantly ahead of Kris on Twitter Trending Topics (see picture at right), showing more overall buzz in the 