This is Part 2 of a two-part blog to help brands and agencies gain a better understanding of how to select an enterprise-level social media analytics tool, and why it is important to go beyond the sales pitch and dig deeper to learn what lies under the hood. In the first part of this blog, I posed the following four questions that I believe CxO’s should be asking their analytics tool vendors and analytics teams:
- How accurate and complete are the data driving the analytics?
- Can you easily filter or clean the input data?
- Do you know how the metrics/analytics are computed or is it a black box claiming the use of AI and machine learning?
- Can you use the tool’s analytics as a filter to derive actionable insight?
In Part 1 we discussed questions 1 and 2. In this blog we further drill down into questions 3 and 4.
3. Do you know how the metrics/analytics are computed or is it a black box claiming the use of AI and machine learning?
Social media metrics and analytics provided by most tools today can broadly be categorized into the following categories (i) basic statistics – “count-” related metrics associated with top users, mentions, posts, demographics, devices, timelines, etc.; (ii) social engagement metrics – those related to reach, engagement, impressions. etc.; (iii) content analytics – those related to named entity recognition, topic modeling, term frequency counts (word clouds), and sentiment analysis; (iv) graph analytics – influence detection, community detection, link analysis, social graphs, and friendship graphs; (v) geo-spatial analytics; (vi) media analytics; and (vii) predictive analytics.
Most tools only provide a subset of these capabilities and it is important to understand how the metrics and analytics are calculated so that you understand their value in providing you with actionable insights about your data. This is especially important since there has been no standardization of metrics in the field of social media analytics, nor is there a standardized data set against which performance of vendor analytics can be compared. As an example, even a simple metric such as Twitter engagement has different definitions. Some tools define it as the sum of mentions and retweets, some include likes in the definition, and others include clicks and follows. For other more complicated analytics such as sentiment and influence detection, many vendors will not divulge the details of the underlying algorithms for proprietary reasons.
In the context of metrics and analytics some questions that should be asked include:
- What type of advanced analytics are offered and how many are directly relevant to your objectives?
- How are social engagement metrics defined and how do they differ from the native definitions of the social media platforms?
- How large a data set can be analyzed? Unfortunately, due to the computational complexity of several of these advanced analytics, some tools only run these analytics on a small subset of the data or on a random sampling. The details are hidden in a footnote. Depending on the analytic, results run on small data sets may not be representative of the full data set.
- What level of sentiment analysis in needed for your analysis ? Is sentiment analysis being performed using a dictionary-based approach, or is machine learning being used? If machine learning is being used, what is the training data? How many different languages are supported? Are only aggregate statistics available or can you view results post-by post?
- Is topic analysis being performed or is it merely a term frequency count? How does topic modeling work across multiple languages?
- How is named entity recognition being performed? is it gazetteer-based or NLP-based? How many languages are supported? Can entity resolution be performed?
- Can analytics be performed across multiple data sources?
4. Can you use the tool’s analytics as a filter to derive actionable insight?
Most importantly, does the tool allow for the use of analytics as a filter to find actionable insights or are the results “static”? Once you look past the pretty graphics, a true analyst or a senior, decision-making executive will want to evaluate alternate scenarios.
For example, in the context of understanding your brand’s reach and engagement, can you drill down and understand who the primary contributors of the reach and engagement metrics are? If the large reach is primarily due to a few individuals or influencers amplifying your brand, what would your reach and engagement metric look like without them? Similarly, are your engagement metrics primarily being driven by bots? Can you eliminate the bots and re-compute your engagement?
Similarly, when looking at brand sentiment analytics, it’s important to understand what is driving the positive or negative sentiment. Is the sentiment being dominated by a few posts and their re-posts, by a particular topic, or by a particular community? What are the sentiment trends outside these communities? For brand reputation management, it’s important to be able to execute workflows in which one can drill down into particular subset of posts, topics or communities, filter by these communities, and then re-compute all the basic and advanced analytics. If a brand can identify the influencers in the communities that are driving the negative (or positive) sentiment, then they can they can engage with these influencers and execute the appropriate courses of action needed to maintain brand reputation.
While these examples illustrate some simple use cases, there are numerous ways to analyze and view your data. Your social media analytics tool should support an unlimited number of workflows so that you can get actionable insights from your analytics.
If you are planning to use, or are currently using enterprise-level social media analytics tools to inform decision processes regarding marketing campaigns, investment strategies, or public relations, it is really important to be able to understand, and drill down into any analytics being presented to you by various tools. Without the ability to do so, analytics are nothing but a set of pretty graphics that can drive you into making the wrong choices with respect to your brand. Unfortunately, as the market is flooded with social media analytics tools and all the buzz about engagement metrics and ROI, choosing the right tool can be quite a challenge. If your enterprise does not have a team to help you navigate this complex maze of social media tools, engage with an unbiased set of subject matter experts with in-depth understanding of this space – those who know how to use a proven analytic process to help brands quantify performance, drive strategic decision making, and improve brand awareness.