A full list can be found here https://www.scraawl.com/product/twitter-premium-advanced-search-rules/.

A full list can be found here https://www.scraawl.com/product/rss-advanced-search-rules_/available-news-sources-search/.

Use the link under the search box on the Raw Data tab. The operators vary by data source.

SocL can currently analyze data from Twitter, Facebook, Tumblr, YouTube, VK, Reddit, and other PAI sources. Contact us about other sources.

Due to restrictions imposed by the PAI source data providers regarding data collection and use of their data, SocL does not collect data from certain PAI sources. If users want to use SocL’s managed access APIs to collect data from these sources using the feeds’ public APIs, SocL can be configured to do so. In such cases, depending on the PAI source and their terms of service, SocL may require users to create and link an application/user token that is approved by the PAI source and configure a dedicated collection service. We respect user privacy, therefore data collected on behalf of a user is neither shared nor distributed to any third party or third party application. Contact us if you’d like to use SocL’s managed APIs for data collection.

No. Your user account and associated application token is only used when you create reports with your Scraawl login. Other Scraawl users either use Scraawl’s dedicated application tokens or their own personal tokens. You can always choose to unlink your personal social media accounts from the My Profile page.

Multi-Feed Search allows the user to search multiple PAI sources at once using keywords or by location. The results are collected in one report. Premium Search provides a much wider range of search options such as streaming and historic search, and search by region, user, language, etc. The number of posts returned can be specified up to the report limit. Premium Advanced Search allows users to write custom queries using Boolean logic and advanced search operators.

In Multi-Feed and Premium Search, multiple keywords separated with commas are executed as OR logic, i.e., posts that contain one or more of the keywords are returned. Multiple keywords separated by spaces are executed as AND logic, i.e., posts that contain all keywords are returned. Premium Advanced Search uses a separate search grammar for Twitter (https://www.scraawl.com/product/twitter-premium-advanced-search-rules) and Other PAI Sources (https://www.scraawl.com/product/rss-advanced-search-rules-2/).

Account Monitoring is a search used to collect posts related to a brand’s social media account (posted by or mentioning) for purposes of brand reputation management. Multiple brand handles can be tracked in a single report. Account Monitoring reports appear in a designated area on the My Reports page. Account Monitoring reports include the Social Metrics advanced analytic which provides statistics on a brand’s reach, engagement, and impressions. Social Metrics is not available for other search reports.

Yes. With SocL you can search in 100+ languages. Use the Translate keywords function to enter keywords in other languages. Any foreign language posts returned from your search can also be translated into English using SocL’s Translate function which is integrated throughout the tool.

The choice of search options is driven by the APIs available from the data providers and the meta data associated with each post. Feeds with richer APIs and metadata provide a wider range of search options.

Use streaming if you are interested in collecting real-time data. That is, posts matching your search criteria will be captured as they are posted (until you reach your report limit). This is often referred to as “listening” in social media. Depending on the current social media activity matching your search criteria, your search may take time to return posts. Recent search options go back into the specified time window to find posts matching your search criteria that have already been posted.

If you are conducting a recent search, it can take up to two minutes for the first batch of search results to be returned. New results are then returned in batches every few minutes. If you are conducting a streaming search, posts will be returned within minutes of being posted.

If you are conducting a recent search, make sure that your search criteria is not too constraining. Try using broader search criteria. If you are conducting a streaming search, there must not be any content being posted that matches your search criteria.

The number of posts collected depends on the report limit specified by you (within the limits imposed by the data source). When there are more posts available than the report limit, the most recent posts are collected.

Scraawl can only collect posts from public Facebook pages. The number of posts collected depends on the report limit specified by you (within the limits imposed by Facebook). When there are more posts available than the report limit, the most recent posts are collected.

Make sure the page you are searching for is public. If you’re still having trouble, try these rules for specifying the page name:

1. If the Facebook page URL ends in a numeric string, use it to create Scraawl report.
   1. E.g., when searching for this page: https://www.facebook.com/pg/scraawl-100078, use 100078 as the search term
2. If the Facebook URL does not end in numeric string then use the name of the page.
   1. E.g., when searching for this page: https://www.facebook.com/homedepot/ use homedepot
   2. E.g., when searching for this page: https://www.facebook.com/harley-davidson/ use harley-davidson

In Premium Search, use AND/OR logic with keywords. In Premium Advanced Search, use Boolean logic with advanced search operators to create search expressions. The advanced search operators can be found here (or the link can be found under the Premium Advanced Search box on the New Report page).

SocL has over a thousand PAI Sources and that number continues to grow. You can browse the available news sources in the repository here or by clicking the link next to Other PAI Sources on the Premium Search or Premium Advanced Search new report pages. Please note, collection of articles from PAI Sources began in January 2017 so no articles published before that time are accessible in SocL.

To request the addition of a PAI Source, please send a link to the RSS feed to contact@scraawl.com.

On the Premium Advanced Search page, use the drop-down menus to select from the full list of regions and countries represented in the repository.

Many of the search operators from Premium Advanced Search such as country: and category: are also available as search operators in Raw Data. For the full list of operators, use the link under the Raw Data search box.

Streaming search is not yet available for this data source.

The maximum number of reports and posts per report depends on your license. Check the My Profile page to see your limits.

Yes, your license determines the number of reports that you can share. Check the My Profile page to see your limits.

A Streaming Report is one that is actively “listening” and collecting posts that match the search criteria as they are posted. On the My Report page, Streaming Reports will be highlighted in green and show status as Running.

A streaming search can be stopped at any time by selecting Stop Report from the Actions menu.

Your license allows you to have a certain number of reports. You will get this message if you’ve reached your limit and are trying to create a new report. You can delete or archive one or more of your existing reports to get around this. Note that the number of archived reports is also limited – check the My Profile page to review your limits.

This means that you have applied some filters (or excluded some posts). Once a report has been filtered, use the Update statistics button to refresh the Basics Statistics calculations to reflect the selected data. Similarly, you can use the Update analytics button to refresh any of the Advanced Analytics. The original full dataset can easily be retrieved using Undo All Edits from the Actions pulldown.

A report is considered complete when the number of posts collected reaches the maximum posts per report limit (which can be found on your My Profile page). Sometimes, you will reach this limit before you have collected from the full time window because of the volume of posts matching your search terms.

A search is considered timed out when the search has been executing for an extended duration without collecting data. At this point the search will timeout even if the post count in the report is less that the maximum allowable posts per report.

You can share a report with anyone using the Share/Unshare feature which can be accessed from the Actions menu. To share a report, enter the e-mail address of the person with whom you want to share the report. That person will receive an e-mail informing them that a report has been shared with them. The person will need an active Scraawl license to view the report.

Yes, if you are the owner of the report, you can revoke access to a particular recipient or all recipients of a shared report. To revoke access, select Share/Unshare from the Actions menu and then select Revoke Access for individual users. The Revoke All button can be used to revoke access for all users.

All shared reports can be seen in the My Folders section of the My Reports page. The Shared Reports folder includes reports you have shared with others. the Reports Shared With Me folder includes reports others have shared with you.

Only the owner of a report can see the raw data and run analytics on a report. All viewers of a shared report have read-only access – they will see any filters you have applied and any analytics you have run.

You can bookmark posts from any report (or from multiple reports) and export them to a Case Report. Case Reports then behave like any other report – you can see Basic Statistics, Advanced Analytics, and Raw Data.

Once posts that match your search criteria are collected, SocL computes Basic Statistics related to top users, words, hashtags, mentions, URLs, retweets, languages, and locations. The top 50 for each of these categories are shown, with each element acting as an interactive drill-down for data exploration, and an option to filter and exclude posts associated with selected elements. Basic Statistics are updated as data is collected and only those basic statistics that are relevant to the data feed are computed.

Not all statistics and analytics are relevant to all feeds. Only those statistics and analytics that can be computed from the available post data and metadata are included for each feed.

Each element in any of the Basic Statistics (e.g., Top Users, Top Words, etc.) can be used as an interactive drill-down to see the posts associated with the element. This can be done from the elements displayed in the Basic Statistics dashboard or from the full list of elements displayed when the Details button is clicked. Click the Filter button on the Details page to select individual elements, and then click the Exclude selected button to hide the posts associated with the selected elements. Use Undo All Edits from the Actions menu to undo any applied filters and get back to your full data set.

SocL only collects user profiles for the top 50 users in the report. For profiles of the other users (e.g., those mentioned in a post), SocL redirects you to the social media site to view the user profile.

Geographic data is associated with posts in three ways: (1) If the user has location services turned on or checks in to a location, the post is geo-coded using GPS coordinates. (2) If the user has a geographic location specified in his/her profile, the post is geo-profiled. (3) If the user mentions a geographic location in the body of the post, the post is geo-referenced. Given these definitions, it is possible for a post to be associated with multiple locations, e.g., a user from Washington DC tweets about upcoming trip to Paris while visiting friends in Pittsburgh Pennsylvania with location services on.

SocL offers a wide range of Advanced Analytics including geo-spatial analytics, social network analytics, content/text analytics, behavioral analytics, and image analytics. Please visit the SocL Features page for a full list of available analytics.

No. Advanced Analytics vary by the data source, and depend in part on the data that is available. Only those relevant to a particular feed are shown in the Advanced Analytics tab of the report.

The Heat Map provides a graphical representation of the density of posts across geographic locations (derived from geo-coded, geo-referenced, and geo-profiled information). SocL provides zoom in and out capability and the ability to toggle to the Location Map (a Basic Statistic) where you can zoom to show the individual posts associated with locations.

Influence Discovery uses a form of page-rank centrality to compute normalized scores that represent the influence of users and hashtags in the social interaction graph. The top 10 most influential users/hashtags and are shown. You can drill down on any influential user or hashtag to see the influence scores for the top 10 neighbors of the user/hashtag and their impact ratios (which represent their relative contributions to the influence score of the selected user/hashtag).

Community Detection uses a graph-theoretic approach to partition the social interaction graph into communities based on the level of interaction between nodes (users and hashtags). SocL provides community statistics including the number of communities, average community size, and maximum community size. Details of the top 10 communities and the most connected users/hashtags in these communities are shown.

Sentiment Analysis uses a machine learning-based approach that scores each record based on textual and structural features and then classifies each as positive, negative, or neutral. Classification accuracy is in the 75-80% range on average and may vary depending on the context of the conversations and on the language analyzed. Misclassifications can occur. Top posts representative of positive and negative sentiments are shown. SocL’s Sentiment Analysis works for 12 different languages (English, Turkish, Spanish, Russian, French, German, Italian, Polish, Portuguese, Indonesian, Tagalog, and Arabic).

Topic Modeling uses a probabilistic model to discover patterns of word use within and across posts. The model uses this information to identify up to eight abstract “topics” that occur in the data set. Up to 15 top words are shown for each topic along with scores that represent the contributions of each word to the topic.

The number of topics shown is a function of the data that is collected. If your search criteria were broad, your data is likely to have more topics than if you had used a more focused set of search criteria.

Media Gallery collects images and videos contained in the posts, sorts them by frequency, and presents them in a visual gallery. Simply click any image to view more details. Use Filter by Media Type to see media of a particular type. Export images to PixL for deeper analysis.

Avatar Gallery collects the avatar images of the users posting in your report and presents them in a visual gallery. Export avatar images to PixL for deeper analysis.

Named Entity Recognition uses a statistical matching approach to identify named entities by applying fast search with various large-scale gazetteers. Named entities recognized by SocL include persons, locations, and organizations. SocL shows the top 50 named entities with their types, entity counts, user counts, and percentages.

Bot Detection uses various statistical, temporal, and textual features that are detected in social media posts and historical user profiles to distinguish potential bots from human users. SocL assigns a bot score to each user and then displays the top users with the most representative bot behavior.

Using the posts collected in your report, SocL constructs a Social Graph in which the nodes are the users and hashtags, and the connections are the interactions between them (e.g., retweet, use of a hashtag, or mention of a user, etc.). A Friendship Graph on the other hand can be created for any user (even users not represented in your report) and shows up to 200 of the most recently added friends and followers of the specified users. There are buttons to generate friendship graphs from the top users or top mentions from your report, or you can manually enter users.

The Social Metrics advanced analytic is only available for Account Monitoring reports. SocL uses the following definitions for its social metrics:

• Engagement is the sum of all replies, retweets, mentions, and likes of brand’s posts over a reporting period. It is a reflection of the number of times a user interacted with the account.
• Reach is the sum of the followers of all unique users who mentioned or retweeted the account during the reporting period plus the number of followers of the account over the reporting period.
• Potential Impressions is the sum of the reach of all posts mentioning or retweeting the account over the reporting period.

When the reporting period is more than a day, the maximum follower count over the reporting period is used in the calculation.

Twitter uses slightly different definitions to calculate social metrics. In addition to including replies, retweets, and mentions, Twitter analytics includes hashtag clicks, link clicks, avatar clicks, username clicks, Tweet expansions, and follows. Similarly, Twitter analytics uses “actual impressions” as opposed to potential impressions. The definition used by Twitter’s new engagement API identifies an impression as “anytime a user sees your Tweet.”

For Facebook we use the following definitions of social metrics for public pages:

• Facebook fans is the number of Facebook users who liked the page (not posts).
• People Talking About is a trailing seven-day Facebook metric that measures how many people are “talking” (creating stories) about the page.
• Check-ins is the number of users who announced their location to their friends on Facebook using the “check in” feature.
• Engagement is the sum of likes, comments, and shares of all downloaded posts in the report.

For user-managed pages, Facebook Insight analytics can provide additional analytics related to Consumptions (engagement), Reach, and Impressions. Consumptions is the number of times users clicked (anywhere) on the page. This includes, Link Clicks, Photo Views, Video plays, and Other Clicks (Likes, comments, shares, etc). Reach is the number of unique people who have seen content associated with the page. Impressions are the number of times a post from the page is displayed on a user’s timeline. People may see multiple impressions of the same post.

The Basic Statistics timeline shows the volume of search results over time. You can filter results using the timeline. In Other PAI Source searches, the Timeline Advanced Analytic shows the all the articles in the report in chronological order. Each document group contains 20 documents and their published times are shown in a timeline view. Users can scan through the articles in each group by swiping through the articles or using left and right arrows. Unlike the Basic Statistics timeline, the Timeline Advanced Analytic pulls in a featured image and links directly to the article.

SocL’s integrated suite of Advanced Analytics leverage a variety of cutting-edge technologies that fall into the following categories:

Geo-spatial: Heat Map (Location Map)
Social Network: Social Graph, Influence Discovery, Community Detection, and Friendship Graph
Natural Language Processing: Sentiment Analysis, Topic Modeling, and Named Entities
Image processing : Media Gallery and Avatar Gallery
Behavioral: Social Metrics and Bot Detection

The Raw Data tab shows all the posts in your report. Posts are displayed in chronological order with the most recent post displayed first. The posts are searchable using the supported operators for each feed (see the operators link in the search tip under the Search box).

There are many ways to filter your data in SocL. (1) You can use the Timeline sliders on the Basic Statistics dashboard to focus on posts during a particular time period. (2) You can select and exclude posts associated with elements from any of the “Top” lists in Basic Statistics. (3) After drilling down on a particular element in a “Top” list or analytic, you can use the View All button to see the posts associated with the selected element and then choose to Keep Only Matching or Exclude posts. (4) You can use the geo-fencing feature on the Location Map to focus in on posts from a particular region – remember to select the relevant geo-type (geo-coded, geo-referenced, or geo-profiled). (5) You can search and filter the Raw Data using Boolean logic and search operators.

Using the supported Boolean logic and search operators for each feed (see the operators link in the search tip under the Search box), you can apply search expressions in Raw Data. In order to use the analytic operators (e.g., sentiment: or community:), you must first run the corresponding Advanced Analytic.

No, the data is not lost. To revert to the original dataset, select Undo All Edits from the Actions menu.

Once you have searched and filtered your data, you may want to bookmark select posts for future reference or a separate analysis. The Bookmark menu provides options to bookmark or un-bookmark selected posts. You can Export Bookmarked posts (from Actions menu) to a Case Report. Case Reports then behave like any other report – you can view Basic Statistics, Advanced Analytics, and Raw Data.