Changes to Scraawl Advanced Search Rules and Operators

Changes to Scraawl Advanced Search Rules and Operators

Twitter(GNIP) is migrating to new set of API’s and rules for Firehose access. All Firehose subscribers, such as Scraawl, will need to complete their migration by Dec 1.  The Scraawl team is in the process of migrating to these new API’s and our migration will be complete by 11/15/2016. This migration will impact a few of the Scraawl’s Advanced Search rules and operators. The search syntax has been changed in some cases, and in some cases, certain filtering rules have been deprecated.  A summary of these changes are listed below.

PLEASE NOTE: After 11/15/2016, any running reports with search rules not in compliance with these new rules will be automatically stopped. Please re-start your reports using these new rules. If your searches did not users any of the operators that have been changed or deprecated your reports will continue to run.

If you have any questions, please do not hesitate to contact us and we will be happy to set-up a telecom/VTC to address any questions or concerns you may have.

New Features and Operators

Below is a comprehensive list of the new operators and features available on 11/15/2016.

Feature/Operator Details
2048 character query support Search now supports 2048 character queries without positive or negative clause limits
has:images operator Query for only tweets with native images
has:videos operator Query for only tweets with native videos
has:symbols operator Also known as cashtags, has:symbols support querying for only tweets that have symbols/cashtags
$ operator (e.g. $AAPL) Query for tweets with a given symbol/cashtag

Replaced Operators

The table below contains operators that have been deprecated in version 2.0 and the suggested replacement operator (if available).

Current Operator Replacement Operator (if exists) Details and Rationale
url_contains: url: url_contains: was originally built as a substring match on the urls of a tweet. The new ?url? operator performs a tokenized match
twitter_lang: lang: Gnip’s language classification is being deprecated and replaced with Twitter’s language classification. With this change, the more commonly used “lang” operator will now be applied to the Twitter language classification, rendering the twitter_lang operator unnecessary and redundant.
has:lang -lang:und Twitter language classification field is present for all tweets and unclassified tweets have a value of “und” (undefined), so a query for only tweets with a language classification would now be -lang:und (i.e. NOT language=undefined).
bio_lang: lang: The “bio_lang” operator had very low adoption and is being replaced by the “lang” operator, which now uses Twitter’s language classification.
country_code: place_country  Same functionality as country_code
profile_country_code: profile_country  Same functionality as profile_country_code

Unsupported Operators

The following operators will not be supported after migration to new Twitter/GNIP API’s on 11/15/2016.

  • time_zone:
  • profile_region:
  • profile_locality:
  • profile_subregion:
  • in_reply_to_status_id:
  • retweets_of_status_id:
  • profile_point_radius:
  • profile_bounding_box:
  • source:
  • has:profile_geo_region
  • has:profile_geo_subregion
  • has:profile_geo_locality



Related Posts---

Back to top