Twitter(GNIP) is migrating to new set of API’s and rules for Firehose access. All Firehose subscribers, such as Scraawl, will need to complete their migration by Dec 1. The Scraawl team is in the process of migrating to these new API’s and our migration will be complete by 11/15/2016. This migration will impact a few of the Scraawl’s Advanced Search rules and operators. The search syntax has been changed in some cases, and in some cases, certain filtering rules have been deprecated. A summary of these changes are listed below.
PLEASE NOTE: After 11/15/2016, any running reports with search rules not in compliance with these new rules will be automatically stopped. Please re-start your reports using these new rules. If your searches did not use any of the operators that have been changed or deprecated your reports will continue to run.
If you have any questions, please do not hesitate to contact us and we will be happy to set-up a telecom/VTC to address any questions or concerns you may have.
New Features and Operators
Below is a comprehensive list of the new operators and features available on 11/15/2016.
|2048 character query support||Search now supports 2048 character queries without positive or negative clause limits|
|has:images operator||Query for only tweets with native images|
|has:videos operator||Query for only tweets with native videos|
|has:symbols operator||Also known as cashtags, has:symbols support querying for only tweets that have symbols/cashtags|
|$ operator (e.g. $AAPL)||Query for tweets with a given symbol/cashtag|
The table below contains operators that have been deprecated in version 2.0 and the suggested replacement operator (if available).
|Current Operator||Replacement Operator (if exists)||Details and Rationale|
|url_contains:||url:||url_contains: was originally built as a substring match on the urls of a tweet. The new ?url? operator performs a tokenized match|
|twitter_lang:||lang:||Gnip’s language classification is being deprecated and replaced with Twitter’s language classification. With this change, the more commonly used “lang” operator will now be applied to the Twitter language classification, rendering the twitter_lang operator unnecessary and redundant.|
|has:lang||-lang:und||Twitter language classification field is present for all tweets and unclassified tweets have a value of “und” (undefined), so a query for only tweets with a language classification would now be -lang:und (i.e. NOT language=undefined).|
|bio_lang:||lang:||The “bio_lang” operator had very low adoption and is being replaced by the “lang” operator, which now uses Twitter’s language classification.|
|country_code:||place_country||Same functionality as country_code|
|profile_country_code:||profile_country||Same functionality as profile_country_code|
The following operators will not be supported after migration to new Twitter/GNIP API’s on 11/15/2016.