Searchtweet – Design patches that respond to twitter posts.


Update: Version 1.1b is available to download. Version 1.0 and 1.1 do not work anymore due to changes in Twitter’s API.

Searchtweet is a mxj external that searches twitter with a given keyword and returns a desired maximum number of most recent posts, with username, date and twitter text body routed to separate outlets. The object returns a specified number of most recent twitter posts that contain a given keyword, #hashtag, or @usertag.

Advanced search queries also work:

https://support.twitter.com/articles/71577-using-advanced-search

Searchtweet uses the twitter4j library (http://twitter4j.org) programmed by Yusuke Yamamoto.

Note:
You must limit your search frequency to an absolute minimum to avoid hitting the API rate-limits. (Ex. Do not connect a [metro 10] to the inlet, your IP will be banned in no second)
Currently the plugin does not require user-authentication, and it is unclear if there is a limit for unauthenticated searches.
For more info:

https://dev.twitter.com/docs/faq

Cheers,
Gokce Kinayoglu


April 23, 2012 | 11:26 am

This has got me thinking! :)


December 8, 2012 | 10:16 pm

This is fantastic. Thank you.



RT White
March 21, 2013 | 6:26 pm

Excellent object and tool! Forgive my naivete as the world of APIs is pretty new to me, but is there a way to limit the output of this device to a single language? The patch I’m working on is mostly pertinent in English.

Thanks for any help.


March 22, 2013 | 10:52 am

Hi RT, and all,
The plugin works using the basic search functionality of the twitter API. I do not know if there is a way to limit searches based on country or language through specific search keywords.
Also, the twit4j offers a push-based update functionality which I did not implement in this plugin. If there are any exciting projects that needs that kind of a more "realtime" update feel free to contact me in person or here.



RT White
April 2, 2013 | 6:58 pm

Thank you Gokce! I will let you know if I get anything interesting working.



David Benson
April 15, 2013 | 6:11 pm

Hey all,

Just for a clarification, if you are using the standard search parameters on Twitter you can search by language and by region. If you are using the Twitter Streaming API, however, the query language is different. I have yet to dig into this patch, but I wanted to see if I could be of service.

For the standard search terms, using lang:en should limit your searches to english only.

For geolocation use: geocode:40.7484918,-73.98428659999999,25km. You will need to use a third party service to determine the Latitude and Longitude of your location (The one I posed is Manhattan) and you can use the km parameter to alter the distance.

I like to use http://geocoder.us/ for finding a latitude and longitude. Just enter an address and it should output what you want.

Hopefully that helps a little to answer your question, @Gokce.


April 16, 2013 | 11:06 am

Thanks for the information David! This is amazing.

The search parameters indeed work well with the script! There is a small trick. Max will interpret the commas in the message as separators. The message sent to the script needs to be formatted as shown below with backslashes before each comma, such as:

search geocode:40.748433,-73.985656,1km 1

Put that in a message box and wire it to the mxj directly. (Do not use the search box in the example patch.) This should give the most recent tweet sent from an area within 1km radius around the Empire State Building in New York. Tested, and here’s the result I received:

"fangirlDANosaur"
"RT @danisnotonfire: I’m currently having that classic tourist fangirl moment where I realise I know all the buildings in Manhattan from every movie/game ever. "

Looks like she is in New York. Exciting! Now go play!

Gokce



Thierry
April 25, 2013 | 5:52 am

Hi David and Gokce,

Thanks for your precisions about language filtering and geocode. I would like to filter the tweets according to their language. But, as "search geocode:40.748433,-73.985656,1km 1" works perfectly for me, "search voyage lang:fr 1" returns the following errors :

java.lang.UnsupportedOperationException: does not contain an int
at com.cycling74.max.Atom.getInt(Atom.java:572)
at searchtweet.search(searchtweet.java:34)

Did I miss something ?
Thanks a lot if you would have an idea…
-Thierry

PS: Gokce: I left a question on your FB page about language filtering, before having found this page, sorry :)



Thierry
April 25, 2013 | 7:08 am

Hi David and Gokce,

I finally found the right syntax to filter by language, which is: search "voyage lang:fr" 10
It needed quotes.
Thanks, Best
-Thierry


May 4, 2013 | 12:07 am

Hi Gotkce and all, is there any advice you can give me on a similar kind of thing I’m doing here? http://cycling74.com/forums/topic.php?id=47084


May 6, 2013 | 8:01 am

Hi Kaibe,

I suggest you to load the plugin and play with it before starting out with your project. Once you are able to get the text output that you need from searchtweet, the rest is figuring out how to "parse" it. There are several functions that does exactly that in MSP and there are also some external libraries for string parsing and search. Good luck!

Gokce


May 13, 2013 | 7:13 pm

Super cool – thanks for posting!


June 11, 2013 | 5:47 pm

Hi – looks like something has gone sour with the api – now i get this message:

TwitterException{exceptionCode=[26d84cad-002e486e], statusCode=410, retryAfter=0, rateLimitStatus=null, version=2.2.0}

any ideas – much obliged!

shane


June 12, 2013 | 5:42 am

Aha – twitter changed the API!

We will begin 410ing inbound API v1 traffic now. We will gradually ramp up over the next several minutes. More info: https://dev.twitter.com/docs/faq#17750

Have a show using this object in a few weeks, so if anyone has an idea, i would be very grateful and have no other choice but to buy you some beers in NYC!


June 12, 2013 | 5:42 am

Aha – twitter changed the API!

We will begin 410ing inbound API v1 traffic now. We will gradually ramp up over the next several minutes. More info: https://dev.twitter.com/docs/faq#17750

Have a show using this object in a few weeks, so if anyone has an idea, i would be very grateful and have no other choice but to buy you some beers in NYC!


June 12, 2013 | 9:04 am

I will take a look at this – hopefully soon!


June 13, 2013 | 5:44 am

Cheers!


June 14, 2013 | 9:47 am

Hey, I too noticed that my project is dead due to the API change. This is going to kill my graduation project pretty much =/

I downloaded the latest stable version of twitter4j, updated the twitter4j.jars in the libs folder, added twitter4j.examples to a Java project in Eclipse, following http://compusition.com/web/articles/maxmsp-eclipse, and added only the following to SearchTweets.jar:
import com.cycling74.max.*;
and
… extends MaxObject {

If I just copied the SearchTweets.class (not java) file to the Max folder, it just returned "Could not load class ‘SearchTweets’"

So I also tried to to create an object in Max: [mxj twitter4j.examples.search.SearchTweets] and I’m getting the following in the Max window:
(mxj) Class twitter4j.examples.search.SearchTweets is not a subclass of com/cycling74/max/MaxObject

Do I need to work on adding INLETS and OUTLETS? Did I miss out anything else I should add?
What can I do to get this working? Any help would be supremely appreciated!!


June 14, 2013 | 10:43 am

Guys do not panic. I will check this today.


June 14, 2013 | 12:58 pm

Done :). Please download and *test* the new version. User discretion advised! Let me know how it works.


June 14, 2013 | 1:24 pm

First off – thanks so much for even taking a look – i’ll never forget my 12th grade math teacher’s sign on her desk – a crisis on your part is not a crisis for me!

very cool – and please let me know if you are ever in NYC – you have some libations coming your way.

cheers!


June 14, 2013 | 7:19 pm

You, sir, are aces.. and spades.. and I can’t really explain how much this means to me! Let’s just say you saved my college Graduation Project – and also me, from breaking my head with Java and Eclipse.. Hehe..

DRKOSS put it very well, and I agree 100% :)

I’m all the way over in India, but if you do ever pop over here, I will be glad to oblige a trip around the city to see the sights and sounds of fair Bangalore, followed by drinks.

Now to get the final bits of my project wrapped up, cheers!


June 15, 2013 | 6:33 pm

You are welcome. It just took slightly more effort than I expected because twitter added an application authorization requirement which was not there before. But you should really thank twitter4j and its author, who seems to have kept his java library up to date till now.


January 11, 2014 | 4:32 pm

Can anyone confirm if this is still working? I can’t seem to get it to even load up on max 6 windows 8.1

A blank max window opens up and the patcher does not load.


January 28, 2014 | 9:17 am

Hmm, doesnt seem to be working for me. Here’s what’s in the Max window:

java.lang.IllegalStateException: Authentication credentials are missing. See http://twitter4j.org/configuration.html for the detail.
at twitter4j.TwitterBaseImpl.ensureAuthorizationEnabled(TwitterBaseImpl.java:200)
at twitter4j.TwitterImpl.get(TwitterImpl.java:1833)
at twitter4j.TwitterImpl.search(TwitterImpl.java:282)
at searchtweet.search(searchtweet.java:38)
java.lang.IllegalStateException: Authentication credentials are missing. See http://twitter4j.org/configuration.html for the detail.
at twitter4j.TwitterBaseImpl.ensureAuthorizationEnabled(TwitterBaseImpl.java:200)
at twitter4j.TwitterImpl.get(TwitterImpl.java:1833)
at twitter4j.TwitterImpl.search(TwitterImpl.java:282)
at searchtweet.search(searchtweet.java:38)
java.lang.IllegalStateException: Authentication credentials are missing. See http://twitter4j.org/configuration.html for the detail.
at twitter4j.TwitterBaseImpl.ensureAuthorizationEnabled(TwitterBaseImpl.java:200)
at twitter4j.TwitterImpl.get(TwitterImpl.java:1833)
at twitter4j.TwitterImpl.search(TwitterImpl.java:282)
at searchtweet.search(searchtweet.java:38)


February 2, 2014 | 12:47 pm

Yeah something must have changed with the API, it’s not working for me anymore either. It’s a shame, I was really hoping to utilize it in my thesis project for school.


February 26, 2014 | 8:11 am

Try downloading the latest version of Twitter4J (http://twitter4j.org/en/index.html) and replace the old by the new in your java folder, it works again for me !


February 26, 2014 | 5:05 pm

Is this still working? I keep getting an error in my max window even after changing the Twitter4J java files to the new ones.


February 27, 2014 | 2:22 am

It’s working for me but indeed you have some warnings from log4j the first time you run it.


March 21, 2014 | 7:39 pm

Just updated the pack with the new twit4j libraries. Tested in Max5 and seems to be working fine again. NNMAR what are the error messages you are getting?


March 23, 2014 | 1:17 pm

Hi Gokce, thanks for your work on this. I get a ‘Could not load class ‘searchtweet’ -error. Is there suppose to be a searchtweet.class in the zip file? You have it in the maxobjects download, but i guess that is the old one that doesn’t work anymore? Thanks again.


March 23, 2014 | 2:45 pm

Thanks GOKCE, when I do the first search, I get log4j:WARN No appenders could be found for logger (twitter4j.internal.http.HttpClientlmpl)

Then it goes on with Please initialize the log4j system properly and see the logging.apache.org/log4j/1.2/faq.html#noconfig for more info.


April 1, 2014 | 5:26 am

The readme says "copy the searchtweet.class under [maxmsp]/Cycling ’74/java/classes folder"… there is no searchtweet.class file included in the current zip file?


April 1, 2014 | 12:13 pm

Thanks Matthew, I must have erased the class file accidentally while rushing the update. Now all files are back in place.


April 1, 2014 | 1:07 pm

Works great now, thanks!


April 1, 2014 | 6:05 pm

Thank you for the update!


May 18, 2014 | 10:21 am

Hey ! I have followed all the steps and tried to run the patch: it opens, but when I search for something, I get this error:
java.lang.NoClassDefFoundError: com/google/appengine/api/urlfetch/HTTPRequest
at twitter4j.AlternativeHttpClientImpl.handleRequest(AlternativeHttpClientImpl.java:43)
at twitter4j.HttpClientBase.request(HttpClientBase.java:53)
at twitter4j.HttpClientBase.get(HttpClientBase.java:71)
at twitter4j.TwitterImpl.get(TwitterImpl.java:1968)
at twitter4j.TwitterImpl.search(TwitterImpl.java:293)
at searchtweet.search(searchtweet.java:48)
Do you know what this means? It’s probably something really stupid that I’m missing, but I’m just starting out and I could really use some help.
Thanks!


July 28, 2014 | 5:04 pm

Hi there, I’ve installed the latest version of twitter4j (4.0.2) and replaced the files in the java>lib folder.. I still get all these errors:

sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Relevant discussions can be found on the Internet at:
http://www.google.co.jp/search?q=506c3b98 or

http://www.google.co.jp/search?q=105d1087

TwitterException{exceptionCode=[506c3b98-105d1087 63e3f388-fb44fc20 63e3f388-fb44fc20 63e3f388-fb44fc20], statusCode=-1, message=null, code=-1, retryAfter=-1, rateLimitStatus=null, version=4.0.2}
at twitter4j.HttpClientImpl.handleRequest(HttpClientImpl.java:178)
at twitter4j.HttpClientBase.request(HttpClientBase.java:53)
at twitter4j.HttpClientBase.get(HttpClientBase.java:71)
at twitter4j.TwitterImpl.get(TwitterImpl.java:1556)
at twitter4j.TwitterImpl.search(TwitterImpl.java:247)
at searchtweet.search(searchtweet.java:48)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid c
ertification path to requested target
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Unknown Source)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(Unknown Source)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.HttpURLConnection.getResponseCode(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
at twitter4j.HttpResponseImpl.<init>(HttpResponseImpl.java:35)
at twitter4j.HttpClientImpl.handleRequest(HttpClientImpl.java:142)
… 5 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(Unknown Source)
at sun.security.validator.PKIXValidator.engineValidate(Unknown Source)
at sun.security.validator.Validator.validate(Unknown Source)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.validate(Unknown Source)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
… 20 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(Unknown Source)
at java.security.cert.CertPathBuilder.build(Unknown Source)
… 26 more
Failed to search tweets: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

I haven’t seen anyone post about these specific errors here.
Thanks to anyone who can help me with this!


July 28, 2014 | 5:04 pm

Hi there, I’ve installed the latest version of twitter4j (4.0.2) and replaced the files in the java>lib folder.. I still get all these errors:

sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Relevant discussions can be found on the Internet at:
http://www.google.co.jp/search?q=506c3b98 or

http://www.google.co.jp/search?q=105d1087

TwitterException{exceptionCode=[506c3b98-105d1087 63e3f388-fb44fc20 63e3f388-fb44fc20 63e3f388-fb44fc20], statusCode=-1, message=null, code=-1, retryAfter=-1, rateLimitStatus=null, version=4.0.2}
at twitter4j.HttpClientImpl.handleRequest(HttpClientImpl.java:178)
at twitter4j.HttpClientBase.request(HttpClientBase.java:53)
at twitter4j.HttpClientBase.get(HttpClientBase.java:71)
at twitter4j.TwitterImpl.get(TwitterImpl.java:1556)
at twitter4j.TwitterImpl.search(TwitterImpl.java:247)
at searchtweet.search(searchtweet.java:48)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid c
ertification path to requested target
at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Unknown Source)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(Unknown Source)
at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.HttpURLConnection.getResponseCode(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
at twitter4j.HttpResponseImpl.<init>(HttpResponseImpl.java:35)
at twitter4j.HttpClientImpl.handleRequest(HttpClientImpl.java:142)
… 5 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(Unknown Source)
at sun.security.validator.PKIXValidator.engineValidate(Unknown Source)
at sun.security.validator.Validator.validate(Unknown Source)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.validate(Unknown Source)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
at com.sun.net.ssl.internal.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
… 20 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(Unknown Source)
at java.security.cert.CertPathBuilder.build(Unknown Source)
… 26 more
Failed to search tweets: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

I haven’t seen anyone post about these specific errors here.
Thanks to anyone who can help me with this!


September 27, 2014 | 1:53 pm

Hi Gokce, first of all thank you so much for the searchtweet object! It is the backbone of my duo "Twittercore", we would have missed a lot of fun without it :) We are using it since 2012 for our performances, working great so far.

My questions is: We performed tonight and after one set got a 429 error from the Twitter API in the max window. Couldn’t search for tweets any more. Waited for some time, no luck. Tried again later from a different IP address and it seemed to work again. We also performed yesterday for way longer then today without problems. If I understood correctly twitter allows 180 requests in a window of 15 min now and I’m very sure we didn’t even come close to a 100.
Do you have an idea what that could be? Does searchtweet function like a regular user in the eyes of the API and can we get banned? How about two people using the object at the same moment on different locations, do their requests count like they are coming from the same person? Say both do a 100 in 15 min, does that count as 200 then for twitter because we use the same object in max?
I’m using max 5.1.9 / searchtweet 1.1b, twitter4j 4.0.1
I’m on a mac, OS 10.9.4

We have quite a big gig in Cape Town next week (we live in Amsterdam, Netherlands) and are pretty nervous about running into problems suddenly.
Thank’s again an hope you can help us!



Spa
October 15, 2014 | 9:01 pm

hi Gokce,
would you mind releasing the source so it can be implemented with more items.
Thanks


Viewing 41 posts - 1 through 41 (of 41 total)