i am working with a seismic lab who updates a http site regularly with data.

i'd like to 'grab' this data and turn into a stream that could then be accessed and used in a live sound performance with M4L.

i have no idea how to go about doing something like this, any tips/tricks and/or advice would be incredibly useful.

i am working with a seismic lab who updates a http site regularly with data.
i'd like to 'grab' this data and turn into a stream that could then be accessed and used in a live sound performance with M4L.

i have no idea how to go about doing something like this, any tips/tricks and/or advice would be incredibly useful. 

possible-use-maxurl-m4l

Even though I haven't done it yet, there is no reason why [maxurl] should not work in M4L. Best would be if the lab offers a RESTful (json or xml) API to access the data, that would make the parsing much easier. What's the url of that lab/site?

hey there, I appreciate your time, unfortunately at this point I do not yet know the http site. I am preparing for a project in Santorini, and still working out with my institution who to contact there. Just trying to gather momentum before jumping in.

however, I would find it really helpful if you could explain what a RESTful (json xml) API is, and why this might be easier to grab data from.

Another question I have for you (and/or others interested), would be to ask how one might go about turning the [maxurl] bang function into something more like a stream (i.e. a fluid dynamic state of data rather than the one punch/click update)...

hey there, I appreciate your time, unfortunately at this point I do not yet know the http site. I am preparing for a project in Santorini, and still working out with my institution who to contact there. Just trying to gather momentum before jumping in. 

however, I would find it really helpful if you could explain what a RESTful (json xml) API is, and why this might be easier to grab data from. 

Another question I have for you (and/or others interested), would be to ask how one might go about turning the [maxurl] bang function into something more like a stream (i.e. a fluid dynamic state of data rather than the one punch/click update)... 

A REST (or RESTful) API is a standardized ways to interact with web-application (i.e. getting or sending data) without that a browser UI is necessary. You send normal http GET or POST request to the server but instead of a full web page you will get back JSON or XML data only. It's also called a web service sometimes. [maxurl] can parse those responses directly into [dict]. This way you have direct access to the content.

Check out the parsing tab in the [maxurl] help file.

In the leftmost example they use the REST API of openweathermap.org. If you want to see what the response looks like just put the used request 

http://api.openweathermap.org/data/2.5/weather?id=5391959&mode=json

If the data is only available via a "normal" web site, you will have to analyze the html code returned and look for reliable patterns that you can use to parse out your data. But as soon as i.e. the site layout changes you are fried, need to identify those changes manually and adapt your parsing algorithm.

For your second question, what you are speaking about is data pushing instead of polling. Meaning that as soon as there are new data on the server they are send to the client (like in a chat). Pushing is not trivial with http as the protocol is not made for that. Chat sites basically all work with a "hack" to realize that. In any case everything depends on your data providing server. As long as you don't have any details there is no way to say if there is a way to realize that. (Most likely it is not without some not sleek workarounds. At some point you will have to poll for data - i.e. setting up your own server that polls the data scans for changes and only then pushes that to Max).

The best way is probably to do the pulling i.e. with metro every few seconds within your patch. For performance reasons you might also want to do the data polling outside Max4Live in a regular Max patch and i.e. send the data to M4L via [udpsend/reveive].

In the leftmost example they use the REST API of openweathermap.org. If you want to see what the response looks like just put the used request http://api.openweathermap.org/data/2.5/weather?id=5391959&mode=json directly into the browser.

If the data is only available via a "normal" web site, you will have to analyze the html code returned and look for reliable patterns that you can use to parse out your data. But as soon as i.e. the site layout changes you are fried, need to identify those changes manually and adapt your parsing algorithm. 

For your second question, what you are speaking about is data pushing instead of polling. Meaning that as soon as there are new data on the server they are send to the client (like in a chat). Pushing is not trivial with http as the protocol is not made for that. Chat sites basically all work with a "hack" to realize that. In any case everything depends on your data providing server. As long as you don't have any details there is no way to say if there is a way to realize that. (Most likely it is not without some not sleek workarounds. At some point you will have to poll for data - i.e. setting up your own server that polls the data scans for changes and only then pushes that to Max). 

The best way is probably to do the pulling i.e. with metro every few seconds within your patch. For performance reasons you might also want to do the data polling outside Max4Live in a regular Max patch and i.e. send the data to M4L via [udpsend/reveive]. 

Jan, thank you for this information! This is really helpful, and sorry about the delay...

metro is a great asset for this project, hopefully the http site the seismic lab will offer JSON / MXL DATA rather than function as a 'regular' site.

One thing that is still unclear to me is the parsing process -- how exactly does the dict. function work with the [MAXURL] object to fetch particular datalines in the http code? and how might one go about re-directing this toward other information within the HTTP code.

Perhaps you could help me understand some objects in the MAXURL program 'Weather Report', and their order.

Below the BANG there is a DICT. object (is this where we designate the URL for the data we're interested in..?) beneath that there is the [MAXURL] object then a trigger object [t b l], (function slightly vague to me)... then there is a MESSAGE. object, followed by another DICT, then a ROUTE object -- then the DICT.UNPACK.

If you could walk through these with a brief description of how they work that would be incredible (also open to any other way to understanding this lineup) THANK YOU!

Jan, thank you for this information! This is really helpful, and sorry about the delay... 

metro is a great asset for this project, hopefully the http site the seismic lab will offer JSON / MXL DATA rather than function as a 'regular' site. 

Perhaps you could help me understand some objects in the MAXURL program 'Weather Report', and their order. 

Below the BANG there is a DICT. object (is this where we designate the URL for the data we're interested in..?) beneath that there is the [MAXURL] object then a trigger object [t b l], (function slightly vague to me)... then there is a MESSAGE. object, followed by another DICT, then a ROUTE object -- then the DICT.UNPACK. 

If you could walk through these with a brief description of how they work that would be incredible (also open to any other way to understanding this lineup) THANK YOU! 

IS IT POSSIBLE TO USE MAXURL IN M4L?