Very interesting Dave - this means a lot coming from you. So I'm curious, not fully understanding the technology yet, what are the benefits of this over hubbub? Also, have you looked at Twitter's proprietary real-time feeds? I'm curious how this compares to that as well - I'd love to see them consider a more open approach that can scale like this. I mentioned hubbub to them and they said it wasn't scalable.
Mark Essel
· 3 months ago
I think hubbub is designed to be scalable. Maybe they meant they couldn't own all the real time streams? The origin of this setup goes pretty far back according to Dave, and I think his familiarity (or refreshed look) was a natural enough reason to develop it further. It could likely be the communication/broadcast method many users go to for open social media.
Mark Essel
· 5 months ago
Just chiming in as a network noob. How does this effort differ functionally from a federated publisher/hub setup like pubsubhubbub? I like the sound of it , just not 100% clear on the concept.
infeeds
· 3 months ago
I think it's the same things but with different name :)
Daniel
· 5 months ago
It's a great name. Look forward to seeing what you come up with!
dave
· 5 months ago
Thanks! If it works, it'll be a bootstrap. That means at first the results will be a few sites pinging each other and updating in realtime. If it gains traction, it'll get support from a lot of tech and media companies. It could happen very quickly.
One thing's for sure -- we'll need implementations in every language and runtime. I'm doing mine in the OPML Editor, of course. But just think of that as a reference implementation. I think the really scalable versions will be in Python or C.
Also thinking about using Amazon's SimpleDB to store the graph. We'll see...
ninjamonk
· 5 months ago
Mason Lee
· 5 months ago
Bravo!
Are you able to think of a good way to syntactically distinguish 140-char RSS feeds from the more popular "news article" style feeds? This would help for getting the LifeLiner feed into the right style reader client (provided you agree that there's no good one-size fits all reader, and sorting feed "styles" manually into different readers is a problem).
I would suggest having LifeLiner put the 140 chars in the RSS "title" element rather than description, only because an Atom version is going to require "title".
Looking forward to this.
dave
· 5 months ago
Ahhh. These are good ideas. Please stay in the loop to help guide the process. Dave
mwendell
· 3 months ago
From a usability standpoint, putting the 140 characters of a Tweet into the title element is a bad idea. Typically on the display side, the title is usually an anchor, jumping back to the original article. If there's a link in the tweet (as is often the case), you can't click it until you've jumped back to the original article.
Of course you could put the 140 chars into both the title and the body (which is, I think, what Twitter is doing now, finally). While that works, it seems somewhat inelegant.
Maybe some kind of smart filter to strip things like RT@'s and URLs from the tweet, and use the result as the title, displaying the full content in the body?
anildash
· 5 months ago
Just curious -- what platform are you creating LifeLiner in? Is that part of the code you'll be sharing? (Not asking because you have any obligation to, just want to avoid duplicating effort.)
dave
· 5 months ago
OPML Editor.
And yes, I will be sharing the code.
And it would be great if others would implement the equivalent app in other environments.
From the output side it will just be putting out RSS with a <cloud> element.
It's the equivalent of Tweetie or Tweetdeck or Seesmic.
Dave
mterenzio
· 5 months ago
Cool. I was working on implementing it before the PubSubHubbub mini-explosion so I'm just about ready to go with a PHP on AWS version. Should scale infinitely. If anyone else has an implementation, let me know at my gmail 'mterenzio' or twiiter 'mterenzio' and we can start bouncing pubsubs back and forth.
dave
· 5 months ago
That's great. Want me to point a rsscloud.org subdomain at your server?
matt.rsscloud.org?
mterenzio
· 5 months ago
I'm honored , with all the famous Matts around. ; ) Let me get it this out on an EC2 server first. Totally untested hack, but I guess that's the point.
One thing I wonder about in the spec. If a Cloud changes it's IP address. . .well, can it? Oh wait, the cloud is identified by it's domain. The subscriber, though, can't change because it's identity IS it's IP address, right?
That's important to know. So a subscriber that had to change IP addresses would have to resubscribe to all it's subscriptions. Right?
dave
· 5 months ago
The subscriber has to resub every 24 hours anyway because the subscriptions lapse after 25 hours.
So the longest you'll have a bad IP address around is 24 hours, 59 minutes and 59 seconds.
mterenzio
· 5 months ago
Right. I forgot about that, though I think it does say that it's by convention, so I took it to mean I COULD subscribe one indefinitely.
Okay, so that's better. No unsubscribe method or anything, just store and update the time of subscription. Got it.
dave
· 5 months ago
I don't know about you but I'm a very conventional guy. :-)
daniel
· 5 months ago
Is there any way to use REST as a protocol in the cloud tag? The docs say "a string indicating which protocol to use (xml-rpc or soap, case-sensitive)" which implies that they are the only valid protocols? If so, so. Just thinking about a possible implementation...
dave
· 5 months ago
Actually the SOAP Meets RSS doc says "http-post" is an acceptable value for protocol.
"A has five required attributes: domain is the domain name or IP address of the cloud, port is the TCP port that the cloud is running on, path is the location of its responder, registerProcedure is the name of the procedure to call to request notification, and protocol is xml-rpc, soap or http-post (case-sensitive), indicating which protocol is to be used."
I'm working on an internal project, but there's no reason to think that bits couldn't be released or talked about. I need to relearn how all this was implemented in Manila and Radio Userland. My memory is *not* that good ;)
dave
· 5 months ago
Took me a bit of puzzling to put it back together. It'll be easier for the next guys because you'll have pieces to test with that are already up and running.
dave
· 5 months ago
I've got the HTTP-POST interface working in my cloud.
Excellent and I hope this pans out. Is there an RSS feed for this site so we can stay updated?
germanstudent
· 5 months ago
This would be so cool! I'm looking forward it, too
mterenzio
· 5 months ago
Regarding using "http-post" as the protocol, it's clear that for the notifications of updates that there is one parameter named "url".
But considering a cloud element that is accepting subscription requests via "http-post" is there a standard for naming the required parameters? My guess:
registerProcedure port path protocol url
mterenzio
· 5 months ago
Should an aggregator only register callbacks in the same protocol that a cloud advertises? In which case the registerProcedure above might not be necessary, nor the protocol.
dave
· 5 months ago
Good question! I'm just about to answer it in the walkthrough doc I'm writing today.
Cool stuff, Dave! We're trying to build a really great RSS featuires into Grabbit, so we'll do our best to support this early on.
invoker
· 3 months ago
Congrats on the wordpress support. Looks like a great project.
Bubbo
· 3 months ago
Thanks to @jesse for mentioning the RSS Cloud on Facebook. I'm all setup for the cloud with my self-hosted WordPress install. Great job integrating with WP so quickly. This looks to be the most promising move towards real-time feed and content syndication I've seen.
Cheers to Matthew Terenzio for securing the @RSScloud on Twitter.
Tane Piper
· 3 months ago
Is there an issue installing River2 at the moment? I'm on Win7 and I followed the instructions from RssCloud, however after I've received the update I try open the Misc -> Tools Catalogue menu but nothing seems to open :/
ShanaC
· 3 months ago
Mazel Tov on the Wordpress support
thingles
· 3 months ago
Question for everyone: Is there any reason why RSS Cloud and PubSubHubbub could not live on the same feed at the same time? I have had the PubSubHubbub WordPress plugin for a while, and just added the RSS Cloud one. I see this in my feed.
Now, I'm thinking that cloud servers will see their part and ignore the PubSubHubbub stuff and vice versa. Make sense?
Or, does this thematically just not make sense? Is it one or the other? I see no reason why that would be the case. Comments?
croldham
· 3 months ago
Dave,
OK, I've read all your documentation and I'm trying to wrap my head around this. So is one of the major goals to be able to build aggregators that can consume "push" messages from rssCloud producers such that microblogging services like Twitter just become another producer? And "following" someone on Twitter would become subscribing to the "push" service pointed to by the <cloud> tag? If so, it seems that one of the value-adds that Twitter provides is the knowledge of who is following you. Have I missed how this information can be transmitted back to Twitter through rssCloud? And what about other features like blocking someone?
bibliacatolica
· 3 months ago
teste
jvenema
· 3 months ago
Hey Dave, I wonder if we could figure out a nice way to integrate what you've got going here with our WebSync, a push-based web client... (http://www.frozenmountain.com/websync).
Ping me when you have a free moment sometime, I'd love to discuss it with you.
BuzzEins
· 3 months ago
nice!
swhitley
· 3 months ago
I ran into a minor issue on a shared webhost. I created a PHP script to call pleaseNotify. The IP address in my request is not the same as the IP address for my web server. Many other people will likely have this problem, even though the script is on an Internet server.
It would be nice to be able to provide a domain to pleaseNotify so that I could fully control the notification. I understand the verification issues. Could a secret also be supplied to pleaseNotify? rssCloud could perform at GET against my web server using the secret. I'd create a file with the same name as the secret on my server. Just a thought.
I was also reading some of the critiques of rssCloud, one of them being that the entire feed will need to be read by the aggregator. Couldn't this be solved by making our feeds smarter? If we provide a guid or pubDate to the feed, the feed could return the latest items instead of the full feed.
mediaista
· 3 months ago
I totally thought RSS already was a 'push' technology. Kinda surprised that services are pinging for updates. So glad to hear that you are taking the bull by the horns on this, Dave.
Guest
· 3 months ago
Trying to run: http://howto.disqus.com/howto_rebooting_the_rss... (this page Subscribe by RSS link) through River2: Subscriptions feed to subscribe. Comes back with the following error: "Can't get the address of "guid" because the table doesn't have an object with that name."
drewryonline
· 3 months ago
You are a talented person for making this plug in. Good looking out my dude :-)
I am a designer and working with developers is my next step forward. But for now this is WAY over my head,
Ido
· 3 months ago
Very Cool! How does it work with Yahoo! Pipes?
ibldtraffic
· 3 months ago
Way over my head as well, however, as a multiple blog owner I need, publish, acquire and syndicate content- sometimes hourly, but at minimum daily. I'll get my head around this if have to learn it a word at a time. If any developer needs a live guinea pig...
seo india
· 3 months ago
Hi Dave, it's very interesting but things is not very clear. this is very new technology which recognized by WordPress, it would be very helpful if you explain all the things by help of screenshot/images.
trust me, new bloggers would like this way and understand your explanation.
Bob Olwig
· 2 months ago
Dave, I enjoy your work (going back to Cluetrain) and subscribe to your blog, however, I'm afraid you're jumping on the "let's-call-everything-CLOUD" bandwagon. Just because a technology or implementation of the technology or, even, the benefits of a technology may be similar to cloud computing doesn't make it cloud computing. Best Regards, Bob
dave
· 2 months ago
You should read up on it -- it was called "cloud" in 2001 -- when the element was added to RSS.
doke01
· 2 months ago
Why should I use this rather then pubsubhubbub? I am trying to decide which is the best but it is not altogether clear to me.
pwestbro
· 2 months ago
I think that the ping that the aggregators receive may need another parameter. Currently, the only parameter is the url of the feed that is updated.
The problem arises during the periodic resubscription requests. It appears that the "clouds" send an update when attempting to resubscribe. Aggregators could ignore all updates during a resubscription, but this could cause some valid updates to be ignored.
It would be great to have an additional parameter that would indicate that the "update" is just validation of a subscription request.
Career Target
· 3 weeks ago
This is a good improvement. I also like to hear this one .
All Comments
http://rpc.rsscloud.org:5337/rsscloud/viewLog
One thing's for sure -- we'll need implementations in every language and runtime. I'm doing mine in the OPML Editor, of course. But just think of that as a reference implementation. I think the really scalable versions will be in Python or C.
Also thinking about using Amazon's SimpleDB to store the graph. We'll see...
Are you able to think of a good way to syntactically distinguish 140-char RSS feeds from the more popular "news article" style feeds? This would help for getting the LifeLiner feed into the right style reader client (provided you agree that there's no good one-size fits all reader, and sorting feed "styles" manually into different readers is a problem).
I would suggest having LifeLiner put the 140 chars in the RSS "title" element rather than description, only because an Atom version is going to require "title".
Looking forward to this.
process. Dave
Of course you could put the 140 chars into both the title and the body (which is, I think, what Twitter is doing now, finally). While that works, it seems somewhat inelegant.
Maybe some kind of smart filter to strip things like RT@'s and URLs from the tweet, and use the result as the title, displaying the full content in the body?
And yes, I will be sharing the code.
And it would be great if others would implement the equivalent app in other
environments.
From the output side it will just be putting out RSS with a <cloud> element.
It's the equivalent of Tweetie or Tweetdeck or Seesmic.
Dave
matt.rsscloud.org?
One thing I wonder about in the spec. If a Cloud changes it's IP address. . .well, can it? Oh wait, the cloud is identified by it's domain. The subscriber, though, can't change because it's identity IS it's IP address, right?
That's important to know. So a subscriber that had to change IP addresses would have to resubscribe to all it's subscriptions. Right?
lapse after 25 hours.
http://www.thetwowayweb.com/soapmeetsrss#rssclo...
So the longest you'll have a bad IP address around is 24 hours, 59 minutes
and 59 seconds.
Okay, so that's better. No unsubscribe method or anything, just store and update the time of subscription. Got it.
protocol.
http://cyber.law.harvard.edu/rss/soapMeetsRss.h...
"A has five required attributes: domain is the domain name or IP address of
the cloud, port is the TCP port that the cloud is running on, path is the
location of its responder, registerProcedure is the name of the procedure to
call to request notification, and protocol is xml-rpc, soap or http-post
(case-sensitive), indicating which protocol is to be used."
As does the RSS 2.0 spec.
http://cyber.law.harvard.edu/rss/rss.html#ltclo...
"It specifies a web service that supports the rssCloud interface which can
be implemented in HTTP-POST, XML-RPC or SOAP 1.1. "
I hope we can work together again...
I'm working on an internal project, but there's no reason to think that bits couldn't be released or talked about. I need to relearn how all this was implemented in Manila and Radio Userland. My memory is *not* that good ;)
next guys because you'll have pieces to test with that are already up and
running.
http://rsscloud.org/#httppostProtocol
Dave
But considering a cloud element that is accepting subscription requests via "http-post" is there a standard for naming the required parameters? My guess:
registerProcedure
port
path
protocol
url
writing today.
http://rsscloud.org/walkthrough.html
Dave
Cheers to Matthew Terenzio for securing the @RSScloud on Twitter.
<atom:link rel="hub" href="http://pubsubhubbub.appspot.com"/><atom:link rel="hub" href="http://superfeedr.com/hubbub"/><cloud domain='things.thingelstad.com' port='80' path='/?rsscloud=notify' registerProcedure='' protocol='http-post' />Now, I'm thinking that cloud servers will see their part and ignore the PubSubHubbub stuff and vice versa. Make sense?
Or, does this thematically just not make sense? Is it one or the other? I see no reason why that would be the case. Comments?
OK, I've read all your documentation and I'm trying to wrap my head around this. So is one of the major goals to be able to build aggregators that can consume "push" messages from rssCloud producers such that microblogging services like Twitter just become another producer? And "following" someone on Twitter would become subscribing to the "push" service pointed to by the <cloud> tag? If so, it seems that one of the value-adds that Twitter provides is the knowledge of who is following you. Have I missed how this information can be transmitted back to Twitter through rssCloud? And what about other features like blocking someone?
Ping me when you have a free moment sometime, I'd love to discuss it with you.
It would be nice to be able to provide a domain to pleaseNotify so that I could fully control the notification. I understand the verification issues. Could a secret also be supplied to pleaseNotify? rssCloud could perform at GET against my web server using the secret. I'd create a file with the same name as the secret on my server. Just a thought.
Here's my experience with rssCloud: http://www.voiceoftech.com/swhitley/?p=773
I was also reading some of the critiques of rssCloud, one of them being that the entire feed will need to be read by the aggregator. Couldn't this be solved by making our feeds smarter? If we provide a guid or pubDate to the feed, the feed could return the latest items instead of the full feed.
http://howto.disqus.com/howto_rebooting_the_rss... (this page Subscribe by RSS link)
through River2: Subscriptions feed to subscribe.
Comes back with the following error:
"Can't get the address of "guid" because the table doesn't have an object with that name."
http://www.ShawnDrewry.com
How does it work with Yahoo! Pipes?
trust me, new bloggers would like this way and understand your explanation.
The problem arises during the periodic resubscription requests. It appears that the "clouds" send an update when attempting to resubscribe. Aggregators could ignore all updates during a resubscription, but this could cause some valid updates to be ignored.
It would be great to have an additional parameter that would indicate that the "update" is just validation of a subscription request.