OStatus

OStatus is, as described on its official site:

an open standard for distributed status updates. [The] goal is a specification that allows different messaging hubs to route status updates between users in near-real-time-

Important Links

Process and work products

Process based on: http://ostatus.org/2010/10/04/how-ostatus-enable-your-application

Activity Streams, can be broken into:
- Provide Atom feed
- Transform Atom feed into activity stream
PubSubHubbub Publish, can be broken into:
- Decide on and declare hub in user activity streams
- Publish to hub when activity entries are created
WebFinger, can be broken into:
- Provide host-meta file for site with user account template url
- Provide account information for specific users

When a federated site has PUSH-enabled ActivityStreams, users on remote services can subscribe to users on the federated site and be notified when their status updates.

ActivityStreams

ActivityStreams is a standardized way to represent activities of individual users on social sites. It is nothing more than an atom feed with additions to further categorize the activity entries. Each activity entry does something to an activity object.

This is the core of the social interactions which the OStatus standard makes distributable across multiple federated sites.

How to Implement

For an example use case, lets say that we have a running MediaGoblin instance on mediagoblin.server.com and we want to provide an ActivityStream for a user with a username yooser at a specific location, e.g. http://mediagoblin.server.com/u/yooser/as.atom

The first thing we have to do is create an Atom feed for yooser. This is relatively simple to do in Django and shouldn't be much of a problem in MediaGoblin's framework. We must keep in mind that we will have to expand the feed with additional information. We should make the atom feed discoverable using a link from the user's profile page: <link rel="alternate" href="http://mediagoblin.server.com/u/yooser/as.atom" type="application/atom+xml" title="Atom feed for yooser"/>

Next step is to choose what a user can do and which objects the user can do it onto. This is a really big deal and should be carefully chosen. The implementation of what gets chosen should be done in such a way that it enables easy additions to the entries and objects.

So, lets start with the activity objects of the system. There are a lot of them to choose from and not all of them have to be implemented (for example, why should MediaGoblin do anything about article, i.e. blog posts). Here's the complete list of object types according to the Activity Base Schema Draft (yes, it's not final):

Article - A lot of paragraphs of text which might incorporate images and stuff
Audio - File containing songs, speech or just about anything somebody can listen to
Badge - Some sort of an award which can be granted to somebody
Bookmark - Basically just a link to a website, called bookmark to avoid confusion
Collection - A generic container for a lot of objects, like an album of photos or folders of files
Comment - A text response to another object
Event - Description of a "happening" which people can attend at some location at a specific time
File - A generic document which cannot be categorized as anything specific (like Audio, Image or Video)
Group - Similar to collection but allows member objects to join or leave
Image - File containing something stationary which can be perceived through eye (or something else)
Note - A dent, tweet, status update or any other form of short, only content article-ish object
Person - A user on the system (or on another system) - core thing really
Place - The draft says it best: 'The "place" object type represents a location on Earth.'
Product - Some object or act which people sell or offer in return for something, like a book
Question - A way to get opinion from others. A poll.
Review - A (hopefully) well formed written opinion about what somebody thinks about something
Service - Not to be confused with service as in product but a blog or something that services others
Video - A file containing a lot of images in sequence, and in most cases combined with audio

There are also some extensions to this base schema described in the draft. These extensions can be added to the other objects but don't describe anything purely on their own (well they do but theoretically let's just say that it wouldn't make sense):

Location - A place object which describes where some other object is at
Mood - An object which describes in what mood the creator was or a person is in
Rating - A number between 1.0 and 5.0 (with one decimal place)... think stars!
Source - The original place where something came from, don't go all the way back to The Big Bang!
Tags - A list of objects which have been marked as a part of something, like a person in a picture

The same schema defines what can be done with these objects.

add - something is being added to something (a photo to an album)
cancel - something is cancelled (an event)
checkin - something/someone has arrived at a place
delete - something has been deleted (does not necessarily mean permanent destruction)
favorite - something/someone really, really likes something (favorite video/status)
follow - someone/something started following the activity stream of someone/something
give - someone/something gives something (like a badge) to someone/something
ignore - someone/something ignores something
invite - someone/something invites someone/something to something
join - someone/something joins something
leave - inverse of join or the inverse of checkin (someone/something leaves something)
like - another form of favorite
make-friend - a verb that says friendship has been formed
play - something is played (a video)
post - creation of an object
receive - someone or something has received something (like a badge)
remove - something is removed from something (a photo from an album)
remove-friend - friendship has been discontinued
request-friend - someone wants to become friends with someone
rsvp-maybe - someone might attend something (an event)
rsvp-no - someone won't attend something (an event)
rsvp-yes - someone will attend something (an event)
save - someone has saved something out of personal interest
share - something is shared with others
stop-following - someone/something has stopped following the activity stream of something
tag - someone/something is said to be present in something
unfavorite - something is not as awesome as it once was
unlike - something is not as ok as it once was
unsave - something is not being used for personal interest anymore
update - something has been updated (a blog post)

So, now we know what we can do and what we can do it onto. For MediaGoblin we should choose a subset of these objects and activity verbs. Let's start with the objects. MediaGoblin is a site primarily to be used for sharing and socializing around media. As such we need media objects and a few other objects make these media objects sociable. Of course MediaGoblin could in the future incorporate other activity objects (like organizing events or give out badges) but let's restrict us to pure media socializing.

Firstly, we need the core thing... the user:

Person - A user of the system represented as an activity object

Secondly, these are the media types:

Image - Users should be able to upload and view photos and other images
Audio - Users should be able to upload and listen to audio files
Video - Users should be able to upload and watch videos
File - Users should be able to upload and download other files (e.g. README documents, books)

Lastly, we need to allow other object types to make MediaGoblin social:

Collection - To create an album of videos or images (thus allowing categorization)
Group - To allow users to find users and media more easily (find and categorize people/media objects)
Comment - To allow users to comment on media files (their own or uploaded by others)
Place - To describe a location (a place shown in an image)

We should also allow a few extensions to the base schema:

Location - The reason we allowed Place
Source - To attribute the right author
Tags - To tag a place or a person in a media object

Now we need to decide what verbs the person (actor) can perform. Regarding favorite or like, we should go with favorite to avoid any confusion around Facebook. The following list also assumes a Status.Net approach (anybody can follow anybody, instead of creating friendship). It also does not include play, just because I (the one who wrote this) doesn't like other people knowing what I'm doing in such detail.

post, delete - To create or destroy media objects (delete should permanently delete stuff)
add, remove - To add or remove media objects from albums (collection)
favorite, unfavorite - To allow other persons to see what one person thinks is awesome
follow, stop-following - To allow persons to add other persons to their stream
join, leave - To allow someone to join/leave a group about some specific topic
share - To allow a person to redistribute a media object from another person on that person's stream
tag - To allow a person to say that another person is in a media object

OK. So how do we do it? Each and every activity object described above includes specific information which is only used for that specific object. One way to form the objects is to create an activity object model class (e.g. ActivityConstruct) which can be subclassed as a specific activity object. Each subclassed activity object then has a method (e.g. def ostatus(self)) which can be called to get the object type and the corresponding information. The activity itself (using a verb on an object) is then collected in another model, e.g. ActivityEntry which forms the timeline (the activity stream). Defining the ActivityEntry model in Django would then be something like:

class ActivityEntry(models.Model):
    atom_id = models.CharField(max_length=256, editable=False) # unique id, using uuid1()
    actor = models.ForeignKey('Person', related_name='actors') # 
    activityobject = models.ForeignKey('ActivityConstruct', related_name='activity_objects')
    target = models.ForeignKey('ActivityConstruct', blank=True, null=True, related_name='activity_targets')
    verb = models.CharField(max_length=128)
    time = models.DateTimeField(auto_now_add=True)
    title = models.CharField(max_length=256) # Short description (usually same as summary)
    summary = models.TextField() # <person> <verb>ed <activityobject> (in <target>)

The problem with this approach (i.e. in Django) is how to access the subclasses since the activity entry points to the parent class. It's messy but is still a good way to achieve this. MediaGoblin's framework might be able to solve this in a better way.

All right. So, how do we show it? Well, we take our Atom feed (which would represent the timeline of activity entries) and append stuff to it. For a regular atom feed an <entry> element consists of the following:

<entry>
  <id>urn:uuid:11bba138-0eb3-11e1-ba8a-13b887ad42f8</id>
  <title>yooser posted a video</title>
  <link rel="alternate" type="text/html" href="http://mediagoblin.server.com/u/yooser/gallery/himom.ogv"/>
  <link rel="self" type="application/atom+xml" href="http://mediagoblin.server.com/u/yooser/111113112602.atom"/>
  <published>2011-11-13T11:26:02Z</published>
  <summary>yooser just uploaded a brand new video</summary>
</entry>

Adding activity stream stuff is just about adding the relevant information. Video information is for example: author (can be omitted if the author is the actor), displayName (title of video), embedCode (code for HTML embedding), id (unique id for video), published (optional time when published), stream (media link to video, summary (optional summary of video), updated (optional time when updated). For unique id's we use uuid1 (simple and easy to create). The resulting <entry> element would be:

<entry>
  <id>urn:uuid:11bba138-0eb3-11e1-ba8a-13b887ad42f8</id>
  <title>yooser posted a video</title>
  <published>2011-11-13T11:26:02Z</published>
  <summary>yooser just uploaded a brand new video</summary>

  <activity:object>
  <activity:object-type>http://activitystrea.ms/schema/1.0/video</activity:object-type>
  <id>urn:uuid:02b0a5ce-0eba-11e1-ba8a-13b887ad42f8</id>
  <activity:displayName>Hi Mom!</activity:displayName>
  <activity:embedCode>
&lt;video width=&quot;320&quot; height=&quot;240&quot; controls&gt;&lt;source src=&quot;http://mediagoblin.server.com/u/yooser/gallery/himom.ogv&quot; type='video/ogg; codecs=&quot;theora, vorbis&quot;'&gt;&lt;source src=&quot;http://mediagoblin.server.com/u/yooser/gallery/himom.webm&quot; type='video/webm; codecs=&quot;vp8, vorbis&quot;'&gt;&lt;source src=&quot;http://mediagoblin.server.com/u/yooser/gallery/himom.mp4&quot; type='video/mp4; codecs=&quot;avc1.42E01E, mp4a.40.2&quot;'&gt;&lt;/video&gt;</activity:embedCode>
  <activity:stream>http://mediagoblin.server.com/u/yooser/gallery/himom.ogv</activity:stream>
  <summary>I wave my hands and yell: Hi Mom!</summary>
  </activity:object>

  <activity:verb>http://activitystrea.ms/schema/1.0/post</activity:verb>

  <link rel="alternate" type="text/html" href="http://mediagoblin.server.com/u/yooser/111113112602/"/>
  <link rel="self" type="application/atom+xml" href="http://mediagoblin.server.com/u/yooser/111113112602.atom"/>
</entry>

The activity namespace is declared as "http://activitystrea.ms/spec/1.0/". These additions are really easy since we only need to look at the activity stream basic schema and see what we need to include for each one of the object types we use. It is well defined their. Then we create a function to provide this information (optimally to provide the atom representation of it).

We also have to declare the author in the feed and since it is a part of Atom already we only have to add a few things. The blog post adds to optional choices. First recommendation is adding extra information using links, like e.g. the user avatar. The second recommendation is using Portable Contacts. These two are not a part of the OStatus standard but could be good additions. Let's say for clarity that we skip both of them and define the author as follows:

<author>
 <activity:object-type>http://activitystrea.ms/schema/1.0/person</activity:object-type>
 <uri>http://mediagoblin.server.com/u/yooser/</uri>
 <name>yooser</name>
</author>

This is quite a lot of work but it should be relatively straightforward to do. Create the atom feed. Create the media objects (but first think really hard about the model structure). Create a good way to create atom representations. Add to Atom feed. Test.

PubSubHubbub Publish

PubSubHubbub (PuSH) avoids polling for updates by creating a place (hub) where content providers publish (pub) their updates and the hub then takes care of notifying the those who have subscribed (sub) to their feeds. It's a simple approach which involves using regular HTTP POSTs.

OStatus uses this to allow users on other federated services to subscribe to activity streams on another service without having to put an extra polling load on the services with the subscribers.

How to Implement

For an example use case, lets say that we have a running MediaGoblin instance on mediagoblin.server.com and we want to publish the activity streams of our users to a given pubsubhubbub service. This must be a configurable variable for those who want to set up MediaGoblin instances (since they should be free to choose which services they use) but the official MediaGoblin might run its own hub which would be provided by default to make MediaGoblin instance creations simpler. There are a lot of different hub implementations available which could be used. This could also be a configurable variable for each user but that would just make MediaGoblin more difficult to use.

For the sake of example, let's say the instance on mediagoblin.server.com is set to use a hub located at: http://pubsubhub.bub.com. Every Atom activity stream on the MediaGoblin server must then provide a link element (marked with a rel attribute "hub") pointing to the PuSH server, so we add <link href="http://pubsubhub.bub.com" rel="hub"/> to the Atom feed.

It is not as important but a common web feed courtesy not to forget a properly set link with self as a rel value (expressing where the feed is). So if we are trying to publish the activity stream of a user called yooser which is located at http://mediagoblin.server.com/u/yooser/as.atom we add the self link in the atom feed with <link rel="self" type="application/atom+xml" href="http://mediagoblin.server.com/u/yooser/as.atom" />. This should be taken care of by the framework but lets be sure.

Next step. When a new activity stream is created or an existing one updated, we need to publish it to the hub. It's a really simple POST operation. We need to encode two variables, hub.url and hub.mode. The hub.url should contain the URL to the activity stream being published, which would in the case of our user yooser be http://mediagoblin.server.com/u/yooser/as.atom and the hub.mode must contain a single word: 'publish'. If the POSTing to the hub is successful the hub will return a 204 error code (No Content). If something goes wrong, the hub will return an appropriate error code.

In Python this is quite easy to do using urllib and urllib2:

hub = 'http://pubsubhub.bub.com'
url = 'http://mediagoblin.server.com/u/yooser/as.atom'

data = urllib.urlencode({'hub.url':url, 'hub.mode':'publish'})
try:
    response = urllib2.urlopen(hub, data)
except (IOError, urllib2.HTTPError), e:
    if hasattr(e, 'code') and e.code == 204:
        print 'Successfully published to hub'
    else:
        print 'Gosh darn it Spencer! It failed!'

This example does not bother with error handling (except to check status code), wrapping it in a function and it uses hard coded values, all for the sake of clarity. A better example is available from the pubsubhubbub source repository under the Apache license.

WebFinger

WebFinger is a way to identify users by email addresses and allow sites to access more information about the specific user. To accomplish this WebFinger uses two resources: A domain specific host-meta file which contains information about how to access information about a specific user, and the individual user description files, called endpoints. Both of these resources are relatively simple XRD documents.

OStatus uses this to allow different federated sites with different routing structures to find the activity streams of users (and perhaps other information as well) easily.

How to Implement

For an example use case, lets say that we have a running MediaGoblin instance on mediagoblin.server.com and we want to provide WebFinger identities for our users and in our example case a user with a username yooser. This user would get the WebFinger identity (email): yooser@mediagoblin.server.com.

The MediaGoblin instance will have to set up a host-meta file and make it available at: http://mediagoblin.server.com/.well-known/host-meta

This is the file other services will GET, based on the host of the email address mediagoblin.server.com to find out where to get access to more information about the user yooser.

The host-meta file can contain a lot of information about the site but the information other federated services are interested in is the Link-based Resource Descriptor Discovery protocol lrdd (pronounced: lard) information which is provided via a Link with a rel attribute lrdd and the real information is the associated template (also an attribute of this Link). The template defines a structure of where other federated services can find information about the user account. It is basically just a URL which contains a {uri} somewhere and this specific {uri} will be replaced with the user account identifier. This account identifier is the user email preceded by acct:, i.e. acct:yooser@mediagoblin.server.com

MediaGoblin will then have to decide on how this template should be structured. This could be something like http://mediagoblin.server.com/describe?uri={uri} (which is provided in the discussion of WebFinger) or http://mediagoblin.server.com/main/xrd?uri={uri} (which is the structure on Status.Net). It doesn't really matter how it is structured, just that it includes the {uri} and conforms with the overall routing structure of MediaGoblin.

We must also provide a Host element in the host-meta file, in accordance with the host-meta description. For our example this would be <hm:Host>mediagoblin.server.com</hm:Host> (where the hm namespace is http://host-meta.net/xrd/1.0).

So let's take a look at an example of what other services might find when they GET the host-meta file for mediagoblin.server.com at http://mediagoblin.server.com/.well-known/host-meta (for this example we'll use the WebFinger template):

<?xml version='1.0' encoding='UTF-8'?>
<XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'
     xmlns:hm='http://host-meta.net/xrd/1.0'>
 
    <hm:Host>mediagoblin.server.com</hm:Host>
 
    <Link rel='lrdd'
          template='http://mediagoblin.server.com/describe?uri={uri}'>
        <Title>Resource Descriptor</Title>
    </Link>
</XRD>

In Django this host-meta file is easily created using Django's template system and reverse URL lookups. This should be just as easy to do in MediaGoblin's framework.

Next we have to create the user specific description, i.e. the stuff other federated services will find when they access the template. This is done, as previously discussed by substituting {uri} with the account information (email preceded by acct:) and GETting it. So the other federated services would GET http://mediagoblin.server.com/describe?uri=acct:yooser@mediagoblin.server.com and that would return a whole lot of interesting user specific stuff.

The insteresting stuff is againg presented with XRD and Link elements and could for example be a Link to the profile page (<Link rel="http://webfinger.net/rel/profile-page" type="text/html" href="http://mediagoblin.server.com/u/yooser"/>), the user avatar (<Link rel="http://webfinger.net/rel/avatar" href="http://mediagoblin.server.com/u/yooser/avatar" />). The blog post says that it should at least include a link to the updates using http://schemas.google.com/g/2010#updates-from which points the the Atom ActivityStream (<Link rel="http://schemas.google.com/g/2010#updates-from" type="application/atom+xml" href="http://mediagoblin.server.com/u/yooser/as.atom" />).

We must also declare the Subject (the user) in the XRD and we do that using the Subject element which contains the account identifier (email preceded by acct:). We can also Alias the user (using a Alias element) and provide a URL pointing to the user profile page.

So if we look at an example to combine all of this and provide the minimum information needed to enable OStatus (we can always provide more interesting stuff later on or add it when we actually implement WebFinger instead of just describing it on a Wiki page since we add it for subscription and salmon based stuff) the XRD returned when a federated service GETs http://mediagoblin.server.com/describe?uri=acct:yooser@mediagoblin.server.com would look like this:

<?xml version='1.0' encoding='UTF-8'?>
<XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'>
 
    <Subject>acct:yooser@mediagoblin.server.com</Subject>
    <Alias>http://mediagoblin.server.com/u/yooser</Alias>
 
    <Link rel="http://schemas.google.com/g/2010#updates-from"
          type="application/atom+xml"
          href="http://mediagoblin.server.com/u/yooser/as.atom" />
</XRD>

Again, this would be easy to do with Django's template system and reverse URL lookups, and should be just as easy to do with MediaGoblin's framework. When that is done MediaGoblin will have the WebFinger part of OStatus implemented.

OStatus

Contents

OStatus

Important Links

Process and work products

ActivityStreams

How to Implement

Related Links

PubSubHubbub Publish

How to Implement

Related Links

WebFinger

How to Implement

Related Links

Navigation menu

OStatus

OStatus

Important Links

Process and work products

ActivityStreams

How to Implement

Related Links

PubSubHubbub Publish

How to Implement

Related Links

WebFinger

How to Implement

Related Links

Navigation menu

Search