GNU MediaGoblin Wiki - User contributions [en]

Main Page

2012-12-29T12:59:50Z

Tryggvib: Undo revision 1048 by Patrica11upps (talk)

= Want to Join the MediaGoblin Community? =

We’re really glad that you want to join the MediaGoblin community!

There are a variety of ways to help and support MediaGoblin and to join the team. If you want to code, great, if not, even better! MediaGoblin interested contributors in many different roles: users, system administrators, technical writers, testers, evangelists, UI/UX and graphics designers, cheerleaders, and dreamers.

This wiki covers a variety of ways that you can get involved with MediaGoblin as well as instructions on how to get started.

== Hang out with the MediaGoblin folk ==

MediaGoblin has a mailing list and an IRC channel where we hang out. See [http://mediagoblin.org/pages/join.html our join page] for links.

Please drop by and say “Hi!” And, if you’re looking for something to do, just ask---there’s always work to be done.

== Take Part in the Monthly Meetings ==

Each month is a [[Meeting]]. You can take part and help decide on the future of MediaGoblin. Or just be around and see what's happening live!

== File Bugs / Triage Bugs ==

Issue reports are critical for all projects. Identified bugs give developers a basis for beginning work, and providing an idea of what features and issues are most important to users and the overall usability of the software. If you identify errors, flaws, unexpected behaviors, or deficits that impede use, file a bug.

* [[File Bugs]] -- notes on filing new bugs/issues/feature requests
* [[Feature Ideas]] -- notes on possible features
* [[Triage Bugs]] -- notes on triaging
* [[BugTriageDay]] -- every other Thursday is bug triage day where anyone can help out triaging bugs

== Write Code / Fix Code ==

If you are a coder and you would like to write code, the repository is hosted on gitorious. Clone or fork the repository and start poking around. Become familiar with this manual for an overview of how the software works and is used. Consider the contributor wiki for more information about the project, our preferred methods, and guides for developing MediaGoblin. We even have tips on becoming a coder and we’re willing to help!

* [[HackingHowto|Hacking]] - notes on making and sending in code contributions
** [[BeginnersCorner|Beginner's Corner]] - resources for those who are new to Python or Git.
** ''Started from an older version of the Hacking Howto? We switched from buildout->virtualenv, so look at [[Moving from buildout to virtualenv]] for information on how to move over.''
* [[Git workflow]] - How to go about submitting patches via git.
* [[Templating]] - How our templating structure is set up
* [[Code overview]] - Overview of the structure of the codebase

== Send Encouragement / Spread the Word ==

Sometimes, a nice word, simple encouragement, and interest in the work we’re doing is enough to inspire a tizzy of productive work. Just a bit more interest and encouragement can even make the difference between a complete feature and limited functionality; between a completed milestone and lost momentum.

Similarly, MediaGoblin, and the movement for free network services, is always in need of encouragement. Use free network services, understand the principals behind the movement, be able to articulate the benefits of free network services and the problems with psudo-free applications that don’t respect the users’ freedom.

Write a blog post, post a status update, drop by the listserv or join #mediagoblin on freenode.net and let us know. See [http://mediagoblin.org/join/ our join page] for links.

== Write Documentation / Edit Documentation ==

* [[Documentation quick start]] - How to contribute to the documentation effort.
* [[ManualStandards]] - covers the standards for writing the user manual (forthcoming.)

== Test MediaGoblin ==

Do you have access to the web? Do you like sharing your opinions? If so, we need your help to test MediaGoblin! Testers play around with the current test instance, note what operating system and browser they use (notes on multiple set-ups are also helpful) and take some notes. That's it! It's a very important task that doesn't require any special knowledge and you're done in under an hour. Ready to help?

* [[User Experience]] - user experience testing. Includes link to an instance you can try!
* [[UnitTests|Unit Tests]] - all about the unit tests

== Translate MediaGoblin ==

If you know English and another language and feel comfortable translating elements of the interface or even the documentation, we’d love to have help translating the software and resources.
Translating MediaGoblin is very easy with a web interface, so there is no programming knowledge required at all.

* [[Translations]] - How to translate stuff or update the translations

== Become a User ==

Coming soon!

We’re building MediaGoblin for us and for you but really you’re one of us and I am you and we are we and MediaGoblin is the walrus.

We're planning to launch our own public instance of MediaGoblin in the near future--probably in the September/October 2011 time frame. When we do, sign up for an account, use the service and relish in the thought that this service comes with a heaping side of Freedom and you can salt and pepper it to your liking.

== Help Others ==

Have you spent time with MediaGoblin? If so, your experience and wisdom are invaluable and you’re the best person we can think of to help other users with their questions.

Hang out on the IRC channel and help answer new peoples' questions. See [http://mediagoblin.org/join/ our join page] for links.

== Run your own MediaGoblin Instance ==

Are there things about our instance you want to change? Are there things about other instances you wish were different? Want to test upcoming changes? Want to create patches to implement things you need? That’s great—you can run your own instance!

* [[Configure_MediaGoblin|Configuration]] - Learn about MediaGoblin configuration files and file options.
* [[Deployment]] - General deployment advice
* [[Scaling Down]] - Minimizing MediaGoblin's resource requirements
* [[Virtual Machine Hosting]] - Deploy your own publicly available MediaGoblin server using [http://aws.amazon.com/free/?utm_source=adwords&utm_medium=cpc&utm_campaign=CPC_Google_AWS_ec2&utm_content=TextV01_PP_V01_EC2&trk=CPC_Google_AWS_ec2 Amazon's free EC2 tier].

== Create a Theme ==

Coming soon!

MedaGoblin development is premised on the idea that the entire interface for the platform be completely theme-able. If you have a design or theming background, consider developing themes for MediaGoblin. New themes help test the theming system, provide attractive and appealing interfaces for prospective users. If you want to start a new theme but don’t know where to start, touch base with the development community on the list or in the IRC channel for more information.

= Technical project documentation =

* [[DesignDecisions]] - covers design decisions (FIXME - this needs to be split up)
* [[Storage]] - How MediaGoblin's internal storage system works.
* [[Processing]] - What happens after you submit your image/video/etc? Processing! More about that.
* [https://gitorious.org/mediagoblin/mediagoblin/blobs/master/extlib/README External Library Policy] - covers use of external libraries
* [[User:Cwebber/braindumps]] - Chris Webber's braindumps (you can help refactoring these into real sections of the site!)
* [[Multiple media support]] - Design plan for multiple media support

= Inner workings of the secret sanctum =

* [[IRCBot]] - covers our irc bot
* [[ReleaseProcess|Release Process]] - covers the release process
* [[Update the website]] - Learn how to update mediagoblin.org!

Main Page

2012-12-29T12:59:32Z

Tryggvib: Undo revision 1049 by Patrica22upps (talk)

It a multi functional combination for [http://www.mkol.us michael kors handbags] instance particular emblems all of which equals a multi function put together for instance bar lines. These bar lines are offered on temporary,far together with various spaces bewteen barefoot and shoes. Without it applications,the choices procedure that was spoke of that has reached over is mainly because all the way through unimaginable to find performed.

There aren't any handbags in blue, brown or red. But Coach lead a trend of developing bags in Pink as well as other colors. Now, you will observe most bags of Coach are typically in pink or blue. Then came the Lv Black Murakami Collection. Not only will you start to see the wanna be girls for the mall, hoping that in case they turn the tag in and don let anyone be aware of the clasp, they may just get individuals reckon that they carrying a geniune LV bag. You are aware of the visible difference.

It is best to decide washable fabrics,burberry bags who have a fine texture. Make sure to get yourself a bag set that has a pocket within 3-5 inches with the mattress depth to prevent yourself from [http://www.mkmk.us michael kors outlet] slippage. The thread-count in the bed [http://www.headphonesbybeats.com beats by dre] sheet is why the sheets durable and soft, so be sure you choose sheets of high thread-count.

In the huge money that cost about the designer bag, [http://www.shoptnf.com north face outlet] you can easily do all kinds of other significant things. So inevitably, from any of the Designer bags during my wardrobe are replica bags. Keep in mind, I never embarrass myself for carrying the replica Dior bags, Replica Chanel bags and Replica LV bags.

Size is important. The larger the better for many, and small, small persons. Design for a large bag will change the selling price when it is a very high end designer purse. Credibility - When you finally would've provided your phrase, if or not it's truly a promise or perhaps not, do this. A poor impression was developed rrnside your boss by chance you usually do not use a term of honor. In some way, it tends to make you peer to get an untrustworthy person.

Or if at all close family celebration, you would possibly have every balloon printed together with your family photograph. Also, assume associated with an awesome phrase of greetings as you will also make sure they printed blended with pics. The greetings will express your feelings and ideas to the celebrant, or simply a method of expressing the style the celebration was created for.

Now, lots of personal funds to overseas, however it is big, it will likewise make foreign rates rising, overseas to bid the cost real, to work as China's skill and ability. Another way to point out that China's rich bought abroad, you get some foreign countries economy recovers, reducing high unemployment rates, a change in the economical situation. The rapid progression of China's economy, plus "rescued" by some Western countries appears to be cheap.

http://metropolistower.com/wiki/index.php5/index.php?title=
http://instrumentisto.com/index.php/index.php?title=
http://readme.campus.mephi.ru/index.php?title=
http://www.virtualurth.com/index.php/member/420448/

Personas

2012-11-08T10:22:19Z

Tryggvib: Description of Personas and three personas added: Gavroche (from joar), Shaquannah and Spencer

'''Personas''' in MediaGoblin development (not to be confused with [https://persona.org/ Mozilla Persona], formerly known as BrowserID) are a way to identify a specific target group of users via anthropomorphic goblins. This allows for a quick identification of a particular MediaGoblin user, e.g. in user stories.

Personas should never be regarded as ''the'' only way to identify users and MediaGoblin developers should take care when using them since they might drive new developers off (instead of encouraging them to contribute). However these personas do make development more fun which in turn can encourage more people to participate. Ideally when using these personas in an environment accessible to ''would-be-developers'', they should always be accompanied by their titles (user group) for clarification, i.e. ''Gavroche the artist uploads a new SVG image''.

= Gavroche =

== Title (user group) ==

Artist

== Bio ==

Gavroche is an independent artist living in a suburb of Paris, France. Gavroche creates audible, text, visual and moving pictures often under the CC-BY-SA license.

Gavroche is currently a GNOME3 user. Gavroche uses desktop applications such as

* Blender, for animations and VFX
* Shotwell/digiKam, for digital photo development
* GIMP, for photo editing
* Vim/Emacs/$FOO

Gavroche would like to be able to submit stuff directly from any of these applications, but he should not be required to launch any of them to be able to upload anything.

= Shaquannah =

== Title (user group) ==

System administrator / sysadmin

== Bio ==

Shaquannah lives in a castle upon a hill, surrounded by farms and fields where lowly, plebeian goblins live. Contrary to the stereotypical castle-dweller she does not rule over her poor, neighbouring goblins. She tries to help and support them by the only charitable way imaginable. She runs a MediaGoblin instance for them. Thanks to Shaquannah the famished goblins, who live off the lands surrounding her castle, can take pictures and videos on their spiffy, expensive smartphones and upload them to a MediaGoblin instance where they serve as a documentation, a proof, of the tyranny of the other castle-dwelling neighbour (a weird little goblin who goes by the nickname ''The government'' - probably because he's the big brother of some lucky goblin).

Shaquannah runs her MediaGoblin instance on a Debian based system. Shaquannah is able to write some code but she absolutely loves fiddling around with configurations and enabling this or that in the system (she goes all giddy when a small change from "false" to "true" allows for some breathtaking features that get her penniless neighbours to worship her all over again).

In her spare time, Shaquannah enjoys kicking people out of her castle... they're interfering with her highly artistic sysadmin work!

= Spencer =

== Title (user group) ==

Visitor / website viewer / guest

== Bio ==

Spencer lives in cyberspace. Does he have a home in meatspace? Nobody knows. His goal in life is to become a living, breathing goblin Google (his IRC handle is ''Goobln''). To achieve this goal Spencer spends his entire day visiting all the websites he can find and memorizes its contents. He even visits the websites in different browsers just to be sure that the website developers don't serve up different content based on the user's browser.

Spencer's favorite browsers are '''Mozilla Firefox''' and '''Google Chromium (Chrome)''' because they make cyberspace a better home for him, but he also secretly enjoys checking how websites handle ''restricted'' browsers, e.g. how the website is presented in a standards in-compliant browser such as Internet Explorer, how the website is presented in a text based browser such as Lynx, and how responsive the website is by visiting it via the smallest smartphone he can get his hands on (because everybody knows smartphones and hands exist in cyberspace).

Websites that support all browsers score extra in Spencer's mental search engine ranking system.

User:Joar/Proposed usecase

2012-11-08T00:35:15Z

Tryggvib: Made a link to a new page on Personas (will copy Gavroche description over there)

This is the proposed default usecase for newly deployed GNU MediaGoblin applications.

I learned that this kind of writing is called a "persona", some might find it creepy. [[Personas]] help me to look beyond my own needs, to design the system for someone else rather than myself.

= Gavroche =

Gavroche is an independent artist living in a suburb of Paris, France. Gavroche creates audible, text, visual and moving pictures often under the CC-BY-SA license.

Gavroche is ''currently'' a GNOME3 user. Gavroche uses desktop applications such as

* Blender, for animations and VFX
* Shotwell/digiKam, for digital photo development
* GIMP, for photo editing
* Vim/Emacs/$FOO

Gavroche would like to be able to submit stuff directly from any of these applications, but he should not be required to launch any of them to be able to upload anything.

<blockquote>
<p>Desktop application integration is certainly possible sine the API. The main limitation would be the desktop application itself.</p>
--[[User:Joar|Joar]] ([[User talk:Joar|talk]]) 07:24, 3 November 2012 (EDT)
</blockquote>

== Submitting via the web ==

Gavroche uses Firefox or Chromium/Chrome, he might use IE10, but that's a rare occasion. He avoids IE <= 9 because he has a feeling that they might not be usable everywhere on the web.

Gavroche might have several files, or even a folder of files that he wants to upload. The files might be videos, images, audio, ascii, source files such as .xcf, .blend, etc.

Gavroche drags and drops the files from nautilus to his browser window where the MediaGoblin instance is running.

First he drops the files ''flight.ogv'', ''superman-wallpaper.png'' and ''swoosh.ogg'' in the browser window, this causes three boxes to be shown to him, each box representing one of the files.

Gavroche then navigates to his project folder for the ''superman-wallpaper.png'' file then selects ''superman-wallpaper.blend'', ''README.txt'' and ''superman-glasses-texture.png'' then drops them onto the box representing ''superman-wallpaper.png'' in the browser window, this causes three smaller boxes representing each additional file to be shown.

HackingHowto

2012-01-06T12:22:49Z

Tryggvib: /* Fedora / RedHat(?) */

= Hacking HOWTO =

== So you want to hack on GNU MediaGoblin? ==

First thing to do is check out the [http://mediagoblin.org/join/ web site] where we list all the project
infrastructure including:

* the IRC channel
* the mailing list
* the issue tracker

Additionally, we have information on how to get involved, who to talk
to, what needs to be worked on, and other things besides!

Second thing to do is take a look at [http://docs.mediagoblin.org/codebase.html codebase chapter] where
we've started documenting how GNU MediaGoblin is built and how to add
new things.

Third you'll need to get the requirements.

Fourth, you'll need to build a development environment. We use an
in-package checkout of virtualenv. This isn't the convenional way to
install virtualenv (normally you don't install virtualenv inside the
package itself) but we've found that it's significantly easier for
newcomers who aren't already familiar with virtualenv. If you *are*
already familiar with virtualenv, feel free to just install
mediagoblin in your own virtualenv setup... the necessary adjustments
should be obvious.

== Getting requirements ==

First, you need to have the following installed before you can build
an environment for hacking on GNU MediaGoblin:

* Python 2.6 or 2.7 - http://www.python.org/ (You'll need Python as well as the dev files for building modules.)
* python-lxml - http://lxml.de/
* git - http://git-scm.com/
* MongoDB - http://www.mongodb.org/
* Python Imaging Library (PIL) - http://www.pythonware.com/products/pil/
* virtualenv - http://www.virtualenv.org/
* Python GStreamer Bindings - http://gstreamer.freedesktop.org/modules/gst-python.html

Note, MongoDB can get to be pretty huge. You might want to consider
looking at [[Scaling Down]] if you are installing this on a smallish
instance.

=== GNU/Linux ===

==== Debian and derivatives ====

If you're running Debian GNU/Linux or a Debian-derived distribution
such as Debian, Mint, or [http://bugs.foocorp.net/issues/478 Ubuntu 10.10+], running the following should install these
requirements:

{{Cmd|sudo apt-get install mongodb git-core python python-dev python-lxml python-imaging python-virtualenv python-gst0.10}}

==== Fedora / RedHat(?) ====

On Fedora:

{{Cmd|yum install mongodb-server python-paste-deploy python-paste-script git-core python python-devel python-lxml python-imaging python-virtualenv gstreamer-python}}

=== Mac OS X ===

On Mac OS X Lion:

Download the Newest Python.

Git is already installed.

Install MongoDB from these instructions: http://www.mongodb.org/display/DOCS/Quickstart+OS+X

Python-lxml: http://muffinresearch.co.uk/archives/2009/03/05/install-lxml-on-osx/ with sudo

Python Imaging Library (PIL): http://code.google.com/appengine/docs/python/images/installingPIL.html#mac

Libjpeg & Libpng: http://ethan.tira-thompson.org/Mac_OS_X_Ports.html Combo Installer

----

'''You can help:'''

If you have instructions for other GNU/Linux distributions, Windows, or Mac OS X to set
up requirements, let us know!

== How to set up and maintain an environment for hacking with virtualenv ==

'''Requirements'''

No additional requirements.

'''Create a development environment'''

After installing the requirements, follow these steps:

# Clone the repository: {{Cmd|git clone <nowiki>git://gitorious.org/mediagoblin/mediagoblin.git</nowiki>}}
# Change directories to your new checkout: {{Cmd|cd mediagoblin}}
# Set up the in-package virtualenv: {{Cmd|virtualenv . && ./bin/python setup.py develop}}

That's it!

(If you have troubles in the remaining steps, consider try installing
virtualenv with one of the flags --setuptools, --distribute or possibly --no-site-packages. Additionally, if your system has python3.X as the default, you might need to do virtualenv --python=python2.7 or --python=python2.6)

If you have problems, please [http://mediagoblin.org/join/ let us know]!

'''Updating for dependency changes'''

While hacking on GNU MediaGoblin over time, you'll eventually have to
update your development environment because the dependencies have
changed.

To do that, run:

{{Cmd|./bin/python setup.py develop --upgrade && ./bin/gmg migrate}}

'''Updating for code changes'''

{{Cmd|git pull -u}}

== Running Mongo Database ==

Startup the Database if you haven't already. On most systems, this
should be started automatically. If not, consult the documentation
for your distribution or the MongoDB docs themselves.

== Running the server ==

If you want to get things running quickly and without hassle, just
run:

{{Cmd|./lazyserver.sh}}

This will start up a python server where you can begin playing with
mediagoblin, listening on 127.0.0.1:6543. It will also run celery in "always eager" mode so you
don't have to start a separate process for it.

By default, the instance is not sending out confirmation mails. Instead they are redirected to the standard output (the console) of lazyserver.sh.

You can change this behavior setting <code>email_debug_mode</code> to <code>false</code> in mediagoblin.ini

This is fine in development, but if you want to actually run celery
separately for testing (or deployment purposes), you'll want to run
the server independently:

{{Cmd|./bin/paster serve paste.ini --reload}}

== Running celeryd ==

If you aren't using <tt>./lazyserver.sh</tt> or otherwise aren't running celery
in always eager mode, you'll need to do this if you want your media to
process and actually show up. It's probably a good idea in
development to have the web server (above) running in one terminal and
celeryd in another window.

Run:

{{Cmd|<nowiki>CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd</nowiki>}}

== Running the test suite ==

Run:

{{Cmd|./runtests.sh}}

== Running a shell ==

If you want a shell with your database pre-setup and an instantiated
application ready and at your fingertips....

Run:

{{Cmd|./bin/gmg shell}}

== Troubleshooting ==

=== pymongo.errors.AutoReconnect: could not find master/primary ===

If you see this:

pymongo.errors.AutoReconnect: could not find master/primary

then make sure mongodb is installed and running.

If it's installed, check the mongodb log. On my machine, that's
<tt>/var/log/mongodb/mongodb.log</tt>. If you see something like:

old lock file: /var/lib/mongodb/mongod.lock. probably means...

in that case you might have had an unclean shutdown. Try:

{{Cmd|sudo mongod --repair}}

If that didn't work, just delete the lock file and relaunch mongodb.

Anyway, then start the mongodb server in whatever way is appropriate
for your distro / OS.

== Wiping your user data ==

You can completely wipe all data from the instance by doing:

{{Cmd|gmg wipealldata}}

'''Note:'''

Unless you're doing development and working on and testing creating
a new instance, you will probably never have to do this. Will
plans to do this work and thus he documented it.

== Quickstart for Django programmers ==

We're not using Django, but the codebase is very Django-like in its
structure.

* <tt>routing.py</tt> is like <tt>urls.py</tt> in Django
* <tt>models.py</tt> has mongokit ORM definitions
* <tt>views.py</tt> is where the views go

We're using MongoDB. Basically, instead of a relational database with
tables, you have a big JSON structure which acts a lot like a Python
dict.

'''YouCanHelp'''

If there are other things that you think would help orient someone
new to GNU MediaGoblin but coming from Django, let us know!

== Bite-sized bugs to start with ==

Now you should visit our latest list of [http://bugs.foocorp.net/projects/mediagoblin/issues?query_id=3 bite-sized issues] because squishing bugs is messy fun. If you're interested in other things to work on, or need help getting started on a bug, let us know on [http://mediagoblin.org/join/ the mailing list] or on the [http://mediagoblin.org/join/ IRC channel].

HackingHowto

2012-01-06T00:41:39Z

Tryggvib: /* Getting requirements */ Added Python GStreamer bindings

= Hacking HOWTO =

== So you want to hack on GNU MediaGoblin? ==

First thing to do is check out the [http://mediagoblin.org/join/ web site] where we list all the project
infrastructure including:

* the IRC channel
* the mailing list
* the issue tracker

Additionally, we have information on how to get involved, who to talk
to, what needs to be worked on, and other things besides!

Second thing to do is take a look at [http://docs.mediagoblin.org/codebase.html codebase chapter] where
we've started documenting how GNU MediaGoblin is built and how to add
new things.

Third you'll need to get the requirements.

Fourth, you'll need to build a development environment. We use an
in-package checkout of virtualenv. This isn't the convenional way to
install virtualenv (normally you don't install virtualenv inside the
package itself) but we've found that it's significantly easier for
newcomers who aren't already familiar with virtualenv. If you *are*
already familiar with virtualenv, feel free to just install
mediagoblin in your own virtualenv setup... the necessary adjustments
should be obvious.

== Getting requirements ==

First, you need to have the following installed before you can build
an environment for hacking on GNU MediaGoblin:

* Python 2.6 or 2.7 - http://www.python.org/ (You'll need Python as well as the dev files for building modules.)
* python-lxml - http://lxml.de/
* git - http://git-scm.com/
* MongoDB - http://www.mongodb.org/
* Python Imaging Library (PIL) - http://www.pythonware.com/products/pil/
* virtualenv - http://www.virtualenv.org/
* Python GStreamer Bindings - http://gstreamer.freedesktop.org/modules/gst-python.html

Note, MongoDB can get to be pretty huge. You might want to consider
looking at [[Scaling Down]] if you are installing this on a smallish
instance.

=== GNU/Linux ===

==== Debian and derivatives ====

If you're running Debian GNU/Linux or a Debian-derived distribution
such as Debian, Mint, or [http://bugs.foocorp.net/issues/478 Ubuntu 10.10+], running the following should install these
requirements:

{{Cmd|sudo apt-get install mongodb git-core python python-dev python-lxml python-imaging python-virtualenv python-gst0.10}}

==== Fedora / RedHat(?) ====

On Fedora:

{{Cmd|yum install mongodb-server python-paste-deploy python-paste-script git-core python python-devel python-lxml python-imaging python-virtualenv}}

=== Mac OS X ===

On Mac OS X Lion:

Download the Newest Python.

Git is already installed.

Install MongoDB from these instructions: http://www.mongodb.org/display/DOCS/Quickstart+OS+X

Python-lxml: http://muffinresearch.co.uk/archives/2009/03/05/install-lxml-on-osx/ with sudo

Python Imaging Library (PIL): http://code.google.com/appengine/docs/python/images/installingPIL.html#mac

Libjpeg & Libpng: http://ethan.tira-thompson.org/Mac_OS_X_Ports.html Combo Installer

----

'''You can help:'''

If you have instructions for other GNU/Linux distributions, Windows, or Mac OS X to set
up requirements, let us know!

== How to set up and maintain an environment for hacking with virtualenv ==

'''Requirements'''

No additional requirements.

'''Create a development environment'''

After installing the requirements, follow these steps:

# Clone the repository: {{Cmd|git clone <nowiki>git://gitorious.org/mediagoblin/mediagoblin.git</nowiki>}}
# Change directories to your new checkout: {{Cmd|cd mediagoblin}}
# Set up the in-package virtualenv: {{Cmd|virtualenv . && ./bin/python setup.py develop}}

That's it!

(If you have troubles in the remaining steps, consider try installing
virtualenv with one of the flags --setuptools, --distribute or possibly --no-site-packages. Additionally, if your system has python3.X as the default, you might need to do virtualenv --python=python2.7 or --python=python2.6)

If you have problems, please [http://mediagoblin.org/join/ let us know]!

'''Updating for dependency changes'''

While hacking on GNU MediaGoblin over time, you'll eventually have to
update your development environment because the dependencies have
changed.

To do that, run:

{{Cmd|./bin/python setup.py develop --upgrade && ./bin/gmg migrate}}

'''Updating for code changes'''

{{Cmd|git pull -u}}

== Running Mongo Database ==

Startup the Database if you haven't already. On most systems, this
should be started automatically. If not, consult the documentation
for your distribution or the MongoDB docs themselves.

== Running the server ==

If you want to get things running quickly and without hassle, just
run:

{{Cmd|./lazyserver.sh}}

This will start up a python server where you can begin playing with
mediagoblin, listening on 127.0.0.1:6543. It will also run celery in "always eager" mode so you
don't have to start a separate process for it.

By default, the instance is not sending out confirmation mails. Instead they are redirected to the standard output (the console) of lazyserver.sh.

You can change this behavior setting <code>email_debug_mode</code> to <code>false</code> in mediagoblin.ini

This is fine in development, but if you want to actually run celery
separately for testing (or deployment purposes), you'll want to run
the server independently:

{{Cmd|./bin/paster serve paste.ini --reload}}

== Running celeryd ==

If you aren't using <tt>./lazyserver.sh</tt> or otherwise aren't running celery
in always eager mode, you'll need to do this if you want your media to
process and actually show up. It's probably a good idea in
development to have the web server (above) running in one terminal and
celeryd in another window.

Run:

{{Cmd|<nowiki>CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd</nowiki>}}

== Running the test suite ==

Run:

{{Cmd|./runtests.sh}}

== Running a shell ==

If you want a shell with your database pre-setup and an instantiated
application ready and at your fingertips....

Run:

{{Cmd|./bin/gmg shell}}

== Troubleshooting ==

=== pymongo.errors.AutoReconnect: could not find master/primary ===

If you see this:

pymongo.errors.AutoReconnect: could not find master/primary

then make sure mongodb is installed and running.

If it's installed, check the mongodb log. On my machine, that's
<tt>/var/log/mongodb/mongodb.log</tt>. If you see something like:

old lock file: /var/lib/mongodb/mongod.lock. probably means...

in that case you might have had an unclean shutdown. Try:

{{Cmd|sudo mongod --repair}}

If that didn't work, just delete the lock file and relaunch mongodb.

Anyway, then start the mongodb server in whatever way is appropriate
for your distro / OS.

== Wiping your user data ==

You can completely wipe all data from the instance by doing:

{{Cmd|gmg wipealldata}}

'''Note:'''

Unless you're doing development and working on and testing creating
a new instance, you will probably never have to do this. Will
plans to do this work and thus he documented it.

== Quickstart for Django programmers ==

We're not using Django, but the codebase is very Django-like in its
structure.

* <tt>routing.py</tt> is like <tt>urls.py</tt> in Django
* <tt>models.py</tt> has mongokit ORM definitions
* <tt>views.py</tt> is where the views go

We're using MongoDB. Basically, instead of a relational database with
tables, you have a big JSON structure which acts a lot like a Python
dict.

'''YouCanHelp'''

If there are other things that you think would help orient someone
new to GNU MediaGoblin but coming from Django, let us know!

== Bite-sized bugs to start with ==

Now you should visit our latest list of [http://bugs.foocorp.net/projects/mediagoblin/issues?query_id=3 bite-sized issues] because squishing bugs is messy fun. If you're interested in other things to work on, or need help getting started on a bug, let us know on [http://mediagoblin.org/join/ the mailing list] or on the [http://mediagoblin.org/join/ IRC channel].

OStatus

2011-11-14T12:40:42Z

Tryggvib: Created a page with information about OStatus (how to publish through OStatus to begin with)

= OStatus =

OStatus is, as described on its [http://ostatus.org official site]:

<blockquote>an open standard for distributed status updates. [The] goal is a specification that allows different messaging hubs to route status updates between users in near-real-time-</blockquote>

== Important Links ==

* http://ostatus.org/
* http://ostatus.org/sites/default/files/ostatus-1.0-draft-2-specification.html

== Process and work products ==

Process based on: http://ostatus.org/2010/10/04/how-ostatus-enable-your-application

* Activity Streams, can be broken into:
** Provide Atom feed
** Transform Atom feed into activity stream
* PubSubHubbub Publish, can be broken into:
** Decide on and declare hub in user activity streams
** Publish to hub when activity entries are created
* WebFinger, can be broken into:
** Provide host-meta file for site with user account template url
** Provide account information for specific users

When a federated site has PUSH-enabled ActivityStreams, users on remote services can subscribe to users on the federated site and be notified when their status updates.

== ActivityStreams ==

ActivityStreams is a standardized way to represent activities of individual users on social sites. It is nothing more than an atom feed with additions to further categorize the activity entries. Each activity entry ''does something'' to an activity object.

This is the core of the social interactions which the OStatus standard makes distributable across multiple federated sites.

=== How to Implement ===

For an example use case, lets say that we have a running MediaGoblin instance on ''mediagoblin.server.com'' and we want to provide an ActivityStream for a user with a username ''yooser'' at a specific location, e.g. http://mediagoblin.server.com/u/yooser/as.atom

The first thing we have to do is create an Atom feed for yooser. This is relatively simple to do in Django and shouldn't be much of a problem in MediaGoblin's framework. We must keep in mind that we will have to expand the feed with additional information. We should make the atom feed discoverable using a link from the user's profile page: <link rel="alternate" href="http://mediagoblin.server.com/u/yooser/as.atom" type="application/atom+xml" title="Atom feed for yooser"/>

Next step is to choose what a user can do and which objects the user can do it onto. This is a really big deal and should be carefully chosen. The implementation of what gets chosen should be done in such a way that it enables easy additions to the entries and objects.

So, lets start with the activity objects of the system. There are a lot of them to choose from and not all of them have to be implemented (for example, why should MediaGoblin do anything about article, i.e. blog posts). Here's the complete list of object types according to the Activity Base Schema Draft (yes, it's not final):

* Article - A lot of paragraphs of text which might incorporate images and stuff
* Audio - File containing songs, speech or just about anything somebody can listen to
* Badge - Some sort of an award which can be granted to somebody
* Bookmark - Basically just a link to a website, called bookmark to avoid confusion
* Collection - A generic container for a lot of objects, like an album of photos or folders of files
* Comment - A text response to another object
* Event - Description of a "happening" which people can attend at some location at a specific time
* File - A generic document which cannot be categorized as anything specific (like Audio, Image or Video)
* Group - Similar to collection but allows member objects to join or leave
* Image - File containing something stationary which can be perceived through eye (or something else)
* Note - A dent, tweet, status update or any other form of short, only content article-ish object
* Person - A user on the system (or on another system) - core thing really
* Place - The draft says it best: 'The "place" object type represents a location on Earth.'
* Product - Some object or act which people sell or offer in return for something, like a book
* Question - A way to get opinion from others. A poll.
* Review - A (hopefully) well formed written opinion about what somebody thinks about something
* Service - Not to be confused with service as in product but a blog or something that services others
* Video - A file containing a lot of images in sequence, and in most cases combined with audio

There are also some extensions to this base schema described in the draft. These extensions can be added to the other objects but don't describe anything purely on their own (well they do but theoretically let's just say that it wouldn't make sense):

* Location - A place object which describes where some other object is at
* Mood - An object which describes in what mood the creator was or a person is in
* Rating - A number between 1.0 and 5.0 (with one decimal place)... think stars!
* Source - The original place where something came from, don't go all the way back to The Big Bang!
* Tags - A list of objects which have been marked as a part of something, like a person in a picture

The same schema defines what can be done with these objects.

* add - something is being added to something (a photo to an album)
* cancel - something is cancelled (an event)
* checkin - something/someone has arrived at a place
* delete - something has been deleted (does not necessarily mean permanent destruction)
* favorite - something/someone really, really likes something (favorite video/status)
* follow - someone/something started following the activity stream of someone/something
* give - someone/something gives something (like a badge) to someone/something
* ignore - someone/something ignores something
* invite - someone/something invites someone/something to something
* join - someone/something joins something
* leave - inverse of join or the inverse of checkin (someone/something leaves something)
* like - another form of favorite
* make-friend - a verb that says friendship has been formed
* play - something is played (a video)
* post - creation of an object
* receive - someone or something has received something (like a badge)
* remove - something is removed from something (a photo from an album)
* remove-friend - friendship has been discontinued
* request-friend - someone wants to become friends with someone
* rsvp-maybe - someone might attend something (an event)
* rsvp-no - someone won't attend something (an event)
* rsvp-yes - someone will attend something (an event)
* save - someone has saved something out of personal interest
* share - something is shared with others
* stop-following - someone/something has stopped following the activity stream of something
* tag - someone/something is said to be present in something
* unfavorite - something is not as awesome as it once was
* unlike - something is not as ok as it once was
* unsave - something is not being used for personal interest anymore
* update - something has been updated (a blog post)

So, now we know what we can do and what we can do it onto. For MediaGoblin we should choose a subset of these objects and activity verbs. Let's start with the objects. MediaGoblin is a site primarily to be used for sharing and socializing around media. As such we need media objects and a few other objects make these media objects sociable. Of course MediaGoblin could in the future incorporate other activity objects (like organizing events or give out badges) but let's restrict us to pure media socializing.

Firstly, we need the core thing... the user:

* Person - A user of the system represented as an activity object

Secondly, these are the media types:

* Image - Users should be able to upload and view photos and other images
* Audio - Users should be able to upload and listen to audio files
* Video - Users should be able to upload and watch videos
* File - Users should be able to upload and download other files (e.g. README documents, books)

Lastly, we need to allow other object types to make MediaGoblin social:

* Collection - To create an album of videos or images (thus allowing categorization)
* Group - To allow users to find users and media more easily (find and categorize people/media objects)
* Comment - To allow users to comment on media files (their own or uploaded by others)
* Place - To describe a location (a place shown in an image)

We should also allow a few extensions to the base schema:

* Location - The reason we allowed Place
* Source - To attribute the right author
* Tags - To tag a place or a person in a media object

Now we need to decide what verbs the person (actor) can perform. Regarding favorite or like, we should go with favorite to avoid any confusion around Facebook. The following list also assumes a Status.Net approach (anybody can follow anybody, instead of creating friendship). It also does not include play, just because I (the one who wrote this) doesn't like other people knowing what I'm doing in such detail.

* post, delete - To create or destroy media objects (delete should permanently delete stuff)
* add, remove - To add or remove media objects from albums (collection)
* favorite, unfavorite - To allow other persons to see what one person thinks is awesome
* follow, stop-following - To allow persons to add other persons to their stream
* join, leave - To allow someone to join/leave a group about some specific topic
* share - To allow a person to redistribute a media object from another person on that person's stream
* tag - To allow a person to say that another person is in a media object

OK. So how do we do it? Each and every activity object described above includes specific information which is only used for that specific object. One way to form the objects is to create an activity object model class (e.g. ActivityConstruct) which can be subclassed as a specific activity object. Each subclassed activity object then has a method (e.g. def ostatus(self)) which can be called to get the object type and the corresponding information. The activity itself (using a verb on an object) is then collected in another model, e.g. ActivityEntry which forms the timeline (the activity stream). Defining the ActivityEntry model in Django would then be something like:

<source lang="python">
class ActivityEntry(models.Model):
atom_id = models.CharField(max_length=256, editable=False) # unique id, using uuid1()
actor = models.ForeignKey('Person', related_name='actors') #
activityobject = models.ForeignKey('ActivityConstruct', related_name='activity_objects')
target = models.ForeignKey('ActivityConstruct', blank=True, null=True, related_name='activity_targets')
verb = models.CharField(max_length=128)
time = models.DateTimeField(auto_now_add=True)
title = models.CharField(max_length=256) # Short description (usually same as summary)
summary = models.TextField() # <person> <verb>ed <activityobject> (in <target>)
</source>

The problem with this approach (i.e. in Django) is how to access the subclasses since the activity entry points to the parent class. It's messy but is still a good way to achieve this. MediaGoblin's framework might be able to solve this in a better way.

All right. So, how do we show it? Well, we take our Atom feed (which would represent the timeline of activity entries) and append stuff to it. For a regular atom feed an <entry> element consists of the following:

<source lang="xml">
<entry>
<id>urn:uuid:11bba138-0eb3-11e1-ba8a-13b887ad42f8</id>
<title>yooser posted a video</title>
<link rel="alternate" type="text/html" href="http://mediagoblin.server.com/u/yooser/gallery/himom.ogv"/>
<link rel="self" type="application/atom+xml" href="http://mediagoblin.server.com/u/yooser/111113112602.atom"/>
<published>2011-11-13T11:26:02Z</published>
<summary>yooser just uploaded a brand new video</summary>
</entry>
</source>

Adding activity stream stuff is just about adding the relevant information. Video information is for example: author (can be omitted if the author is the actor), displayName (title of video), embedCode (code for HTML embedding), id (unique id for video), published (optional time when published), stream (media link to video, summary (optional summary of video), updated (optional time when updated). For unique id's we use uuid1 (simple and easy to create). The resulting <entry> element would be:

<source lang="xml">
<entry>
<id>urn:uuid:11bba138-0eb3-11e1-ba8a-13b887ad42f8</id>
<title>yooser posted a video</title>
<published>2011-11-13T11:26:02Z</published>
<summary>yooser just uploaded a brand new video</summary>

<activity:object>
<activity:object-type>http://activitystrea.ms/schema/1.0/video</activity:object-type>
<id>urn:uuid:02b0a5ce-0eba-11e1-ba8a-13b887ad42f8</id>
<activity:displayName>Hi Mom!</activity:displayName>
<activity:embedCode>
<video width="320" height="240" controls><source src="http://mediagoblin.server.com/u/yooser/gallery/himom.ogv" type='video/ogg; codecs="theora, vorbis"'><source src="http://mediagoblin.server.com/u/yooser/gallery/himom.webm" type='video/webm; codecs="vp8, vorbis"'><source src="http://mediagoblin.server.com/u/yooser/gallery/himom.mp4" type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'></video></activity:embedCode>
<activity:stream>http://mediagoblin.server.com/u/yooser/gallery/himom.ogv</activity:stream>
<summary>I wave my hands and yell: Hi Mom!</summary>
</activity:object>

<activity:verb>http://activitystrea.ms/schema/1.0/post</activity:verb>

<link rel="alternate" type="text/html" href="http://mediagoblin.server.com/u/yooser/111113112602/"/>
<link rel="self" type="application/atom+xml" href="http://mediagoblin.server.com/u/yooser/111113112602.atom"/>
</entry>
</source>

The activity namespace is declared as "http://activitystrea.ms/spec/1.0/". These additions are really easy since we only need to look at the [http://activitystrea.ms/head/activity-schema.html activity stream basic schema] and see what we need to include for each one of the object types we use. It is well defined their. Then we create a function to provide this information (optimally to provide the atom representation of it).

We also have to declare the author in the feed and since it is a part of Atom already we only have to add a few things. The [http://ostatus.org/2010/10/04/how-ostatus-enable-your-application blog post] adds to optional choices. First recommendation is adding extra information using links, like e.g. the user avatar. The second recommendation is using Portable Contacts. These two are not a part of the OStatus standard but could be good additions. Let's say for clarity that we skip both of them and define the author as follows:

<source lang="xml">
<author>
<activity:object-type>http://activitystrea.ms/schema/1.0/person</activity:object-type>
<uri>http://mediagoblin.server.com/u/yooser/</uri>
<name>yooser</name>
</author>
</source>

This is quite a lot of work but it should be relatively straightforward to do. Create the atom feed. Create the media objects (but first think really hard about the model structure). Create a good way to create atom representations. Add to Atom feed. Test.

=== Related Links ===

* http://activitystrea.ms/
* http://activitystrea.ms/head/activity-schema.html
* http://atomenabled.org/
* http://portablecontacts.net/

== PubSubHubbub Publish ==

PubSubHubbub (PuSH) avoids polling for updates by creating a place (hub) where content providers publish (pub) their updates and the hub then takes care of notifying the those who have subscribed (sub) to their feeds. It's a simple approach which involves using regular HTTP POSTs.

OStatus uses this to allow users on other federated services to subscribe to activity streams on another service without having to put an extra polling load on the services with the subscribers.

=== How to Implement ===

For an example use case, lets say that we have a running MediaGoblin instance on ''mediagoblin.server.com'' and we want to publish the activity streams of our users to a given pubsubhubbub service. This must be a configurable variable for those who want to set up MediaGoblin instances (since they should be free to choose which services they use) but the official MediaGoblin might run its own hub which would be provided by default to make MediaGoblin instance creations simpler. There are a lot of different hub implementations available which could be used. This could also be a configurable variable for each user but that would just make MediaGoblin more difficult to use.

For the sake of example, let's say the instance on ''mediagoblin.server.com'' is set to use a hub located at: ''http://pubsubhub.bub.com''. Every Atom activity stream on the MediaGoblin server must then provide a link element (marked with a rel attribute "hub") pointing to the PuSH server, so we add <link href="http://pubsubhub.bub.com" rel="hub"/> to the Atom feed.

It is not as important but a common web feed courtesy not to forget a properly set link with self as a rel value (expressing where the feed is). So if we are trying to publish the activity stream of a user called yooser which is located at http://mediagoblin.server.com/u/yooser/as.atom we add the self link in the atom feed with <link rel="self" type="application/atom+xml" href="http://mediagoblin.server.com/u/yooser/as.atom" />. This should be taken care of by the framework but lets be sure.

Next step. When a new activity stream is created or an existing one updated, we need to publish it to the hub. It's a really simple POST operation. We need to encode two variables, ''hub.url'' and ''hub.mode''. The hub.url should contain the URL to the activity stream being published, which would in the case of our user ''yooser'' be http://mediagoblin.server.com/u/yooser/as.atom and the hub.mode must contain a single word: 'publish'. If the POSTing to the hub is successful the hub will return a 204 error code (No Content). If something goes wrong, the hub will return an appropriate error code.

In Python this is quite easy to do using urllib and urllib2:

<source lang="python">
hub = 'http://pubsubhub.bub.com'
url = 'http://mediagoblin.server.com/u/yooser/as.atom'

data = urllib.urlencode({'hub.url':url, 'hub.mode':'publish'})
try:
response = urllib2.urlopen(hub, data)
except (IOError, urllib2.HTTPError), e:
if hasattr(e, 'code') and e.code == 204:
print 'Successfully published to hub'
else:
print 'Gosh darn it Spencer! It failed!'
</source>

This example does not bother with error handling (except to check status code), wrapping it in a function and it uses hard coded values, all for the sake of clarity. A better example is available from the [http://code.google.com/p/pubsubhubbub/source/browse/trunk/publisher_clients/python/pubsubhubbub_publish.py pubsubhubbub source repository] under the Apache license.

=== Related Links ===

* http://code.google.com/p/pubsubhubbub/
* http://pubsubhubbub.googlecode.com/svn/trunk/pubsubhubbub-core-0.3.html
* http://code.google.com/p/pubsubhubbub/source/browse/trunk/publisher_clients/python/pubsubhubbub_publish.py (Apache License)

== WebFinger ==

WebFinger is a way to identify users by email addresses and allow sites to access more information about the specific user. To accomplish this WebFinger uses two resources: ''A domain specific host-meta file'' which contains information about how to access information about a specific user, and the individual user description files, called endpoints. Both of these resources are relatively simple XRD documents.

OStatus uses this to allow different federated sites with different routing structures to find the activity streams of users (and perhaps other information as well) easily.

=== How to Implement ===

For an example use case, lets say that we have a running MediaGoblin instance on ''mediagoblin.server.com'' and we want to provide WebFinger identities for our users and in our example case a user with a username ''yooser''. This user would get the WebFinger identity (email): ''yooser@mediagoblin.server.com''.

The MediaGoblin instance will have to set up a host-meta file and make it available at: http://mediagoblin.server.com/.well-known/host-meta

This is the file other services will GET, based on the host of the email address ''mediagoblin.server.com'' to find out where to get access to more information about the user ''yooser''.

The host-meta file can contain a lot of information about the site but the information other federated services are interested in is the Link-based Resource Descriptor Discovery protocol ''lrdd'' (pronounced: lard) information which is provided via a Link with a rel attribute ''lrdd'' and the real information is the associated template (also an attribute of this Link). The template defines a structure of where other federated services can find information about the user account. It is basically just a URL which contains a ''{uri}'' somewhere and this specific ''{uri}'' will be replaced with the user account identifier. This account identifier is the user email preceded by ''acct:'', i.e. acct:yooser@mediagoblin.server.com

MediaGoblin will then have to decide on how this template should be structured. This could be something like http://mediagoblin.server.com/describe?uri={uri} (which is provided in the discussion of WebFinger) or http://mediagoblin.server.com/main/xrd?uri={uri} (which is the structure on Status.Net). It doesn't really matter how it is structured, just that it includes the ''{uri}'' and conforms with the overall routing structure of MediaGoblin.

We must also provide a Host element in the host-meta file, in accordance with the host-meta description. For our example this would be <hm:Host>mediagoblin.server.com</hm:Host> (where the hm namespace is http://host-meta.net/xrd/1.0).

So let's take a look at an example of what other services might find when they GET the host-meta file for mediagoblin.server.com at http://mediagoblin.server.com/.well-known/host-meta (for this example we'll use the WebFinger template):

<source lang="xml">
<?xml version='1.0' encoding='UTF-8'?>
<XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'
xmlns:hm='http://host-meta.net/xrd/1.0'>

<hm:Host>mediagoblin.server.com</hm:Host>

<Link rel='lrdd'
template='http://mediagoblin.server.com/describe?uri={uri}'>
<Title>Resource Descriptor</Title>
</Link>
</XRD>
</source>

In Django this host-meta file is easily created using Django's template system and reverse URL lookups. This should be just as easy to do in MediaGoblin's framework.

Next we have to create the user specific description, i.e. the stuff other federated services will find when they access the template. This is done, as previously discussed by substituting {uri} with the account information (email preceded by acct:) and GETting it. So the other federated services would GET http://mediagoblin.server.com/describe?uri=acct:yooser@mediagoblin.server.com and that would return a whole lot of interesting user specific stuff.

The insteresting stuff is againg presented with XRD and Link elements and could for example be a Link to the profile page (<Link rel="http://webfinger.net/rel/profile-page" type="text/html" href="http://mediagoblin.server.com/u/yooser"/>), the user avatar (<Link rel="http://webfinger.net/rel/avatar" href="http://mediagoblin.server.com/u/yooser/avatar" />). The [http://ostatus.org/2010/10/04/how-ostatus-enable-your-application blog post] says that it should at least include a link to the updates using http://schemas.google.com/g/2010#updates-from which points the the Atom ActivityStream (<Link rel="http://schemas.google.com/g/2010#updates-from" type="application/atom+xml" href="http://mediagoblin.server.com/u/yooser/as.atom" />).

We must also declare the Subject (the user) in the XRD and we do that using the Subject element which contains the account identifier (email preceded by acct:). We can also Alias the user (using a Alias element) and provide a URL pointing to the user profile page.

So if we look at an example to combine all of this and provide the minimum information needed to enable OStatus (we can always provide more interesting stuff later on or add it when we actually implement WebFinger instead of just describing it on a Wiki page since we add it for subscription and salmon based stuff) the XRD returned when a federated service GETs http://mediagoblin.server.com/describe?uri=acct:yooser@mediagoblin.server.com would look like this:

<source lang="xml">
<?xml version='1.0' encoding='UTF-8'?>
<XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'>

<Subject>acct:yooser@mediagoblin.server.com</Subject>
<Alias>http://mediagoblin.server.com/u/yooser</Alias>

<Link rel="http://schemas.google.com/g/2010#updates-from"
type="application/atom+xml"
href="http://mediagoblin.server.com/u/yooser/as.atom" />
</XRD>
</source>

Again, this would be easy to do with Django's template system and reverse URL lookups, and should be just as easy to do with MediaGoblin's framework. When that is done MediaGoblin will have the WebFinger part of OStatus implemented.

=== Related Links ===

* https://code.google.com/p/webfinger/
* http://hueniverse.com/2009/08/introducing-webfinger/
* http://hueniverse.com/2009/09/implementing-webfinger/
* http://hueniverse.com/2009/03/the-discovery-protocol-stack/
* http://hueniverse.com/xrd/
* http://hueniverse.com/2009/11/host-meta-aka-site-meta-and-well-known-uris/

API

2011-11-11T12:35:58Z

Tryggvib: Broke up Submission and Metadata into two sections

This page describes a JSON-based API for MediaGoblin. It is a '''WORK IN PROGRESS''' at the moment while it is being drafted before it gets implemented. Currently it only includes a submission API, based on a specific scenario which might not work in general for MediaGoblin but can be used as a springboard for further API improvements and specifications.

= Scenario =

A MediaGoblin server (''MGS'') is used for media storage/repository on behalf of a few other servers, called user servers (''US'') . A user of one ''US'' submits a multimedia file. The file is sent to the ''MGS'' and categorized on the ''US'' in some server specific way. All future use of the multimedia file on the ''US'' is processed by the ''MGS''.

The collection of multimedia files from all of the ''US'''s is accessible through the ''MGS'' interface and through specific categorizations on each ''US'' via API or direct links to the content on the ''MGS''.

== Submission ==

Since all files are processed automatically when they are uploaded there is no need for a specific submission api. Media files should therefore just be uploaded using a normal HTTP POST which is made available via the API using some sort of an authentication mechanism. An extra variable, containing a webhook/callback URL, can be sent alongside the file upload. The callback provides a URL to the file in the public store. The ''MGS'' allows the ''US'' to send metadata (for example license information) for the file.

The process is as follows:

# User fills in a form on a ''US''. The form is in fact based up of two forms
#* One for categorization on the ''US''
#* One is a file form for the ''MGS''
# ''US'' saves a pending object with the categorization
# The ''MGS'' processes the file as usual
# When the ''MGS'' has finished processing it notifies the ''US'' through a special callback url provided that the processing is finished and the file is available in the public store through a specific url
# ''US'' moves the object from a pending state to a real object with direct links (or api specific information which might be designed later) to the ''MGS''
# ''US'' sends metadata to attach to the file.

=== Callback/finished processing API ===

The callback function of the ''US'' should read and accept MediaGoblin JSON objects according to the following proposed schema when sent to the callback url provided alongside the original upload:

<source lang="javascript">
{
"name":"Callback",
"properties":
{
"success":
{
"type":"boolean",
"description":"Boolean to indicate whether the processing of the file was successful or not",
"required":true
},
"message":
{
"type":"string",
"description":"Optional message used when processing was unsuccessful to explain the reason for the failure",
"required":false
},
"location":
{
"type":"array",
"required":true,
"description":"Array of locations for the processed file (or files, e.g. when video file is encoded into the three possible HTML5 video formats",
"items":
{
"type":"object",
"properties":
{
"mimetype":
{
"type":"string",
"description":"Mimetype to categorize the file",
"required":true,
},
"location":
{
"type":"string",
"format":"url",
"description":"URL of the file in the public store",
"required":true,
},
}
}
"mediatype":
{
"type":"string",
"description":"Media type of the file categorization, e.g. video, audio, image",
"required":false
}
}
}
</source>

The MediaGoblin expects status codes 204 (No content) on success (to imitate pubsubhubbub protocol) or normal error codes (4xx or 5xx) to indicate failures. It is possible to further define JSON objects in responses but probably unnecessary.

== Metadata ==

It should be possible to add metadata to a specific file at any time. This could be information such as license information, related files (e.g. derivative work), views on site or something. In this case we only start with a Creative Commons license.

=== Metadata API ===

A proposed json-based api to attach metadata to a file uploaded to a MediaGoblin instance. Other metadata information can be added later. This is just something to start with. It is necessary to check if the calling application is the same as the uploading application (to restrict others from setting metadata to other people's files).

The following is the JSON Schema for metadata:

<source lang="javascript">
{
"name":"Metadata",
"properties":
{
"file":
{
"type":"string",
"format":"url",
"description":"URL to the resource (file) to which metadata should be attached",
"required":true
},
"metadata":
{
"type":"array",
"description":"URL to the resource (file) to which metadata should be attached",
"required":true,
"items":
{
"type":"object",
"properties":
{
"creativecommons":
{
"type":"object",
"required":false,
"properties":
{
"author":
{
"type":"string",
"description":"Original author of the work, used for attribution",
"required":true,
},
"license":
{
"type":"string",
"format":"url",
"description":"URL pointing to the Creative Commons license used",
"required":true,
}
}
}
}
}
}
}
}
</source>

An example of a Claim in JSON would be (it remains to decide how to handle user uploads from sites where users aren't registerd on MediaGoblin):

<source lang="javascript">
{
"file":"http://media.goblin.com/u/artcollection/water-lilies.jpg",
"metadata":
[
"creativecommons":
{
"author":"Claude Monet",
"licence":"http://creativecommons.org/publicdomain/mark/1.0/"
}
]
}
</source>

== Possible design problems ==

* MediaGoblin is user based but who is the user when files are sent in by users on a specific site? Use OAuth/OpenID? Perhaps tie closely with OStatus federation. Probably best to follow Twitter API where applications must be registered and get an API key for uploading.
* Mediatype for callback needs a better definition in the API (what can be sent, e.g. .)

API

2011-11-11T12:19:53Z

Tryggvib: Changes to API proposal according to IRC discussion on #mediagoblin (I hope I didn't screw things up)

This page describes a JSON-based API for MediaGoblin. It is a '''WORK IN PROGRESS''' at the moment while it is being drafted before it gets implemented. Currently it only includes a submission API, based on a specific scenario which might not work in general for MediaGoblin but can be used as a springboard for further API improvements and specifications.

= Scenario =

A MediaGoblin server (''MGS'') is used for media storage/repository on behalf of a few other servers, called user servers (''US'') . A user of one ''US'' submits a multimedia file. The file is sent to the ''MGS'' and categorized on the ''US'' in some server specific way. All future use of the multimedia file on the ''US'' is processed by the ''MGS''.

The collection of multimedia files from all of the ''US'''s is accessible through the ''MGS'' interface and through specific categorizations on each ''US'' via API or direct links to the content on the ''MGS''.

== Submission ==

Since all files are processed automatically when they are uploaded there is no need for a specific submission api. Media files should therefore just be uploaded using a normal HTTP POST which is made available via the API using some sort of an authentication mechanism. An extra variable, containing a webhook/callback URL, can be sent alongside the file upload. The callback provides a URL to the file in the public store. The ''MGS'' allows the ''US'' to send metadata (for example license information) for the file.

The process is as follows:

# User fills in a form on a ''US''. The form is in fact based up of two forms
#* One for categorization on the ''US''
#* One is a file form for the ''MGS''
# ''US'' saves a pending object with the categorization
# The ''MGS'' processes the file as usual
# When the ''MGS'' has finished processing it notifies the ''US'' through a special callback url provided that the processing is finished and the file is available in the public store through a specific url
# ''US'' moves the object from a pending state to a real object with direct links (or api specific information which might be designed later) to the ''MGS''
# ''US'' sends metadata to attach to the file.

=== Callback/finished processing API ===

The callback function of the ''US'' should read and accept MediaGoblin JSON objects according to the following proposed schema when sent to the callback url provided alongside the original upload:

<source lang="javascript">
{
"name":"Callback",
"properties":
{
"success":
{
"type":"boolean",
"description":"Boolean to indicate whether the processing of the file was successful or not",
"required":true
},
"message":
{
"type":"string",
"description":"Optional message used when processing was unsuccessful to explain the reason for the failure",
"required":false
},
"location":
{
"type":"array",
"required":true,
"description":"Array of locations for the processed file (or files, e.g. when video file is encoded into the three possible HTML5 video formats",
"items":
{
"type":"object",
"properties":
{
"mimetype":
{
"type":"string",
"description":"Mimetype to categorize the file",
"required":true,
},
"location":
{
"type":"string",
"format":"url",
"description":"URL of the file in the public store",
"required":true,
},
}
}
"mediatype":
{
"type":"string",
"description":"Media type of the file categorization, e.g. video, audio, image",
"required":false
}
}
}
</source>

The MediaGoblin expects status codes 204 (No content) on success (to imitate pubsubhubbub protocol) or normal error codes (4xx or 5xx) to indicate failures. It is possible to further define JSON objects in responses but probably unnecessary.

=== Metadata API ===

A proposed json-based api to attach metadata to a file uploaded to a MediaGoblin instance. Other metadata information can be added later. This is just something to start with. It is necessary to check if the calling application is the same as the uploading application (to restrict others from setting metadata to other people's files).

The following is the JSON Schema for metadata:

<source lang="javascript">
{
"name":"Metadata",
"properties":
{
"file":
{
"type":"string",
"format":"url",
"description":"URL to the resource (file) to which metadata should be attached",
"required":true
},
"metadata":
{
"type":"array",
"description":"URL to the resource (file) to which metadata should be attached",
"required":true,
"items":
{
"type":"object",
"properties":
{
"creativecommons":
{
"type":"object",
"required":false,
"properties":
{
"author":
{
"type":"string",
"description":"Original author of the work, used for attribution",
"required":true,
},
"license":
{
"type":"string",
"format":"url",
"description":"URL pointing to the Creative Commons license used",
"required":true,
}
}
}
}
}
}
}
}
</source>

An example of a Claim in JSON would be:

<source lang="javascript">
{
"file":"http://media.goblin.com/b5d9f282-08ce-11e1-93a3-b32377194cc9.png",
"metadata":
[
"creativecommons":
{
"author":"Claude Monet",
"licence":"http://creativecommons.org/publicdomain/mark/1.0/"
}
]
}
</source>

=== Possible design problems ===

* MediaGoblin is user based but who is the user when files are sent in by users on a specific site? Use OAuth/OpenID? Perhaps tie closely with OStatus federation. Probably best to follow Twitter API where applications must be registered and get an API key for uploading.
* Mediatype needs a better definition in the API (what can be sent, e.g. .)

API

2011-11-06T23:52:42Z

Tryggvib: Created a page with a proposed start for the MediaGoblin API

This page describes a JSON-based API for MediaGoblin. It is a '''WORK IN PROGRESS''' at the moment while it is being drafted before it gets implemented. Currently it only includes a submission API, based on a specific scenario which might not work in general for MediaGoblin but can be used as a springboard for further API improvements and specifications.

= Scenario =

A MediaGoblin server (''MGS'') is used for media storage/repository on behalf of a few other servers, called user servers (''US'') . A user of one ''US'' submits a multimedia file. The file is sent to the ''MGS'' and categorized on the ''US'' in some server specific way. All future use of the multimedia file on the ''US'' is processed by the ''MGS''.

The collection of multimedia files from all of the ''US'''s is accessible through the ''MGS'' interface and through specific categorizations on each ''US'' via API or direct links to the content on the ''MGS''.

== Submission ==

For submission files are sent directly to the ''MGS'' for processing. Each ''US'' has to claim the uploaded multimedia file to start processing. The process is as follows:

# User fills in a form on a ''US''. The form is in fact based up of two forms
#* One for categorization on the ''US''
#* One is a file form for the ''MGS''
# File is sent to the ''MGS'' with a unique name
#* The name distinction should be designed to ensure uniqueness (perhaps uuid?)
# ''US'' saves a pending object with the categorization
# ''US'' claims the file through the unique name via an API which start the processing of the file
# The ''MGS'' processes the file as usual
# When the ''MGS'' has finished processing it notifies the ''US'' through a special callback url provided that the processing is finished and the file is available in the public store through a specific url
# ''US'' moves the object from a pending state to a real object with direct links (or api specific information which might be designed later) to the ''MGS''

=== File claim/start processing API ===

A proposed json-based api to claim a file uploaded to a MediaGoblin instance. When a file is claimed, the necessary variables are the unique filename, the callback url and the uploader (uses OStatus WebFinger id). The request can of course also contains other metadata and in this scenario we add Creative Commons metadata like original author and license.

The following is the JSON Schema for the file claim:

<source lang="javascript">
{
"name":"Claim",
"properties":
{
"filename":
{
"type":"string",
"description":"Unique, claimable filename of an uploaded file on the MediaGoblin server",
"required":true
},
"callback":
{
"type":"string",
"format":"url",
"description":"URL to call when processing is completed/failed for notification",
"required":true
},
"uploader":
{
"type":"string",
"format":"email",
"description":"Webfinger email address of the uploader for identification",
"required":true
},
"creativecommons":
{
"type":"object",
"required":false,
"properties":
{
"author":
{
"type":"string",
"description":"Original author of the work, used for attribution",
"required":true,
},
"license":
{
"type":"string",
"format":"url",
"description":"URL pointing to the Creative Commons license used",
"required":true,
}
}
}
}
}
</source>

An example of a Claim in JSON would be:

<source lang="javascript">
{
"filename":"b5d9f282-08ce-11e1-93a3-b32377194cc9.png",
"callback":"http://example.com/pending/b5d9f282-08ce-11e1-93a3-b32377194cc9/",
"uploader":"user@example.com",
"creativecommons":
{
"author":"Claude Monet",
"licence":"http://creativecommons.org/publicdomain/mark/1.0/"
}
}
</source>

=== Callback/finished processing API ===

The callback function of the ''US'' should read and accept MediaGoblin JSON objects according to the following proposed schema when sent to the callback url provided in the original claim:

<source lang="javascript">
{
"name":"Callback",
"properties":
{
"success":
{
"type":"boolean",
"description":"Boolean to indicate whether the processing of the file was successful or not",
"required":true
},
"message":
{
"type":"string",
"description":"Optional message used when processing was unsuccessful to explain the reason for the failure",
"required":false
},
"location":
{
"type":"array",
"required":true,
"description":"Array of locations for the processed file (or files, e.g. when video file is encoded into the three possible HTML5 video formats",
"items":
{
"type":"object",
"properties":
{
"mediatype":
{
"type":"string",
"description":"Optional string to categorize the file, e.g. ogv, webm",
"required":false,
},
"location":
{
"type":"string",
"format":"url",
"description":"URL of the file in the public store",
"required":true,
},
}
}
"mimetype":
{
"type":"string",
"description":"Mimetype of the file to allow receiver to categorize pending object",
"required":true
}
}
}
</source>

=== Possible design problems ===

* Requires a separate upload (or possibly a rewrite of the current upload) to enable ''file claims''
* A secure and unique filename to be claimed is needed
* MediaGoblin is user based but who is the user when files are sent in by users on a specific site? Use OAuth/OpenID? Perhaps tie closely with OStatus federation.
* Need a service to remove old unclaimed files (and what would be the time window?)
* Mediatype needs a better definition in the API (what can be sent, e.g. webm or WebM etc.)