GNU MediaGoblin Wiki - User contributions [en]

HackingHowto

2017-06-10T00:43:24Z

Breton: added info about rabbitmq-server for debian

= Hacking HOWTO =

== So you want to hack on GNU MediaGoblin? ==

First thing to do is check out the [http://mediagoblin.org/join/ web site] where we list all the project
infrastructure including:

* the IRC channel
* the mailing list
* the issue tracker

Additionally, we have information on how to get involved, who to talk
to, what needs to be worked on, and other things besides!

Second thing to do is take a look at [http://docs.mediagoblin.org/devel/codebase.html codebase chapter] where
we've started documenting how GNU MediaGoblin is built and how to add
new things. If you're planning on contributing in python, you should be aware
of [http://www.python.org/dev/peps/pep-0008/ PEP-8], the official Python style guide,
which we follow.

Third you'll need to get the requirements.

Fourth, you'll need to build a development environment. We use an
in-package checkout of virtualenv. This isn't the convenional way to
install virtualenv (normally you don't install virtualenv inside the
package itself) but we've found that it's significantly easier for
newcomers who aren't already familiar with virtualenv. If you *are*
already familiar with virtualenv, feel free to just install
mediagoblin in your own virtualenv setup... the necessary adjustments
should be obvious.

== Python 2 vs Python 3 ==

GNU MediaGoblin was originally written for Python 2, and much of the documentation on the wiki reflects that. However, since release v0.9.0, GNU MediaGoblin has attempted to provide support for Python 3. Unfortunately, not everything is working for Python 3 environments yet. Below is a non-exhaustive list of things that might not work if you set up GNU MediaGoblin a Python 3 environment.

=== Python 3 Issues ===

* fcgi - none of the flup packages ([https://pypi.python.org/pypi/flup flup], [https://pypi.python.org/pypi/flup-py3 flup-py3], [https://pypi.python.org/pypi/flup6 flup6]) appear to have proper support for Python 3.5.
* audio file spectrographs - [https://pypi.python.org/pypi/scikits.audiolab scikits.audiolab] [https://issues.mediagoblin.org/ticket/5467 appears to not work] in Python 3.
* Rackspace storage - the [https://pypi.python.org/pypi/python-cloudfiles cloudfiles] plugin may not work with Python 3.

== Getting requirements ==

First, you need to have the following installed before you can build
an environment for hacking on GNU MediaGoblin:

* Python 2.7 - http://www.python.org/ (You'll need Python as well as the dev files for building modules.)
* python-lxml - http://lxml.de/
* git - http://git-scm.com/
* SQLAlchemy 0.7.0 or higher - http://www.sqlalchemy.org/
* Pillow - http://python-pillow.org/
* virtualenv - http://www.virtualenv.org/
* Python GStreamer Bindings - http://gstreamer.freedesktop.org/modules/gst-python.html
* Node.js
* RabbitMQ server

=== GNU/Linux ===

==== Debian and derivatives ====

If you're running Debian GNU/Linux or a Debian-derived distribution
such as Debian, Mint, or [http://bugs.foocorp.net/issues/478 Ubuntu 10.10+], running the following should install these
requirements:

{{Cmd|sudo apt-get install git-core python python-dev python-lxml python-imaging python-virtualenv python-gst-1.0 libjpeg8-dev autoconf nodejs npm nodejs-legacy rabbitmq-server}}

==== Debian GNU/Linux jessie ====

{{Cmd|sudo apt-get install -y git-core python python-dev python-lxml python-imaging python-virtualenv libjpeg-dev autoconf nodejs npm nodejs-legacy python-gst-1.0 gstreamer1.0-plugins-base gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-libav python-numpy python-scipy libsndfile1-dev libasound2-dev libgstreamer-plugins-base1.0-dev rabbitmq-server}}

==== Fedora / RedHat(?) / CentOS ====

===== On Fedora: =====

{{Cmd|yum install python-paste-deploy python-paste-script git-core python python-devel python-lxml python-imaging python-virtualenv gstreamer-python nodejs}}

===== On CentOS7: =====

'''epel-release''' is the prerequisite for most of the below listed packages

{{Cmd|sudo yum install epel-release}}

{{Cmd|sudo yum update}}

{{Cmd|sudo yum install python-paste-deploy python-paste-script \ git-core python python-devel python-lxml python-imaging \ python-virtualenv npm automake nginx gcc}}

for video to work correctly (as per [https://mediagoblin.readthedocs.io/en/stable/siteadmin/media-types.html?highlight=video#video Enabling Media-Types Video]) you may choose to install AWEL 3rd party repo which provides with the latest gstreamer bindings and more:

* [http://awel.domblogger.net/7/media/x86_64/repoview/awel-media-release.html AWEL Repo],[https://media.librelamp.com/gst AWEL Gstreamer]

once installed:

{{Cmd|sudo yum install gstreamer1 gstreamer1-libav gstreamer1-plugins-base gstreamer1-plugins-base-tools gstreamer1-plugins-good gstreamer1-plugins-ugly gstreamer1-plugins-bad gstreamer1-python ffmpeg }}

==== openSUSE ====

This instructions might be incomplete.
Try on openSUSE 13.2 or 13.1:

{{Cmd|zypper install python-devel python-gstreamer-0_10 git-core python python-lxml python-Pillow python-virtualenv npm}}

==== ArchLinux / Parabola ====

The following command should work (<del>not tested on a new ArchLinux / Parabola install</del>. tested, it works):

{{Cmd|pacman -S git python2 python2-lxml python2-pillow python2-virtualenv gstreamer0.10-python}}

=== Mac OS X ===

==== Mac OS X Lion ====

Download the Newest Python.

Git is already installed.

* Note for Pillow and lxml, you can: pip install Pillow lxml

Python-lxml: http://muffinresearch.co.uk/archives/2009/03/05/install-lxml-on-osx/ with sudo

Pillow: http://pillow.readthedocs.org/en/3.0.x/installation.html#os-x-installation

Libjpeg & Libpng: http://ethan.tira-thompson.com/Mac_OS_X_Ports.html Combo Installer

==== Mac OS X Snow Leopard ====

# You will probably want to install MacPorts this will give you access to many free software packages in the same manner to apt-get and yum: https://www.macports.org/install.php
# Ensure you install Git and the command line tools: https://help.github.com/articles/set-up-git#platform-mac
# Once both of those are installed type this in your terminal and enter your password when prompted for it {{Cmd|sudo port install python27 py27-lxml py27-sqlalchemy py27-Pillow py27-virtualenv py27-gst-python py27-pastescript}}

=== Microsoft Windows ===

''Thanks wctype!''

==== Getting requirements ====

* Python 2.7 - [http://www.python.org/download/ Download] 
* git - [https://github.com/msysgit/git/downloads Download] 
* python-lxml - [http://pypi.python.org/pypi/lxml/2.3.5#downloads Tarball] [http://www.lfd.uci.edu/~gohlke/pythonlibs/#pil Binaries] 
* Pillow - [http://pillow.readthedocs.org/en/3.0.x/installation.html#windows-installation Download] 
* virtualenv - [http://pypi.python.org/pypi/virtualenvwrapper-win/1.0.8#downloads Download] 
* OSSBuild project provides reasonably up-to-date binaries of GStreamer - [https://code.google.com/p/ossbuild/downloads/list Download] 
* py-bcrypt - [https://bitbucket.org/alexandrul/py-bcrypt/downloads/ Download] 
* Node JS

----

'''You can help:'''

If you have instructions for other GNU/Linux distributions, Windows, or Mac OS X to set
up requirements, [http://mediagoblin.org/join/ let us know]!

== How to set up and maintain an environment for hacking with virtualenv ==

'''Requirements'''

No additional requirements.

'''Create a development environment'''

After installing the requirements, follow these steps:

* Clone the repository: {{Cmd|git clone <nowiki>git://git.savannah.gnu.org/mediagoblin.git</nowiki>}}
* Make the environment: {{Cmd|./bootstrap.sh && ./configure && make}}
* Init the database:
{{Cmd|./bin/gmg dbupdate}}

That's it!

If you want to make sure things are working, consider running the test suite:
{{Cmd|./runtests.sh}}

(If you have troubles in the remaining steps, consider try installing
virtualenv with one of the flags --setuptools, --distribute or possibly --no-site-packages. Additionally, if your system has python3.X as the default, you might need to do virtualenv --python=python2.7 or --python=python2.6)

If you have problems, please [http://mediagoblin.org/join/ let us know]!

== Updating an existing environment ==

'''Updating for dependency changes'''

While hacking on GNU MediaGoblin over time, you'll eventually have to
update your development environment because the dependencies have
changed.

To do that, run:

{{Cmd|./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate}}

'''Updating for code changes'''

{{Cmd|git pull -u}}
{{Cmd|git submodule update}}

'''NOTE:''' MediaGoblin used to be hosted on gitorious.org, but that is moving to read-only mode, so we're now on Savannah. If you have an old checkout, please update it by running the following:

{{Cmd|git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git}}

== Running the server ==

If you want to get things running quickly and without hassle, just
run:

{{Cmd|./lazyserver.sh}}

This will start up a python server where you can begin playing with
mediagoblin, listening on 127.0.0.1:6543. It will also run celery in "always eager" mode so you
don't have to start a separate process for it.

By default, the instance is not sending out confirmation mails. Instead they are redirected to the standard output (the console) of lazyserver.sh.

You can change this behavior setting <code>email_debug_mode</code> to <code>false</code> in mediagoblin.ini

This is fine in development, but if you want to actually run celery
separately for testing (or deployment purposes), you'll want to run
the server independently:

{{Cmd|./bin/paster serve paste.ini --reload}}

== Running celeryd ==

If you aren't using <tt>./lazyserver.sh</tt> or otherwise aren't running celery
in always eager mode, you'll need to do this if you want your media to
process and actually show up. It's probably a good idea in
development to have the web server (above) running in one terminal and
celeryd in another window.

Run:

{{Cmd|<nowiki>CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd</nowiki>}}

== Running the test suite ==

See also [[UnitTests]] and [[Manual Functional Testing]].

Run:

{{Cmd|./runtests.sh}}

== Running a shell ==

If you want a shell with your database pre-setup and an instantiated
application ready and at your fingertips ......

Run:

{{Cmd|./bin/gmg shell}}

== Troubleshooting ==

If you see a python error, search for the full line in [https://issues.mediagoblin.org/report trac] to find issues that mention the same . If it has not been fixed, you can update corresponding bug reports or open a new one.

For developers it is a special pleasure to test and apply user submitted patches. Here is a good minimal [https://issues.mediagoblin.org/timeline?from=2016-02-28T17%3A16%3A39-05%3A00&precision=second git workflow] as example.

== Wiping your user data ==

You can completely wipe all data from the instance by doing:

{{Cmd|rm -rf mediagoblin.db kombu.db celery.db user_dev; ./bin/gmg dbupdate}}

'''Note:'''

Unless you're doing development and working on and testing creating
a new instance, you will probably never have to do this.

== Quickstart for Django programmers ==

We're not using Django, but the codebase is very Django-like in its
structure.

* <tt>routing.py</tt> is like <tt>urls.py</tt> in Django
* <tt>models.py</tt> has SQLAlchemy ORM definitions
* <tt>views.py</tt> is where the views go

We're using SQLAlchemy, which is semi-similar to the Django ORM, but
not really because you can get a lot more fine-grained. The
[http://docs.sqlalchemy.org/en/latest/orm/tutorial.html SQLAlchemy ORM tutorial] is a great place to start.

'''YouCanHelp'''

If there are other things that you think would help orient someone
new to GNU MediaGoblin but coming from Django, let us know!

== Showing off your work with PageKite ==

If you're doing development with MediaGoblin, it's sometimes helpful to show off your work to gather feedback from other contributors. A number of the MediaGoblin developers use something called [http://pagekite.net PageKite], which is a fellow free software web service which makes temporarily showing off work on your machine easy. There's a [http://pagekite.net/wiki/Howto/UsePageKiteWithMediaGoblin/ tutorial on how to use PageKite and MediaGoblin together] available on the PageKite wiki.

If you are doing a lot of MediaGoblin development, the PageKite people have graciously offered us a good amount of bandwidth at no cost in an effort to help out fellow free software projects. If you've been making significant contributions, PM Chris Webber on freenode (who is paroneayea there) and ask if you can be added to our group plan.

== Bite-sized bugs to start with ==

Now you should visit our latest list of [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bite-sized issues] because squishing bugs is messy fun. If you're interested in other things to work on, or need help getting started on a bug, let us know on [http://mediagoblin.org/join/ the mailing list] or on the [http://mediagoblin.org/join/ IRC channel].

GSOC 2017

2017-03-04T19:08:29Z

Breton: /* Chunked uploads / multi-upload UI. */

We are hoping to participate in [https://developers.google.com/open-source/gsoc/ GSOC 2017] under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net !

= How do I apply as a student ? =

* [https://developers.google.com/open-source/gsoc/ Submit your application]. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start ! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship ! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We would love to hear from you !

= Possible projects with mentors =
== Multiple qualities of video ==

Now MediaGoblin supports only a single quality of video files. We want to fix that and support multiple qualities: 360p, 480p, 720p, 1080p, [https://en.wikipedia.org/wiki/Ultra-high-definition_television 4k]. This requires changes to frontend and to backend. To implement it, one must have some experience in JavaScript and Python.

There are some hidden issues though. For example, when a user uploads a video in 1080p, it will have to be transcoded to all other qualities. But it will take a lot of time. Ideally, we don't want to wait for all variants to be ready before we can already display the video. Also, if several videos were uploaded, we don't want to get blocked on the first one getting transcoded for a long time.

* '''Mentor''': Boris Bobrov (breton)
* '''Difficulty''': Medium or High, depending on the design
* '''Required skills''': Python, JavaScript. Nice to have: celery, gstreamer

== Port federation code to ActivityPub ==

Right now we have a federation branch to be merged shortly by Jessica Tallon. This adds basic federation support to MediaGoblin. Right now the federation code is based on the [https://github.com/e14n/pump.io/blob/master/API.md Pump API]; however, we'd like to port it to the work we're doing on https://www.w3.org/TR/activitypub/[https://www.w3.org/TR/activitypub/ ActivityPub] as part of the [https://www.w3.org/wiki/Socialwg W3C Social Working Group]. ActivityPub is fairly close to the Pump API, but there are some differences (most notably a move from ActivityStreams 1.0 to ActivityStreams 2.0, although there are others). Your mission, if you choose to accept it, is to port MediaGoblin's federation code to ActivityPub, ideally while supporting the Pump API in a backwards compatible fashion (if possible).

Jessica Tallon willing to personally mentor this one. So if you wanted to learn about federation from the source itself, here's the way to do it!

* '''Possible mentors''': Jessica Tallon (tsyesika)
* '''Difficulty''': TODO
* '''Required skills''': Python

GSOC 2017

2017-03-04T17:36:25Z

Breton: drop access control task

GSOC 2017

2017-02-22T20:25:15Z

Breton:

GSOC 2017

2017-02-22T20:11:48Z

Breton: /* Chunked uploads / multi-upload UI. */

We are hoping to participate in [https://developers.google.com/open-source/gsoc/ GSOC 2017] under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net !

= How do I apply as a student ? =

* [https://developers.google.com/open-source/gsoc/ Submit your application]. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start ! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship ! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We would love to hear from you !

= Possible projects with mentors =
== Multiple qualities of video ==

Now MediaGoblin supports only a single quality of video files. We want to fix that and support multiple qualities: 360p, 480p, 720p, 1080p, [https://en.wikipedia.org/wiki/Ultra-high-definition_television 4k]. This requires changes to frontend and to backend. To implement it, one must have some experience in JavaScript and Python.

There are some hidden issues though. For example, when a user uploads a video in 1080p, it will have to be transcoded to all other qualities. But it will take a lot of time. Ideally, we don't want to wait for all variants to be ready before we can already display the video. Also, if several videos were uploaded, we don't want to get blocked on the first one getting transcoded for a long time.

* '''Mentor''': Boris Bobrov (breton)
* '''Difficulty''': Medium or High, depending on the design
* '''Required skills''': Python, JavaScript. Nice to have: celery, gstreamer

== Port federation code to ActivityPub ==

Right now we have a federation branch to be merged shortly by Jessica Tallon. This adds basic federation support to MediaGoblin. Right now the federation code is based on the [https://github.com/e14n/pump.io/blob/master/API.md Pump API]; however, we'd like to port it to the work we're doing on https://www.w3.org/TR/activitypub/[https://www.w3.org/TR/activitypub/ ActivityPub] as part of the [https://www.w3.org/wiki/Socialwg W3C Social Working Group]. ActivityPub is fairly close to the Pump API, but there are some differences (most notably a move from ActivityStreams 1.0 to ActivityStreams 2.0, although there are others). Your mission, if you choose to accept it, is to port MediaGoblin's federation code to ActivityPub, ideally while supporting the Pump API in a backwards compatible fashion (if possible).

Is Jessica Tallon willing to personally mentor this one? /* so if you wanted to learn about federation from the source itself, here's the way to do it! */

* '''Possible mentors''': Jessica Tallon (tsyesika)
* '''Difficulty''': TODO
* '''Required skills''': Python

== Access control ==

Today MediaGoblin doesn't handle permissions very well. Anybody can see anything. What should be is proper access control. Owner of media or collection should be able to set who can view, edit and comment their mediafiles. Users could be added to groups, and groups could participate in permissions too. Different permissions might also apply to guests and registered users. Who can add users to groups should be controlled too.

A good proposal would include description of capabilities of the access control system. Don't forget that REST should be covered too. Also, it should be clear how the whole thing works with federation.

* '''Possible mentors''': Boris Bobrov (breton)
* '''Difficulty''': Medium
* '''Required skills''': Python, HTML, CSS

On top of that, audit system might be implemented. Who added user to group? Who edited or deleted a file? Who changed a permission?

Adding audit will bump '''difficulty''' of the task to 4.

== Chunked uploads / multi-upload UI. ==

We want to have multi-upload from cli (usecase: go to my directory with photos and upload them all) and from web-interface (using maybe zip, maybe html5 feature). There is already https://github.com/VenKamikaze/mediagoblin-html5-multi-upload which might be what we need, so before choosing this task please verify.

* '''Possible Mentors:''' Christopher Allan Webber (paroneayea), Boris Bobrov (breton).
* '''Difficulty''': low
* '''Required skills''': Python, HTML, CSS

GSOC 2017

2017-02-22T20:06:41Z

Breton:

We are hoping to participate in [https://developers.google.com/open-source/gsoc/ GSOC 2017] under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net !

= How do I apply as a student ? =

* [https://developers.google.com/open-source/gsoc/ Submit your application]. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start ! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship ! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We would love to hear from you !

= Possible projects with mentors =
== Multiple qualities of video ==

Now MediaGoblin supports only a single quality of video files. We want to fix that and support multiple qualities: 360p, 480p, 720p, 1080p, [https://en.wikipedia.org/wiki/Ultra-high-definition_television 4k]. This requires changes to frontend and to backend. To implement it, one must have some experience in JavaScript and Python.

There are some hidden issues though. For example, when a user uploads a video in 1080p, it will have to be transcoded to all other qualities. But it will take a lot of time. Ideally, we don't want to wait for all variants to be ready before we can already display the video. Also, if several videos were uploaded, we don't want to get blocked on the first one getting transcoded for a long time.

* '''Mentor''': Boris Bobrov (breton)
* '''Difficulty''': Medium or High, depending on the design
* '''Required skills''': Python, JavaScript. Nice to have: celery, gstreamer

== Port federation code to ActivityPub ==

Right now we have a federation branch to be merged shortly by Jessica Tallon. This adds basic federation support to MediaGoblin. Right now the federation code is based on the [https://github.com/e14n/pump.io/blob/master/API.md Pump API]; however, we'd like to port it to the work we're doing on https://www.w3.org/TR/activitypub/[https://www.w3.org/TR/activitypub/ ActivityPub] as part of the [https://www.w3.org/wiki/Socialwg W3C Social Working Group]. ActivityPub is fairly close to the Pump API, but there are some differences (most notably a move from ActivityStreams 1.0 to ActivityStreams 2.0, although there are others). Your mission, if you choose to accept it, is to port MediaGoblin's federation code to ActivityPub, ideally while supporting the Pump API in a backwards compatible fashion (if possible).

Is Jessica Tallon willing to personally mentor this one? /* so if you wanted to learn about federation from the source itself, here's the way to do it! */

* '''Possible mentors''': Jessica Tallon (tsyesika)
* '''Difficulty''': TODO
* '''Required skills''': Python

== Access control ==

Today MediaGoblin doesn't handle permissions very well. Anybody can see anything. What should be is proper access control. Owner of media or collection should be able to set who can view, edit and comment their mediafiles. Users could be added to groups, and groups could participate in permissions too. Different permissions might also apply to guests and registered users. Who can add users to groups should be controlled too.

A good proposal would include description of capabilities of the access control system. Don't forget that REST should be covered too. Also, it should be clear how the whole thing works with federation.

* '''Possible mentors''': Boris Bobrov (breton)
* '''Difficulty''': Medium
* '''Required skills''': Python, HTML, CSS

On top of that, audit system might be implemented. Who added user to group? Who edited or deleted a file? Who changed a permission?

Adding audit will bump '''difficulty''' of the task to 4.

== Chunked uploads / multi-upload UI. ==

We want to have multi-upload from cli (usecase: go to my directory with photos and upload them all) and from web-interface (using maybe zip, maybe html5 feature). There is already https://github.com/VenKamikaze/mediagoblin-html5-multi-upload which might be what we need, so before choosing this task please verify.

* '''Possible Mentors:''' Christopher Allan Webber (paroneayea), Boris Bobrov (breton).
* '''Difficulty''': low
* '''Required skills''': python, html, css

GSOC 2017

2017-02-22T19:48:39Z

Breton: /* Access control */

We are hoping to participate in [https://developers.google.com/open-source/gsoc/ GSOC 2017] under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net !

= How do I apply as a student ? =

* [https://developers.google.com/open-source/gsoc/ Submit your application]. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start ! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship ! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We would love to hear from you !

= Possible projects with mentors =
== Multiple qualities of video ==

Now MediaGoblin supports only a single quality of video files. We want to fix that and support multiple qualities: 360p, 480p, 720p, 1080p, [https://en.wikipedia.org/wiki/Ultra-high-definition_television 4k]. This requires changes to frontend and to backend. To implement it, one must have some experience in JavaScript and Python.

There are some hidden issues though. For example, when a user uploads a video in 1080p, it will have to be transcoded to all other qualities. But it will take a lot of time. Ideally, we don't want to wait for all variants to be ready before we can already display the video. Also, if several videos were uploaded, we don't want to get blocked on the first one getting transcoded for a long time.

* '''Mentor''': Boris Bobrov (breton)
* '''Difficulty''': 2-4 out of 5 depending on what will be implemented
* '''Required skills''': Python, JavaScript. Nice to have: celery, gstreamer

== Port federation code to ActivityPub ==

Right now we have a federation branch to be merged shortly by Jessica Tallon. This adds basic federation support to MediaGoblin. Right now the federation code is based on the [https://github.com/e14n/pump.io/blob/master/API.md Pump API]; however, we'd like to port it to the work we're doing on https://www.w3.org/TR/activitypub/[https://www.w3.org/TR/activitypub/ ActivityPub] as part of the [https://www.w3.org/wiki/Socialwg W3C Social Working Group]. ActivityPub is fairly close to the Pump API, but there are some differences (most notably a move from ActivityStreams 1.0 to ActivityStreams 2.0, although there are others). Your mission, if you choose to accept it, is to port MediaGoblin's federation code to ActivityPub, ideally while supporting the Pump API in a backwards compatible fashion (if possible).

Is Jessica Tallon willing to personally mentor this one? /* so if you wanted to learn about federation from the source itself, here's the way to do it! */

* '''Possible mentors''': Jessica Tallon (tsyesika)
* '''Difficulty''': TODO
* '''Required skills''': Python

== Access control ==

Today MediaGoblin doesn't handle permissions very well. Anybody can see anything. What should be is proper access control. Owner of media or collection should be able to set who can view, edit and comment their mediafiles. Users could be added to groups, and groups could participate in permissions too. Different permissions might also apply to guests and registered users. Who can add users to groups should be controlled too.

A good proposal would include description of capabilities of the access control system. Don't forget that REST should be covered too. Also, it should be clear how the whole thing works with federation.

* '''Possible mentors''': Boris Bobrov (breton)
* '''Difficulty''': 3 out of 5 if only access control is implemented. 4 out of 5 if audit is implemented
* '''Required skills''': Python, HTML, CSS

On top of that, audit system might be implemented. Who added user to group? Who edited or deleted a file? Who changed a permission?

Adding audit will bump '''difficulty''' of the task to 4.

GSOC 2017

2017-02-22T19:41:14Z

Breton:

We are hoping to participate in [https://developers.google.com/open-source/gsoc/ GSOC 2017] under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net !

= How do I apply as a student ? =

* [https://developers.google.com/open-source/gsoc/ Submit your application]. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start ! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship ! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We would love to hear from you !

= Possible projects with mentors =
== Multiple qualities of video ==

Now MediaGoblin supports only a single quality of video files. We want to fix that and support multiple qualities: 360p, 480p, 720p, 1080p, [https://en.wikipedia.org/wiki/Ultra-high-definition_television 4k]. This requires changes to frontend and to backend. To implement it, one must have some experience in JavaScript and Python.

There are some hidden issues though. For example, when a user uploads a video in 1080p, it will have to be transcoded to all other qualities. But it will take a lot of time. Ideally, we don't want to wait for all variants to be ready before we can already display the video. Also, if several videos were uploaded, we don't want to get blocked on the first one getting transcoded for a long time.

* '''Mentor''': Boris Bobrov (breton)
* '''Difficulty''': 2-4 out of 5 depending on what will be implemented
* '''Required skills''': Python, JavaScript. Nice to have: celery, gstreamer

== Port federation code to ActivityPub ==

Right now we have a federation branch to be merged shortly by Jessica Tallon. This adds basic federation support to MediaGoblin. Right now the federation code is based on the [https://github.com/e14n/pump.io/blob/master/API.md Pump API]; however, we'd like to port it to the work we're doing on https://www.w3.org/TR/activitypub/[https://www.w3.org/TR/activitypub/ ActivityPub] as part of the [https://www.w3.org/wiki/Socialwg W3C Social Working Group]. ActivityPub is fairly close to the Pump API, but there are some differences (most notably a move from ActivityStreams 1.0 to ActivityStreams 2.0, although there are others). Your mission, if you choose to accept it, is to port MediaGoblin's federation code to ActivityPub, ideally while supporting the Pump API in a backwards compatible fashion (if possible).

Is Jessica Tallon willing to personally mentor this one? /* so if you wanted to learn about federation from the source itself, here's the way to do it! */

* '''Possible mentors''': Jessica Tallon (tsyesika)
* '''Difficulty''': TODO
* '''Required skills''': Python

== Access control ==

Today MediaGoblin doesn't handle permissions very well. Anybody can see anything. What should be is proper access control. Owner of media or collection should be able to set who can view, edit and comment their mediafiles. Users could be added to groups, and groups could participate in permissions too. Different permissions might also apply to guests and registered users. Who can add users to groups? Who added user to group? Who edited or deleted a file? Who changed a permission?

A good proposal would include description of capabilities of the access control system and, maybe, audit system. Don't forget that REST should be covered too. Also, it should be clear how the whole thing works with federation.

* '''Possible mentors''': Boris Bobrov (breton).
* '''Difficulty''': 3 out of 5 if only access control is implemented. 4 out of 5 if you
* '''Required skills''': Python, HTML, CSS.

GSOC 2017

2017-02-22T19:28:53Z

Breton: Added required skills

We are hoping to participate in [https://developers.google.com/open-source/gsoc/ GSOC 2017] under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net !

= How do I apply as a student ? =

* [https://developers.google.com/open-source/gsoc/ Submit your application]. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start ! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship ! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We would love to hear from you !

= Possible projects with mentors =
== Multiple qualities of video ==

Now MediaGoblin supports only a single quality of video files. We want to fix that and support multiple qualities: 360p, 480p, 720p, 1080p, [https://en.wikipedia.org/wiki/Ultra-high-definition_television 4k]. This requires changes to frontend and to backend. To implement it, one must have some experience in JavaScript and Python.

There are some hidden issues though. For example, when a user uploads a video in 1080p, it will have to be transcoded to all other qualities. But it will take a lot of time. Ideally, we don't want to wait for all variants to be ready before we can already display the video. Also, if several videos were uploaded, we don't want to get blocked on the first one getting transcoded for a long time.

* '''Mentor''': Boris Bobrov (breton)
* '''Difficulty''': 2-4 out of 5 depending on what will be implemented
* '''Required skills''': Python, JavaScript. Nice to have: celery, gstreamer

== Port federation code to ActivityPub ==

Right now we have a federation branch to be merged shortly by Jessica Tallon. This adds basic federation support to MediaGoblin. Right now the federation code is based on the [https://github.com/e14n/pump.io/blob/master/API.md Pump API]; however, we'd like to port it to the work we're doing on https://www.w3.org/TR/activitypub/[https://www.w3.org/TR/activitypub/ ActivityPub] as part of the [https://www.w3.org/wiki/Socialwg W3C Social Working Group]. ActivityPub is fairly close to the Pump API, but there are some differences (most notably a move from ActivityStreams 1.0 to ActivityStreams 2.0, although there are others). Your mission, if you choose to accept it, is to port MediaGoblin's federation code to ActivityPub, ideally while supporting the Pump API in a backwards compatible fashion (if possible).

Is Jessica Tallon willing to personally mentor this one? /* so if you wanted to learn about federation from the source itself, here's the way to do it! */

* '''Possible mentors''': Jessica Tallon (tsyesika)
* '''Difficulty''': TODO
* '''Required skills''': Python

== Access control ==

Today MediaGoblin doesn't handle permissions very well. Anybody can see anything. What should be is proper access control. Owner of media or collection should be able to set who can view, edit and comment their mediafiles. Users could be added to groups, and groups could participate in permissions too. Different permissions might also apply to guests and registered users. Who can add users to groups? Who added user to group? Who edited or deleted a file? Who changed a permission?

A good proposal would include description of capabilities of the access control system and, maybe, audit system. Don't forget that REST should be covered too. Also, it should be clear how the whole thing works with federation.

* '''Possible mentors''': Boris Bobrov (breton).
* '''Difficulty''': 2-4 out of 5 depending on what will be implemented
* '''Required skills''': Python, HTML, CSS.

GSOC 2017

2017-02-22T19:01:48Z

Breton:

We are hoping to participate in [https://developers.google.com/open-source/gsoc/ GSOC 2017] under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net !

= How do I apply as a student ? =

* [https://developers.google.com/open-source/gsoc/ Submit your application]. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start ! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship ! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We would love to hear from you !

= Possible projects with mentors =
== Multiple qualities of video ==

Now MediaGoblin supports only a single quality of video files. We want to fix that and support multiple qualities: 360p, 480p, 720p, 1080p, [https://en.wikipedia.org/wiki/Ultra-high-definition_television 4k]. This requires changes to frontend and to backend. To implement it, one must have some experience in JavaScript and Python.

There are some hidden issues though. For example, when a user uploads a video in 1080p, it will have to be transcoded to all other qualities. But it will take a lot of time. Ideally, we don't want to wait for all variants to be ready before we can already display the video. Also, if several videos were uploaded, we don't want to get blocked on the first one getting transcoded for a long time.

* '''Mentor''': Boris Bobrov (breton)
* '''Difficulty''': 2-4 out of 5 depending on what will be implemented

== Port federation code to ActivityPub ==

Right now we have a federation branch to be merged shortly by Jessica Tallon. This adds basic federation support to MediaGoblin. Right now the federation code is based on the [https://github.com/e14n/pump.io/blob/master/API.md Pump API]; however, we'd like to port it to the work we're doing on https://www.w3.org/TR/activitypub/[https://www.w3.org/TR/activitypub/ ActivityPub] as part of the [https://www.w3.org/wiki/Socialwg W3C Social Working Group]. ActivityPub is fairly close to the Pump API, but there are some differences (most notably a move from ActivityStreams 1.0 to ActivityStreams 2.0, although there are others). Your mission, if you choose to accept it, is to port MediaGoblin's federation code to ActivityPub, ideally while supporting the Pump API in a backwards compatible fashion (if possible).

Is Jessica Tallon willing to personally mentor this one? /* so if you wanted to learn about federation from the source itself, here's the way to do it! */

'''Possible mentors''': Jessica Tallon (tsyesika)

== Access control ==

Today MediaGoblin doesn't handle permissions very well. Anybody can see anything. What should be is proper access control. Owner of media or collection should be able to set who can view, edit and comment their mediafiles. Users could be added to groups, and groups could participate in permissions too. Different permissions might also apply to guests and registered users. Who can add users to groups? Who added user to group? Who edited or deleted a file? Who changed a permission?

A good proposal would include description of capabilities of the access control system and, maybe, audit system. Don't forget that REST should be covered too. Also, it should be clear how the whole thing works with federation.

* '''Possible mentors''': Boris Bobrov (breton).
* '''Difficulty''': 2-4 out of 5 depending on what will be implemented

GSOC 2017

2017-02-22T14:39:00Z

Breton:

We are hoping to participate in [https://developers.google.com/open-source/gsoc/ GSOC 2017] under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net !

= How do I apply as a student ? =

* [https://developers.google.com/open-source/gsoc/ Submit your application]. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start ! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship ! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We would love to hear from you !

= Possible projects with mentors. =
== Multiple qualities of video. ==

Now MediaGoblin supports only a single quality of video files. We want to fix that and support multiple qualities: 360p, 480p, 720p, 1080p, [https://en.wikipedia.org/wiki/Ultra-high-definition_television 4k]. This requires changes to frontend and to backend. To implement it, one must have some experience in JavaScript and Python.

There are some hidden issues though. For example, when a user uploads a video in 1080p, it will have to be transcoded to all other qualities. But it will take a lot of time. Ideally, we don't want to wait for all variants to be ready before we can already display the video. Also, if several videos were uploaded, we don't want to get blocked on the first one getting transcoded for a long time.

* '''Mentor''': Boris Bobrov (breton)
* '''Difficulty''': 2-4 out of 5 depending on what will be implemented

== Port federation code to ActivityPub ==

Right now we have a federation branch to be merged shortly by Jessica Tallon. This adds basic federation support to MediaGoblin. Right now the federation code is based on the [https://github.com/e14n/pump.io/blob/master/API.md Pump API]; however, we'd like to port it to the work we're doing on https://www.w3.org/TR/activitypub/[https://www.w3.org/TR/activitypub/ ActivityPub] as part of the [https://www.w3.org/wiki/Socialwg W3C Social Working Group]. ActivityPub is fairly close to the Pump API, but there are some differences (most notably a move from ActivityStreams 1.0 to ActivityStreams 2.0, although there are others). Your mission, if you choose to accept it, is to port MediaGoblin's federation code to ActivityPub, ideally while supporting the Pump API in a backwards compatible fashion (if possible).

Is Jessica Tallon willing to personally mentor this one? /* so if you wanted to learn about federation from the source itself, here's the way to do it! */

'''Possible mentors''':Jessica Tallon (tsyesika)

== Access control ==

Today MediaGoblin doesn't handle permissions very well. Anybody can see anything. What should be is proper access control. Owner of media or collection should be able to set who can view, edit and comment their mediafiles. Users could be added to groups, and groups could participate in permissions too. Different permissions might also apply to guests and registered users. Who can add users to groups? Who added user to group? Who edited or deleted a file? Who changed a permission?

A good proposal would include description of capabilities of the access control system and, maybe, audit system. Don't forget that REST should be covered too. Also, it should be clear how the whole thing works with federation.

'''Possible mentors''': Boris Bobrov (breton).
'''Difficulty''': 2-4 out of 5 depending on what will be implemented

GSOC 2017

2017-02-04T22:08:39Z

Breton: /* Possible projects with mentors. */

We are hoping to participate in [https://developers.google.com/open-source/gsoc/ GSOC 2017] under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net !

= How do I apply as a student ? =

* [https://developers.google.com/open-source/gsoc/ Submit your application]. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start ! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship ! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We would love to hear from you !

= Possible projects with mentors. =
== Chunked uploads / multi-upload UI. ==

We want to have multi-upload from cli (usecase: go to my directory with photos and upload them all) and from web-interface (using maybe zip, maybe html5 feature). There is already https://github.com/VenKamikaze/mediagoblin-html5-multi-upload which might be what we need, so before choosing this task please verify.

'''Possible Mentors:''' Christopher Allan Webber, Boris Bobrov (breton).

== Multiple qualities of video. ==

Now MediaGoblin supports only a single quality of video files. We want to fix that and support multiple qualities: 360p, 480p, 720p, 1080p, [https://en.wikipedia.org/wiki/Ultra-high-definition_television 4k]. This requires changes to frontend and to backend. To implement it, one must have some experience in JavaScript and Python.

'''Possible mentors''': Boris Bobrov (breton).

== Port federation code to ActivityPub ==

Right now we have a federation branch to be merged shortly by Jessica Tallon. This adds basic federation support to MediaGoblin. Right now the federation code is based on the [https://github.com/e14n/pump.io/blob/master/API.md Pump API]; however, we'd like to port it to the work we're doing on [https://www.w3.org/TR/activitypub/ ActivityPub] as part of the [https://www.w3.org/wiki/Socialwg W3C Social Working Group]. ActivityPub is fairly close to the Pump API, but there are some differences (most notably a move from ActivityStreams 1.0 to ActivityStreams 2.0, though there are others). Your mission, if you choose to accept it, is to port MediaGoblin's federation code to ActivityPub, ideally while supporting the Pump API in a backwards compatible fashion (if possible).

Is Jessica Tallon willing to personally mentor this one? /* so if you wanted to learn about federation from the source itself, here's the way to do it! */

'''Possible mentors''':

== Access control ==

Today MediaGoblin doesn't handle permissions very well. Anybody can see anything. What should be is proper access control. Owner of media or collection should be able to set who can view, edit and comment their mediafiles. Users could be added to groups, and groups could participate in permissions too. Different permisions might also apply to guests and registered users. Who can add users to groups? Who added user to group? Who edited or deleted a file? Who changed a permission?

A good proposal would include description of capabilities of the access control system and, maybe, audit system. Don't forget that REST should be covered too. Also, it should be clear how the whole thing works with federation.

'''Possible mentors''': Boris Bobrov (breton).

GSOC 2017

2017-01-23T19:06:19Z

Breton:

GSOC 2016

2017-01-23T19:05:11Z

Breton: Breton moved page GSOC 2016 to GSOC 2017: It's new GSoC!

#REDIRECT [[GSOC 2017]]

GSOC 2017

2017-01-23T19:05:11Z

Breton: Breton moved page GSOC 2016 to GSOC 2017: It's new GSoC!

We are hoping to participate in [https://developers.google.com/open-source/gsoc/ GSOC 2016] under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net !

= How do I apply as a student ? =

* [https://developers.google.com/open-source/gsoc/ Submit your application]. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start ! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship ! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We would love to hear from you !

= Possible projects with mentors. =
== Chunked uploads / multi-upload UI. ==

We want to have multi-upload from cli (usecase: go to my directory with photos and upload them all) and from web-interface (using maybe zip, maybe html5 feature). There is already https://github.com/VenKamikaze/mediagoblin-html5-multi-upload which might be what we need, so before choosing this task please verify.

'''Possible Mentors:''' Christopher Allan Webber, Boris Bobrov (breton).

== Subtitle interface. ==

'''Possible Mentors:''' Christopher Allan Webber, Boris Bobrov (breton).

We want subtitle support for our player. Subtitles should be supported as separate files and as embedded into video/audio files. Popular formats should be supported. Also, there can be multiple subtitles per video. It will require work on front-end and maybe some fixes to backend.

== Multiple qualities of video. ==

Now MediaGoblin supports only a single quality of video files. We want to fix that and support multiple qualities: 360p, 480p, 720p, 1080p, [https://en.wikipedia.org/wiki/Ultra-high-definition_television 4k]. This requires changes to frontend and to backend. To implement it, one must have some experience in JavaScript and Python.

'''Possible mentors''': Boris Bobrov (breton).

== Port federation code to ActivityPub ==

Right now we have a federation branch to be merged shortly by Jessica Tallon. This adds basic federation support to MediaGoblin. Right now the federation code is based on the [https://github.com/e14n/pump.io/blob/master/API.md Pump API]; however, we'd like to port it to the work we're doing on [https://www.w3.org/TR/activitypub/ ActivityPub] as part of the [https://www.w3.org/wiki/Socialwg W3C Social Working Group]. ActivityPub is fairly close to the Pump API, but there are some differences (most notably a move from ActivityStreams 1.0 to ActivityStreams 2.0, though there are others). Your mission, if you choose to accept it, is to port MediaGoblin's federation code to ActivityPub, ideally while supporting the Pump API in a backwards compatible fashion (if possible).

Jessica Tallon is willing to personally mentor this one, so if you wanted to learn about federation from the source itself, here's the way to do it!

'''Possible mentors''': Jessica Tallon

GSOC 2017

2016-03-01T08:09:14Z

Breton: added my nick

GSOC 2017

2016-02-02T22:44:16Z

Breton: Created page with "We are hoping to participate in [https://developers.google.com/open-source/gsoc/ GSOC 2016] under the GNU umbrella. You may want to check for a "possible mentor" before appl..."

We are hoping to participate in [https://developers.google.com/open-source/gsoc/ GSOC 2016] under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net!

= How do I apply as a student? =

* [https://developers.google.com/open-source/gsoc/ Submit your application]. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We'd love to hear from you!

= Possible projects with mentors =
== Chunked uploads / multi-upload UI ==

We want to have multi-upload from cli (usecase: go to my directory with photos and upload them all) and from web-interface (using maybe zip, maybe html5 feature). There is already https://github.com/VenKamikaze/mediagoblin-html5-multi-upload which might be what we need, so before choosing this task please verify.

'''Possible Mentors:''' Christopher Allan Webber, Boris Bobrov

== Subtitle interface ==

'''Possible Mentors:''' Christopher Allan Webber, Boris Bobrov

We want subtitle support for our player. Subtitles should be supported as separate files and as embedded into video/audio files. Popular formats should be supported. Also, there can be multiple subtitles per video. It will require work on front-end and maybe some fixes to backend.

== Multiple qualities of video ==

Now MediaGoblin supports only a single quality of video files. We want to fix that and support multiple qualities: 360p, 480p, 720p, 1080p, 4k. This requires changes to frontend and to backend. To implement it, one must have some experience in JavaScript and Python.

'''Possible mentors''': Boris Bobrov

Meeting

2015-09-05T16:13:06Z

Breton:

== MediaGoblin Monthly Meeting ==

'''When:''' 9:00 am Pacific Time first Saturday of the month. [http://www.timeanddate.com/worldclock/converter.html Convert time to your timezone]. Print current UTC time: <code>date -u +"It's %F %T UTC"</code>. [https://en.wikipedia.org/wiki/Daylight_saving_time_in_the_United_States#2005_revision_to_dates_of_observance Pacific Time in 2014 has DST between March 9th and November 2nd].

'''Where:''' IRC #mediagoblin on irc.freenode.net

Always announced several days in advance on the [http://lists.mediagoblin.org/pipermail/devel/ mailing list] as is date adjustments, agenda discussion and other meeting preparation.

'''Purpose:''' The idea is to discuss the past month, what happened, what was good, what should be done better. And to create roadmap for the upcoming month and assign tasks to people willing to handle them.

Typical Agenda topics:

* What happened in the last month, what was good, what could be better next time?
* What should be done next month?

Meetings are logged. [http://mediagoblin.org/irclogs/ Logs for past meetings.]

== Next Meeting ==

Every first Saturday of the month with JIT-compiled agenda.

== Past Meetings ==

=== March 7, 2015, 09:00AM Pacific Time, 17:00 UTC ===

* Moving off of gitorious, to?
* 0.8.0 upcoming release
* LibrePlanet and W3C meeting
* GSoC 2015
* Heads up: UserOps list
* Federation status
* Chris to take a temporary break?

=== December 6, 2014, 09:00AM Pacific Time, 17:00 UTC ===

* 0.8.0 upcoming release
** Current state (including looking at issue list)
** How to help!
*** Help us [https://issues.mediagoblin.org/query?status=new&status=in_progress&status=accepted&status=review&group=status&milestone=0.8.0 fix issues!]
*** Documentation (check it! improve it!)
*** Gstreamer 1.0 merge?
*** Help test!
*** Translate? (Maybe, but maybe not yet?)
* Translation talk, if breton is around
* Update on federation
* Update on W3C group

=== November 1, 2014, 09:00AM Pacific Time, 16:00 UTC ===

Loose meeting structure :)

=== October 4, 2014, 09:00AM Pacific Time, 16:00 UTC ===

* 0.8.0 progress
** Python 3 is in!
** Cleanup happening
** What's next
*** Gstreamer 1.0 merge
*** Search merge
*** [https://issues.mediagoblin.org/query?status=new&status=in_progress&status=accepted&status=review&group=status&milestone=0.8.0 Tickets!]
* Autoconf update
* Breton's translation update
* Federation update
* Deployment research update
* Add your things!

=== September 6, 2014, 09:00AM Pacific Time, 16:00 UTC ===

* Things happened! Horray!
** 0.7.0 is out!
** GSoC is done!
* Transifex move... help!
* 0.8.0 planning
* Updates on federation
* Updates on campaign-todo-items
* Premium hosting and deployment
* Documentation updates ( j1mc )
* Triage day! And the [https://etherpad.mozilla.org/gmg-triage new setup]!
* Help on reviewing?
* Epicanis' feature questions

=== August 2, 2014, 09:00AM Pacific Time, 16:00 UTC ===

* GSoC updates
* State of 0.7.0
* State of Federation
* State of post-campaign stuff (rewards, etc)
* General community updates
* Feedback: how are things going?

=== June 7, 2014, 09:00AM Pacific Time, 16:00 UTC ===

* GSoC updates
* Researchcation brief summary
* Campaign-related updates
* Upcoming release
* Code reviews

=== May 3, 2014, 09:00AM Pacific Time, 16:00 UTC ===

* GSoC starts!
* Post campaign updates!
** Horray, we did it!
** Unlocked features
** The rewards
** Premium hosting brainstorm
** Extra features brainstorm
* [http://lists.mediagoblin.org/pipermail/devel/2014-May/000867.html Metadata] discussion
* Chris' researchcation
** Federation stuff
** Deployment stuff
** Other stuff?
* Help review!
* Add your items HERE!

=== March 1st, 2014, 09:00AM Pacific Time, 17:00 UTC ===

* GSoC
* The [http://mediagoblin.org/pages/campaign.html crowdfunding campaign!]
** State of things / Q & A
** You can help!
* MediaGoblin in the meanwhile

=== November 2nd, 2013, 09:00AM Pacific Time, 16:00 UTC ===

Pretty loose on the schedule this month? :)

* Post-conference wrapup stuff?
* What's ahead in the next year for MediaGoblin?

=== October 5th, 2013, 09:00AM Pacific Time, 16:00 UTC ===

* Post-OPW/GSoC wrapup discussion
* Huge pile of code to review
* Planning out 0.6.0
* Road to 1.0
* mediagoblin.org changes
* GNU 30th recap, upcoming conferences
* MediaGoblin's graphic design?

=== September 7th, 2013 ===

* 0.5.0 release recap!
* Next release: 0.6.0!
* GSoC/OPW updates
* Review queue that needs attention
* GNU 30th! (and traveling)
* Other things???

=== August 3rd, 2013 9:00 am Pacific Time (2013-08-03 16:00 UTC) ===

* Configure && make merge
* 0.5.0 plans
* GSoC / OPW updates
* GNU 30th birthday party / GoblinCon? :)
* Documentation overhaul
* Reprocessing
* Bonus topics
** Mumble server
** XUDD

=== July 6th, 2013 9:00 am Pacific Time (2013-07-06 16:00 UTC) ===

* 0.4.0 is out! On to 0.4.1!
* OPW/GSoC update
* Federation status
* Module owners (and reducing the "Chris Webber bottleneck")?
** Projects like Blender, the Linux Kernel, and qemu have "module owners", people responsible for a domain of the code
** Could/should MediaGoblin do the same?
** How would that be broken up?

=== Jun 1st, 2013 9:00 am Pacific Time (2013-06-01 16:00 UTC) ===

* Welcoming our GSoC and OPW interns
* The coming 0.4.0
* What's after 0.4.0?
* RTL

=== May 4th, 2013, 9:00 am Pacific Time (2013-05-04 16:00 UTC) ===

[[meeting-2013-05-04|May 4th, 2013, 9:00 am Pacific Time (2013-05-04 16:00 UTC)]]

* GSOC and Gnome Outreach for Women participation update
* Mention the bug triage meeting
* Improving our unit tests?
* 0.4.0 release coming soon
* Help review code?
* Thanking simonft for sysadmin help!

=== April 6, 2013, 9:00 am Pacific Time (2013-03-30 16:00 UTC) ===

* Plugins update
* Post-libreplanet sprint merging
* Gstreamer 1.0 stuff (if Joar is here!)
* GSOC and Gnome Outreach for Women participation
* ???

=== March 2nd, 2013, 9:00 am Pacific Time (2013-03-02 16:00 UTC) ===

* Bug triage day discussion
* Upgrading Trac to 1.0
* Next release: 0.3.3
* Consider have release dates?
* Plugin stystem
* Consider moving to GStreamer 1.0
** Debian - 1.0 in sid
** Ubuntu - 1.0 in 12.10
*** Possibly backports for 12.04
** Fedora - 1.0 in 18
* Upcoming conferences and hackathons

=== February 9th, 2013, 9:00 am Pacific Time (2013-02-09 16:00 UTC) ===

* 0.3.3 milestone! What about our next release? :)
* Where to put Developer docs?
*: Our Documentation for developers is currently a bit split. Some are on the wiki, some in the main docs. There are pros and cons for both. We should consider where to put things. One place? Which? Or decide on an individual basis?
*: Good about main docs: Easy to integrate source code doc strings. That way internal api docs can be kept mostly up to date.
*: Good about wiki: Doesn't feel so "set in stone".
* wiki spam: Do we want to change something?
* [http://lists.mediagoblin.org/pipermail/devel/2012-November/000307.html Designing features!]
* Plugins, and architecting for plugins
* Chris Webber gives updates on how he's settling into his role ;)

=== October 13th, 2012, 9:00 am Pacific Time (2012-10-13 16:00 UTC) ===

* 0.3.2 release
** What existing features need to be wrapped up?
*** Werkzeug switch
** What time might we do the release?
* Fundraising campaign
** Keeping things going mid-campaign
** You have questions? I have answers, kinda :)
* Getting new contributors involved
* Plugins? New features?

==== Etherpad ====
<pre>
AGENDA

0.3.2 release

What existing features need to be wrapped up?

Werkzeug switch

What time might we do the release?

Congrats to Deb from the mediagoblin team! Congrats de Deb!

Fundraising campaign

Keeping things going mid-campaign

You have questions? I have answers, kinda :)

Getting new contributors involved

Plugins? New features?

-- http://wiki.mediagoblin.org/Meeting#Next_Meeting
Fundraising stuff
Things are going great mostly when people check it out!
But how to spread the word?

should contact more podcasts, etc

currently working with FSF on this

need community to spread the word!

List of places already spreaded list and contacted

VideoThumbnailerMarkII
New video thumbnailer, rewritten to try to eliminate a bug in the old one where processing would stall.
New bugs introduce (of course ;)
Collections
Merged - Thanks aaronw!
WebOb => Werkzeug switch
Made some things break. Need help with testing + bugfixes
borked stuffs:

Accessing paths without trailing slashes, e.g. /submit (instead of /submit/)

Still a lot of legacy WebOb responses (such as webob.exc.HTTPFound() HTTPForbidden() left)

API
Delivered to mrn.is, tryggvib will test it and get back with feedback.
Working, still a lot of room for improvements.
Mostly done, usable, still room for improvements. Example applications:

https://github.com/jwandborg/automgtic

https://github.com/jwandborg/omgmg
</pre>

=== September 1st, 2012, 9:00 am Pacific Time (2012-09-01 16:00 UTC) ===

[http://mediagoblin.org/irclogs/irc_meeting_2012-09-01.txt IRC log]

* FIXME - can someone type in summary here?

=== August 4th, 2012, 9:00 am Pacific Time (2012-08-04 16:00 UTC) ===

[http://mediagoblin.org/irclogs/irc_meeting_2012-08-04.txt IRC log]

* Release schedule
* Plugins and themes! Who's working on one? What problems are you having? -- Please write up issues for problems so they can get fixed!
* Should we namespace plugins? If so, how should we namespace plugins?
** Python 3.3 will have support for namespace plugins. [http://www.python.org/dev/peps/pep-0420/#namespace-packages-today]
** In Python >=2.3, <3.3 it's a hack [http://www.python.org/dev/peps/pep-0402/#the-problem][http://www.python.org/dev/peps/pep-0420/#namespace-packages-today]
** Flask has a workaround[https://github.com/mitsuhiko/flask/blob/master/flask/ext/__init__.py]

=== July 7th, 2012, 9:00 am Pacific Time (2012-07-07 16:00 UTC) ===

[http://mediagoblin.org/irclogs/irc_meeting_2012-07-07.txt IRC log]

Announcements:

* Anyone who wants to edit the wiki needs to be in the goblin army group. Ask Will or Chris to get added.
* Plugin infrastructure landed. If you're interested in writing plugins, talk to Will. Some documentation at http://docs.mediagoblin.org/#part-2-plugin-writer-s-guide

Agenda:

* Keyboard shortcuts ([http://issues.mediagoblin.org/ticket/346 #346])
* Ticket triaging?
* Base plugin stuff!
* Theming
* Conference: OSCON
* Chris Webber's new "office hours"

=== June 2nd, 2012, 9:00 am Pacific Time ===

[http://mediagoblin.org/irclogs/irc_meeting_2012-06-02.txt IRC log]

* docs changes
** Will split the docs/ guide into a Site Administrator's Guide and a Plugin Writer's Guide
** Has anyone looked at the Plugin Writer's Guide, yet?
** Will wants to add a "Contributor's Guide" to docs/ which he'd update from the wiki before every release
* Is there a way to improve our unit tests and motivation to write them?
** Simulating a browser by the way of [http://phantomjs.org/ PhantomJS], [http://seleniumhq.org/ Selenium] instead of having code simulating other code against itself might be more natural to write and even more testing the actual application. I have a good feeling about this, please prove me wrong if I'd be. --[[User:Joar|Joar]] 08:53, 28 May 2012 (EDT)
* Administrative panel/tools and user uploads panel
* Git and tickets
* Plugins
** What's the state of things?
** Documentation
** What plugins might we want to build for this upcoming release?
** What things do we currently have that we might want to pluginify?
* State of kuneco/federation mini-update (Chris)

=== May 5th, 2012, 9:00 am Pacific Time ===

[http://mediagoblin.org/irclogs/irc_meeting_2012-05-05.txt IRC log]

* Post-release reflections
** Woohoo, release!
** How did this release process go?
*** We should talk about that conference.
** What's left to clean up?
*** Mongokit->SQL "style" query conversion?
*** Other cruft code?
* What are our next goals?
** Plugins
** Federation
** Favoriting
*** Take that, Pinterest! ;)
** Galleries
** Theming
*** Using sass would be neat
** Access restrictions
*** User management, or having a "secret url" that is not in the photo index that you can share with friends and generate as needed for any media type
** What about traffic? Some of us will host GMG on limited plans.
** Some kind of coding guidelines? Do we have a philosophy like "Keep it Simple, Stupid"
*** This concerns things like: Should plugins land in core eventually, do we want to support ALL THE MEDIA TYPES, ...
** Podcasting support?
** Things that have been hanging???
** Bugtrackers and milestone?
** More??? We should organize things!
* jancborchardt and his team of UX wizard-students
* Website redesign
* OpenShift?

=== April 7th, 2012, 4:00 pm UTC ===

[http://mediagoblin.org/irclogs/irc_meeting_2012-04-07.txt IRC log]

* Post-SQL stuff
* Pending 0.0.3 release!
* Are there stray patches/branches to be merged?
* Our glorious upcoming plugin future! (Update from Will)

=== March 3rd, 2012, 9:00 am Pacific Time ===

[http://mediagoblin.org/irclogs/irc_meeting_2012-03-03.txt IRC log]

* [[GSOC 2012]]
* State of the SQL transition (preview: it's super close, but we need help!)
* Plugin discussion (Will can't make this, but we should talk about use cases)
* MediaGoblin at upcoming conferences
* PageKite accounts

=== 2012-02 (held on 2012-02-04) ===

[http://mediagoblin.org/irclogs/irc_meeting_2012-02-04.txt IRC log]

* Code style guide? See also: http://issues.mediagoblin.org/ticket/197
* Kuneco/federation
* API
* More testing discussion?
* Theming?
* Preliminary plugin discussion
* Status update from the "SQL Team".

Some of the decisions:

* file an issue about proper <audio> support.
* some TODOs recorded

=== 2011-12 (held on 2011-12-03) ===

[http://mediagoblin.org/irclogs/irc_meeting_2011-12-03.txt IRC log]

* We plan to create a plugin system. Do we want to create that soon or push it off until things settle a bit more? ([[User:Willkg|Willkg]] 08:54, 10 November 2011 (EST))
* [[Feature Ideas]]: What should we do about the wiki page? Keep it and have it as a monthly topic for "what next"? Convert everything to long waiting bugs?
* Possibility of an [[SQL Database Backend]]?
*: 'executive summary' (well, you should read the long docs): "We could move to sql. It's probably replacing one type of pain by another type of pain, but those are somewhat comparable. Leaving the main question: Do we want to occupy our main devs for some long time with this task and loose momentum?"
* Schendje's [http://wiki.mediagoblin.org/Feature_Ideas/Activities activities proposal]
* "Coming up next" blogpost draft by Deb Nicholson
* Jef's requests:
** Ticket #466 "Use of "Submit" in site copy is sterile and not as friendly and welcoming as it could be". I'd really like to change this soon to something more suitable. How can we improve the wording here? Some alternatives have been mentioned in the bug report, but which one should we pick? Link: http://bugs.foocorp.net/issues/466
** The concept and naming of "favourites". We'll (hopefully) be able to "favourite" media soon, which I *think* means that 1) it'll work like a "I like this" comment, a quick token of appreciation, 2) it'll be added to your list of favourites so you can save and promote it, and 3) we could maybe use the number of favourites as a ranking. What I'd like to know is: is that the intended purpose? If so, should we name them favo(u)rites or something else? "Like", "love", "save", "appreciate", "heart", "high five" and many more could all be contenders. And the name should be consistent with the action and purpose, of course. So I'd like to clear up how and why we will use favourites and what we should call them.

=== 2011-11 (held on 2011-11-05) ===

[http://mediagoblin.org/irclogs/irc_meeting_2011-11-05.txt IRC log]

* What happened in the last month, what was good, what could be better next time?
** Release:
*** Good: 0.1.0 released!
*** Bad: postponing vs not postponing
** Sites and deployment documentation:
*** Good: new mediagoblin.org
*** Good: deployment documentation
*** Bad: py-bcrypt’s site was down just after the release, so the virtualenv deployment didn’t work, and it wasn’t clear how to fix it.
** Live instances:
*** Joar has a live instance!
*** But what does it mean? Should ordinary users start using it?
**** Details at [[User:Joar/mg.wandborg.se]] -- [[User:Joar|Joar]] 17:01, 6 November 2011 (EST)
*** nyergler added a note about "heartbeat"/status to API notes
* What should be done next month?
* Starting real work on federation (via OStatus)... and do we split any of this work out into its own library?
* An API
* Creative Commons licensing tools
* Merging in the multimedia/video branch
*: (this is *very close* already actually thanks to the hard work of Joar Wandborg! But we need some help on the gstreamer front to fix a few issues... if you or someone you know is an expert in this area we could really use their help to make the videos that come out smoother!)
* Rollover items from 0.1.0
* Multiple file upload interface
* Drag and drop uploads interface (probably related!)

=== 2011-10 (held on 2011-10-01) ===

[http://mediagoblin.org/irclogs/irc_meeting_2011-10-01.txt IRC log]

This month's meeting was a quickly announced short meeting. The project is getting back on track and next month's meeting will be scheduled more properly. A bunch of people were around.

The most important decisions:
* The project will keep monthly releases. They're the heartbeat of the project.
* Release 0.1.0 this sunday/monday.
* New website will hopefully be deployed in the next few days.
* And the following things are planned to happen during this month: Most importantly federation. The developers have decided to make up their minds on what federation aactually should mean for MediaGoblin. Concerning code, probably "activity streams" are the first goal. If there is no (good) python library for this, a new stand alone library may be created. If so, a name for it has to be found. It should have something about communication in it. And the other thing to happen during this month is an ongoing discussion about "bus factor".

=== 2011-09 (held on 2011-09-03) ===

[http://mediagoblin.org/irclogs/irc_meeting_2011-09-03.txt IRC log]

=== 2011-08 (held on 2011-08-06) ===

[http://mediagoblin.org/irclogs/irc_meeting_2011-08-06.txt IRC log]

CommunityGovernance

2015-06-06T19:27:25Z

Breton: /* MediaGoblin translation platform */

MediaGoblin is a community project and we everybody try to help each other. Chris Webber is our lead developer, but the project should be able to self govern so he can take a break if he needs it or when he or other contributors take deserved holidays :)

In general, you can ask anything in the mailing list or the IRC channel and hopefully somebody will answer soon. Topics are discussed in the mailing list, IRC channel or monthly meetings (in the IRC channel).

In this page we list several tasks or areas along with the names or nicks of MediaGoblin community members that may help on them (for example, because they have permissions to perform certain operations). Please be kind when asking for help, most of us are volunteering for MediaGoblin in our free time, so answers or problems resolution can take a while.

Feel free to edit this page to list more tasks or sign the ones you self-assign.

More information about this in this mailing list thread: http://lists.mediagoblin.org/pipermail/devel/XXXXX

== MediaGoblin codebase ==

* People with permissions to commit/merge and handle the main codebase

* General stuff: breton
* Python 3 support: berkerpeksag (berker on IRC), breton
* Audio/Video: breton

== MediaGoblin website ==

* People with permissions to handle website outages

* Publish pages (for example new releases)

== MediaGoblin wiki ==

* Wiki administration

== MediaGoblin issue tracker ==

* Help triaging bugs:

* People with permissions to create accounts in trac

* People with permissions to delete users/issues/comments in trac

* Volunteers for spam cleaning: LArjona

== MediaGoblin translation platform ==

* If there is need of manual import/export files between Pootle and the MediaGoblin codebase, who can help on that
* LArjona (Spanish translator) can guide new translators, reproduce problems in the translation platform, test changes, and help with Pootle quality checks. Available in the mailing list and IRC channel.
* breton knows how and why translations are downloaded/compiled

== MediaGoblin IRC channel and mailing list ==

* Change IRC topic, channel operators

CommunityGovernance

2015-06-06T19:26:47Z

Breton: /* MediaGoblin codebase */

MediaGoblin is a community project and we everybody try to help each other. Chris Webber is our lead developer, but the project should be able to self govern so he can take a break if he needs it or when he or other contributors take deserved holidays :)

In general, you can ask anything in the mailing list or the IRC channel and hopefully somebody will answer soon. Topics are discussed in the mailing list, IRC channel or monthly meetings (in the IRC channel).

In this page we list several tasks or areas along with the names or nicks of MediaGoblin community members that may help on them (for example, because they have permissions to perform certain operations). Please be kind when asking for help, most of us are volunteering for MediaGoblin in our free time, so answers or problems resolution can take a while.

Feel free to edit this page to list more tasks or sign the ones you self-assign.

More information about this in this mailing list thread: http://lists.mediagoblin.org/pipermail/devel/XXXXX

== MediaGoblin codebase ==

* People with permissions to commit/merge and handle the main codebase

* General stuff: breton
* Python 3 support: berkerpeksag (berker on IRC), breton
* Audio/Video: breton

== MediaGoblin website ==

* People with permissions to handle website outages

* Publish pages (for example new releases)

== MediaGoblin wiki ==

* Wiki administration

== MediaGoblin issue tracker ==

* Help triaging bugs:

* People with permissions to create accounts in trac

* People with permissions to delete users/issues/comments in trac

* Volunteers for spam cleaning: LArjona

== MediaGoblin translation platform ==

* If there is need of manual import/export files between Pootle and the MediaGoblin codebase, who can help on that
* LArjona (Spanish translator) can guide new translators, reproduce problems in the translation platform, test changes, and help with Pootle quality checks. Available in the mailing list and IRC channel.

== MediaGoblin IRC channel and mailing list ==

* Change IRC topic, channel operators

GSOC 2015

2015-03-20T20:33:54Z

Breton: /* How do I apply as a student? */

We are hoping to participate in GSOC 2015 under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net!

= How do I apply as a student? =

* Submit your application. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We'd love to hear from you!

= Possible projects with mentors =
== Chunked uploads / multi-upload UI ==

We want to have multi-upload from cli (usecase: go to my directory with photos and upload them all) and from web-interface (using maybe zip, maybe html5 feature). There is already https://github.com/VenKamikaze/mediagoblin-html5-multi-upload which might be what we need, so before choosing this task please verify.

'''Possible Mentors:''' Christopher Allan Webber

== subtitle interface ==

'''Possible Mentors:''' Christopher Allan Webber, Boris Bobrov

We want subtitle support for our player. Subtitles should be supported as separate files and as embedded into video/audio files. Popular formats should be supported. Also, there can be multiple subtitles per video. It will require work on front-end and maybe some fixes to backend.

== android UI making use of pump api ==

Have you ever seen instagram app? We want something similar using pump api.

'''Possible mentors''': Boris Bobrov (clueless about android development though)

GSOC 2015

2015-03-16T16:15:55Z

Breton: /* subtitle interface */

We are hoping to participate in GSOC 2015 under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net!

= How do I apply as a student? =

Well, we need to be accepted as a mentoring org first :)

But then:
* Submit your application. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We'd love to hear from you!

= Possible projects with mentors =
== Chunked uploads / multi-upload UI ==

We want to have multi-upload from cli (usecase: go to my directory with photos and upload them all) and from web-interface (using maybe zip, maybe html5 feature). There is already https://github.com/VenKamikaze/mediagoblin-html5-multi-upload which might be what we need, so before choosing this task please verify.

'''Possible Mentors:''' Christopher Allan Webber

== subtitle interface ==

'''Possible Mentors:''' Christopher Allan Webber, Boris Bobrov

We want subtitle support for our player. Subtitles should be supported as separate files and as embedded into video/audio files. Popular formats should be supported. Also, there can be multiple subtitles per video. It will require work on front-end and maybe some fixes to backend.

== android UI making use of pump api ==

Have you ever seen instagram app? We want something similar using pump api.

'''Possible mentors''': Boris Bobrov (clueless about android development though)

GSOC 2015

2015-03-13T09:50:06Z

Breton: /* Possible projects with mentors */

We are hoping to participate in GSOC 2015 under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net!

= How do I apply as a student? =

Well, we need to be accepted as a mentoring org first :)

But then:
* Submit your application. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We'd love to hear from you!

= Possible projects with mentors =
== Chunked uploads / multi-upload UI ==

We want to have multi-upload from cli (usecase: go to my directory with photos and upload them all) and from web-interface (using maybe zip, maybe html5 feature). There is already https://github.com/VenKamikaze/mediagoblin-html5-multi-upload which might be what we need, so before choosing this task please verify.

'''Possible Mentors:''' Christopher Allan Webber

== Podcast support / feed overhaul ==

'''Possible Mentors:''' Christopher Allan Webber

== subtitle interface ==

'''Possible Mentors:''' Christopher Allan Webber, Boris Bobrov

We want subtitle support for our player. Subtitles should be supported as separate files and as embedded into video/audio files. Popular formats should be supported. Also, there can be multiple subtitles per video. It will require work on front-end and maybe some fixes to backend.

== android UI making use of pump api ==

Have you ever seen instagram app? We want something similar using pump api.

'''Possible mentors''': Boris Bobrov (clueless about android development though)

= Possible projects, no mentors claimed yet =

== owncloud backend for mg (hmmmm) ==

GSOC 2015

2015-03-08T21:13:24Z

Breton: some cleanup

We are hoping to participate in GSOC 2015 under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net!

= How do I apply as a student? =

Well, we need to be accepted as a mentoring org first :)

But then:
* Submit your application. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We'd love to hear from you!

= Possible projects with mentors =
== Chunked uploads / multi-upload UI ==

We want to have multi-upload from cli (usecase: go to my directory with photos and upload them all) and from web-interface (using maybe zip, maybe html5 feature). There is already https://github.com/VenKamikaze/mediagoblin-html5-multi-upload which might be what we need, so before choosing this task please verify.

'''Possible Mentors:''' Christopher Allan Webber

== Podcast support / feed overhaul ==

'''Possible Mentors:''' Christopher Allan Webber

== subtitle interface ==

'''Possible Mentors:''' Christopher Allan Webber, Boris Bobrov

== android UI making use of pump api ==

Have you ever seen instagram app? We want something similar using pump api.

'''Possible mentors''': Boris Bobrov (clueless about android development though)

We want subtitle support for our player. Subtitles should be supported as separate files and as embedded into video/audio files. Popular formats should be supported. Also, there can be multiple subtitles per video. It will require work on front-end and maybe some fixes to backend.

= Possible projects, no mentors claimed yet =

== owncloud backend for mg (hmmmm) ==

GSOC 2015

2015-03-08T21:11:31Z

Breton: /* android UI making use of pump api */

We are hoping to participate in GSOC 2015 under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net!

= How do I apply as a student? =

Well, we need to be accepted as a mentoring org first :)

But then:
* Submit your application. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We'd love to hear from you!

= Possible projects with mentors =
== Chunked uploads / multi-upload UI ==

We want to have multi-upload from cli (usecase: go to my directory with photos and upload them all) and from web-interface (using maybe zip, maybe html5 feature). There is already https://github.com/VenKamikaze/mediagoblin-html5-multi-upload which might be what we need, so before choosing this task please verify.

'''Possible Mentors:''' Christopher Allan Webber

== Podcast support / feed overhaul ==

'''Possible Mentors:''' Christopher Allan Webber

== subtitle interface ==

'''Possible Mentors:''' Christopher Allan Webber, Boris Bobrov

We want subtitle support for our player. Subtitles should be supported as separate files and as embedded into video/audio files. Popular formats should be supported. Also, there can be multiple subtitles per video. It will require work on front-end and maybe some fixes to backend.

= Possible projects, no mentors claimed yet ==

== android UI making use of pump api ==

Have you ever seen instagram app? We want something similar using pump api.

'''Possible mentors''': Boris Bobrov (clueless about android development though)

== owncloud backend for mg (hmmmm) ==

GSOC 2015

2015-03-08T21:09:38Z

Breton: /* Possible projects with mentors */ added some info

We are hoping to participate in GSOC 2015 under the GNU umbrella.

You may want to check for a "possible mentor" before applying for this topic. If you have your own idea, probably bring it up in #mediagoblin on irc.freenode.net!

= How do I apply as a student? =

Well, we need to be accepted as a mentoring org first :)

But then:
* Submit your application. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page].
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.
* Set up a development environment via our [[HackingHowto]]
* If you have never done web development in python before, MediaGoblin is a pretty good place to start! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.7/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.
* Also, be aware when submitting: Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship! If you propose, please be sure you are ready to make that level of commitment.

It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We'd love to hear from you!

= Possible projects with mentors =
== Chunked uploads / multi-upload UI ==

We want to have multi-upload from cli (usecase: go to my directory with photos and upload them all) and from web-interface (using maybe zip, maybe html5 feature). There is already https://github.com/VenKamikaze/mediagoblin-html5-multi-upload which might be what we need, so before choosing this task please verify.

'''Possible Mentors:''' Christopher Allan Webber

== Podcast support / feed overhaul ==

'''Possible Mentors:''' Christopher Allan Webber

== subtitle interface ==

'''Possible Mentors:''' Christopher Allan Webber, Boris Bobrov

We want subtitle support for our player. Subtitles should be supported as separate files and as embedded into video/audio files. Popular formats should be supported. Also, there can be multiple subtitles per video. It will require work on front-end and maybe some fixes to backend.

= Possible projects, no mentors claimed yet ==

== android UI making use of pump api ==
== owncloud backend for mg (hmmmm) ==

Translations

2014-12-07T22:52:27Z

Breton:

= For translators =

<pre style="color: red;">This section contains obsolete information. Now Mediagoblin lives on Pootle.
Before the section is updated, you can ask about the process using one of methods listed on
http://mediagoblin.org/pages/join.html</pre>

It is very easy to add or complete the translation of MediaGoblin! We use the web interface provided by Transifex.net to do the translations. You can see the current status of the translations per language [https://www.transifex.net/projects/p/mediagoblin/resource/mediagoblin/ here].

To start translating, just:

# click on your language in [https://www.transifex.net/projects/p/mediagoblin/resource/mediagoblin/ that list] then
# click on "Translate now".

By default, Transifex only shows you strings that currently have no translation. There is a checkbox to enable the display of already translated strings, in case you want to improve any of them.
There are no translation teams, so you can just edit the strings (but try to be nice and consistent with the current translation). If you are a regular contributor to the translations, you might want to add some notes to the [[#Notes for some language teams|per-language section on this page]] to coordinate consistent translations.

Note: please don't translate the words "MediaGoblin" or "GNU" into your native language. Those words should be retained as-is for branding reasons. :)

== Notes for some language teams ==
=== German ===

Interessante Hinweise zur Übersetzung finden sich bei Gnome:
* http://live.gnome.org/de/UebersetzungsRichtlinien

In Abweichung von diesen Richtlinien wird momentan "Du" für die Anrede benutzt. "Du" und "Deine" werden groß geschrieben. Benutze die folgenden Begriffe um konsistent zu bleiben:

Benutzerkonto/Nutzerkonto, E-Mail, E-Mail-Adresse, Aktivierungsmail, Konto "aktivieren" (verify email), Album (collection), Kurztitel (slug)

=== Spanish ===

Se está adoptando la siguiente convención para la traducción al español. Si crees que alguna traducción se puede mejorar, sugiérela en la página de discusión de este artículo.

{|
!Inglés
!Español
!Comentarios
|-
|you
|tú
|(segunda persona del singular, expresión informal)
|-
|media
|contenido
|-
|slug
|ficha
|}

=== '''العربية''' ===

.سنتخذ الاصطلاحات التالية للترجمة العربية. إذا كنت تريد تحسين الترجمة يرجى اقتراح التغبرات في صفحة النقاش في هذا المقال
{|
!الانجليزية
!العربية
|-
|MediaGoblin .
|ميدياغوبلن
|-
|media .
|الميديا
|-
|slug .
|المسار
|}

==Fixing translations from transifex==

= For developers =
== Marking strings for translation ==
=== In Jinja2 ===

See: http://jinja.pocoo.org/docs/templates/#i18n

Mostly like Django template i18n support, if you have any experience with that.

=== In python code ===

Usually, like this:

<pre>
from mediagoblin.util import pass_to_ugettext as _

def some_func(something):
return _(u'This string would tooootally be translatable now.')
</pre>

Except, in form modules we don't actually want to translate things because for various reasons that doesn't work, so we do the translation call template-side. So write the module to use the fake ugettext passthrough (which doesn't do any translation but still wraps things in _() so that Babel will know to extract these strings):

<pre>
from mediagoblin.util import fake_ugettext_passthrough as _

class FunkyMonkeyForm(wtforms.Form):
funky = wtforms.TextAreaField(
_('Funkiness description'))
</pre>

== Extracting translations ==

If you run buildout it should create a script called 'pybabel'. Use it to extract translations like so:

<pre>
./bin/pybabel extract -F babel.ini -o mediagoblin/i18n/en/LC_MESSAGES/mediagoblin.po .
</pre>

== Compiling translations ==

<pre>
./bin/pybabel compile -D mediagoblin -d mediagoblin/i18n/
</pre>

== Pulling translations from Transifex ==

Unfortunately until the next release of transifex-client which has been patched to have proper entry points, you have to run transifex-client from a virtualenv or from site-packages. But assuming that's done, pulling translations is easy:

<pre>
tx pull -a
</pre>

== Pushing new translations to Transifex ==

<pre>
tx push -s
</pre>

= Troubleshooting =

Got an error on Transifex like:

"Translation must start with a newline (\n)"?

The problem here is when we have something like:

{% trans register_url=request.urlgen('mediagoblin.auth.register') %}
<a class="header_submit_highlight" href="{{ register_url }}">Create a free account</a>
or
<a class="header_submit" href="http://wiki.mediagoblin.org/HackingHowto">Set up MediaGoblin on your own server</a>
{% endtrans %}

instead of:

{% trans register_url=request.urlgen('mediagoblin.auth.register') -%}
<a class="header_submit_highlight" href="{{ register_url }}">Create a free account</a>
or
<a class="header_submit" href="http://wiki.mediagoblin.org/HackingHowto">Set up MediaGoblin on your own server</a>
{%- endtrans %}

The distinction might not be obvious... look for the %} versus -%} and {% versus {%- on the first and last lines. The thing about this is this tells Jinja2 to strip the whitespace. Without that, Transifex sees that there's a "\n" on the first line of the original but not on the future translation, and gets upset.

If this happens the template should be updated to strip whitespace and translations should be pushed to transifex.

Deployment

2014-10-05T22:07:02Z

Breton: /* uwsgi with configs */

This page could use a lot of work. For now, a few smaller deployment tips!

See also: http://docs.mediagoblin.org/deploying.html (some of which may belong here)

= uwsgi =

Possibly a cleaner way to do nginx configuration than the official nginx + fastcgi suggestions!

http://qalthos.github.io/blog/FOSS@RIT/deploying-mediagoblin-1-fastcgi-vs-uwsgi.html

= uwsgi with configs =

Below there are 3 configs: for nginx, for uwsgi version 1.2.3-debian and for supervisord, used to run celery.

In future, when newer versions of uwsgi appear in debian, supervisor can be removed.

All instructions below are for Debian 7. They should also work with rpm-based systems, but with other paths.

<pre>apt-get install uwsgi uwsgi-plugin-python nginx-full supervisor;</pre>

/etc/nginx/sites-available/mediagoblin:
<pre>
server {
server_name <domain>;

access_log /var/log/nginx/mg.access.log;
error_log /var/log/nginx/mg.error.log error;

#include global/common.conf;

client_max_body_size 100m;
add_header X-Content-Type-Options nosniff;

root /path/to/mediagoblin//user_dev/;

location /mgoblin_static/ {
alias /path/to/mediagoblin/mediagoblin/static/;
}
location /mgoblin_media/ {
alias /path/to/mediagoblin/user_dev/media/public/;
}
location /theme_static/ {
}
location /plugin_static/ {
}
location / {
uwsgi_pass unix:///tmp/mg.uwsgi.sock;
uwsgi_param SCRIPT_NAME "/";
include uwsgi_params;
}
}
</pre>

/etc/uwsgi/apps-available/mg.yaml:
<pre>
uwsgi:
uid: mediagoblin
gid: mediagoblin
socket: /tmp/mg.uwsgi.sock
chown-socket: www-data:www-data
plugins: python
home: /path/to/mediagoblin/
chdir: /path/to/mediagoblin/
ini-paste: /path/to/mediagoblin/paste_local.ini
</pre>

/etc/supervisor/conf.d/mediagoblin.conf:
<pre>
[program:celery]
command=/path/to/mediagoblin/bin/celery worker -l debug

; Set PYTHONPATH to the directory containing celeryconfig.py
environment=PYTHONPATH="/path/to/mediagoblin",MEDIAGOBLIN_CONFIG="/path/to/mediagoblin/mediagoblin_local.ini",CELERY_CONFIG_MODULE="mediagoblin.init.celery.from_celery"

directory=/path/to/mediagoblin/
user=mediagoblin
numprocs=1
; uncomment below to enable logs saving
;stdout_logfile=/some/logs/path/celeryd_stdout.log
;stderr_logfile=/some/logs/path/celeryd_stderr.log
autostart=true
autorestart=false
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
</pre>

After setting all configs do:
<pre>
ln -s /etc/nginx/sites-available/mediagoblin /etc/nginx/sites-enabled/
ln -s /etc/uwsgi/apps-available/mg.yaml /etc/uwsgi/apps-enabled/
</pre>

And don't forget to restart all your daemons:
<pre>
service uwsgi restart;
service nginx restart;
service supervisor stop; sleep 1; service supervisor start # sleep 1 might be needed here because supervisor really stops not at once
</pre>

= FCGI script =

This works great with the apache FCGID config example in the next section :) in which case you should name it "mg.fcgi".

Before use, make sure you replace '/path/to/mediagoblin/bin/python' with a real path on your server, e.g. '/srv/www/myhomepage.com/mediagoblin/bin/python'. Also replace '/path/to/mediagoblin/paste.ini'.

If you encounter problems, try executing executing the script manually, e.g. <pre>./mg.fcgi</pre>

Script:

<pre>#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

from paste.deploy import loadapp
from flup.server.fcgi import WSGIServer

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
# import os
# os.environ['CELERY_ALWAYS_EAGER'] = 'true'

def launch_fcgi():
ccengine_wsgi_app = loadapp('config:' + CONFIG_PATH)
WSGIServer(ccengine_wsgi_app).run()

if __name__ == '__main__':
launch_fcgi()</pre>

= Apache 2 Config With fcgid =

Note that the libapache2-mod-fcgi in Debian is in the main section. libapache2-mod-fcgid can be used, but requires a slightly different configuration. For yum-based distributions, you may need to install mod_fcgid.

<VirtualHost *:80>
Options +ExecCGI

# Accept up to 16MB requests
FcgidMaxRequestLen 16777216

ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

ScriptAlias / /path/to/mediagoblin/mg.fcgi/
</VirtualHost>

= Apache 2 Config With mod_wsgi =

Apache can manage the MediaGoblin application itself with mod_wsgi. It requires slightly more configuration on the Apache side, but cuts out the FastCGI middleware. See their [http://code.google.com/p/modwsgi/wiki/ConfigurationGuidelines configuration guidelines] for advanced configuration.

<VirtualHost *:80>
ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

WSGIScriptAlias / /path/to/mediagoblin/wsgi.py
# Set user and group to whatever user you used to install mediagoblin (if you used a system account,
# it may have the group 'nogroup' assigned, in that case you can remove the group parameter)
# Remember to change python-path too!
# Replace the python version in the 'python-path' below with the correct one (eg 2.x -> 2.7)
WSGIPassAuthorization On
WSGIDaemonProcess gmg user=mediagoblin group=mediagoblin processes=2 threads=10 \
umask=0007 inactivity-timeout=900 maximum-requests=1000 \
python-path=/path/to/mediagoblin/:/path/to/mediagoblin/lib/python-2.x/site-packages/
WSGIProcessGroup gmg
WSGIApplicationGroup %{GLOBAL}

<Directory "/path/to/mediagoblin/mediagoblin/static/">
Order allow,deny
Allow from all
</Directory>
<Directory "/path/to/mediagoblin/user_dev/media/public">
Order allow,deny
Allow from all
</Directory>
</VirtualHost>

You'll also need to copy this into /path/to/mediagoblin/wsgi.py:

#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

import site
# Replace the python version with the correct one in the line below (eg 2.x -> 2.7)
site.addsitedir('/path/to/mediagoblin/lib/python-2.x/site-packages')

from paste.deploy import loadapp

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
#import os
#os.environ['CELERY_ALWAYS_EAGER'] = 'true'

application = loadapp('config:' + CONFIG_PATH)

= Apache Config Example =
This configuration example uses mod_fastcgi.

To install and enable mod_fastcgi on a Debian/Ubuntu based system:
<pre># apt-get install libapache2-mod-suexec libapache2-mod-fastcgi
# a2enmod suexec
# a2enmod fastcgi</pre>

Sample configuration:
<pre>
<VirtualHost *:80>
ServerName mediagoblin.yourdomain.tld
ServerAdmin webmaster@yourdoimain.tld
DocumentRoot /var/www/
# Custom log files
CustomLog /var/log/apache2/mediagobling_access.log combined
ErrorLog /var/log/apache2/mediagoblin_error.log

# Serve static and media files via alias
Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

# Rewrite all URLs to fcgi, except for static and media urls
RewriteEngine On
RewriteRule ^(mgoblin_static|mgoblin_media)($|/) - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ /mg.fcgi/$1 [QSA,L]

# Allow access to static and media directories
<Directory /path/to/mediagoblin/mediagoblin/static>
Order allow,deny
Allow from all
</Directory>
<Directory /path/to/mediagoblin/mediagoblin/user_dev/media/public>
Order allow,deny
Allow from all
</Directory>

# Connect to fcgi server
FastCGIExternalServer /var/www/mg.fcgi -host 127.0.0.1:26543
</VirtualHost>
</pre>
Then, you need to make sure mediagoblin is running in fcgi mode:
<pre>cd /path/to/mediagoblin
./lazyserver.sh --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543</pre>

Note: there may be several ways to improve this configuration

If it is too slow and you use the deflate module, you could try to use the following option : SetOutputFilter INFLATE

= Lighttpd =

This configuration example uses mod_fastcgi.
Make a symbolic link from <pre>/etc/lighttpd/conf-available/10-fastcgi.conf</pre> to <pre>/etc/lighttpd/conf-enabled/10-fastcgi.conf</pre>

<pre>
ln -s /etc/lighttpd/conf-available/10-fastcgi.conf /etc/lighttpd/conf-enabled/10-fastcgi.conf
</pre>

then edit your /etc/lighttpd/conf-enabled/10-fastcgi.conf file modifying it as below to serve mediagoblin on the /tube/ path of your webserver.

<pre>
## FastCGI programs have the same functionality as CGI programs,
## but are considerably faster through lower interpreter startup
## time and socketed communication
##
## Documentation: /usr/share/doc/lighttpd-doc/fastcgi.txt.gz
## http://www.lighttpd.net/documentation/fastcgi.html

server.modules += ( "mod_fastcgi" )

## Start an FastCGI server for mediagoblin (lazyserver.sh or other deployments scripts should be up and running)
fastcgi.server = (
# url at which the app should be accessible on the server
"/tube" =>
# mnemonical name of the backend
( "mg-local" => (
"host" => "127.0.0.1",
"port" => 6543,
"docroot" => "/path/to/mediagoblin-git-folder/mediagoblin",
"check-local" => "disable"
)
)

)
</pre>

Then open your /etc/lighttpd/lighttpd.conf and add the configuration to serve directly the static/public folders

<pre>
alias.url += (
"/mgoblin_static/" => "/path/to/mediagoblin-git-folder/mediagoblin/static/",
"/mgoblin_media/" => "/path/to/mediagoblin-git-folder/user_dev/media/public/",
"/theme_static/" => "/path/to/mediagoblin-git-folder/user_dev/theme_static/",
"/plugin_static/" => "/path/to/mediagoblin-git-folder/user_dev/plugin_static/"
)

</pre>

Done!

= OpenShift =

Thers's a blogpost saying how to [http://www.sud0.com/gnu-media-goblin-on-openshift.html install mediagoblin on openshift].

= Juju =

There is a juju [https://juju.ubuntu.com/Charms charm] available for deploying mediagoblin into EC2 or on your local box. [https://juju.ubuntu.com/ juju] is available in Ubuntu 11.10 and later, though it is recommended that you pull it from [https://launchpad.net/~juju/+archive/pkgs the juju PPA], which includes backported packages going back to Ubuntu 11.10. To use the juju charm, install juju and configure either the [https://juju.ubuntu.com/docs/provider-configuration-local.html local provider] or one of the cloud API providers, currently the [https://juju.ubuntu.com/docs/provider-configuration-ec2.html EC2 provider] is the best supported and works not only with Amazon Web Services but also OpenStack clouds. There is also a newer native OpenStack API provider which is known to support HP Cloud.
<pre>
# if you have not bootstrapped
juju bootstrap
mkdir ~/charms
bzr init-repo ~/charms/precise
bzr branch lp:~clint-fewbar/charms/precise/mediagoblin/trunk ~/charms/precise/mediagoblin
juju deploy --repository ~/charms local:mediagoblin
juju expose mediagoblin
</pre>

Currently the charm is volatile, deploying from trunk, and deploys a single-server version of MediaGoblin only. It will eventually relate to the existing juju charms for other supported data stores to allow one to scale out their MediaGoblin instance.

= Init scripts =

== Debian init scripts ==

Joar has some scripts for running celery and mediagoblin separately that are designed for Debian.

https://github.com/joar/mediagoblin-init-scripts

== Arch Linux init scripts ==

[http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts Jeremy Pope has written a nice blogpost] on how to add init scripts to deploy MediaGoblin with both the python paste http server and the celery deployments separated.

If you want a simpler setup and don't want to deploy celery separately, consider either turning CELERY_ALWAYS_EAGER to true in the paste init script described above, or check out [http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/ Chimo's guide].

== Generic, simple init script ==

This is a super stupidly simple init script that was used for mediagoblin.com... note that this has Celery running in always eager mode.

You will need to adjust the paths appropriately. This could probably be better written!

#! /bin/sh

## Stupidly simple mediagoblin init script.
#
# Written in 2012 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

PASTER=/srv/mediagoblin.com/bin/paster
PASTE_CONFIG=/srv/mediagoblin.com/paste.ini
OPTIONS="--pid-file=/tmp/mediagoblin.pid \
--log-file=/srv/mediagoblin.com/paster.log \
--server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543"

CELERY_ALWAYS_EAGER=true su -pc "$PASTER serve $PASTE_CONFIG $1 $OPTIONS" webadmin

== systemd init scripts ==

The following systemd scripts provide separate Paste and Celery instances. Replace <code>/path/to/mediagoblin</code> to match your install path. The directories <code>/var/log/mediagoblin</code> and <code>/var/run/mediagoblin</code> are not created by default; you may create them yourself or change the location of the pid and log files. You may omit the log file if you choose. The init scripts provided below assume customized versions of mediagoblin.ini and paste.ini have been created as mediagoblin_local.ini and paste_local.ini, respectively; you should change those lines to point back to the original mediagoblin.ini or paste.ini files if you haven't made customizations.

If you prefer not to run separate Celery, uncomment <code>Environment=CELERY_ALWAYS_EAGER=true</code> and comment <code>Environment=CELERY_ALWAYS_EAGER=false</code>. You won't need the file <code>/etc/systemd/system/gmg-celery.service</code>.

<code>/etc/systemd/system/gmg.service</code>:

[Unit]
Description=Mediagoblin

[Service]
Type=forking
User=mediagoblin
Group=mediagoblin
#Environment=CELERY_ALWAYS_EAGER=true
Environment=CELERY_ALWAYS_EAGER=false
WorkingDirectory=/path/to/mediagoblin
ExecStart=/path/to/mediagoblin/bin/paster serve \
/path/to/mediagoblin/paste_local.ini \
--pid-file=/var/run/mediagoblin/paster.pid \
--log-file=/var/log/mediagoblin/paster.log \
--daemon \
--server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543
ExecStop=/path/to/mediagoblin/bin/paster serve \
--pid-file=/var/run/mediagoblin/paster.pid \
/path/to/mediagoblin/paste_local.ini stop
PIDFile=/var/run/mediagoblin/mediagoblin.pid

[Install]
WantedBy=multi-user.target

<code>/etc/systemd/system/gmg-celeryd.service</code>:

[Unit]
Description=Mediagoblin Celeryd

[Service]
User=mediagoblin
Group=mediagoblin
Type=simple
WorkingDirectory=/path/to/mediagoblin
Environment="MEDIAGOBLIN_CONFIG=/path/to/mediagoblin/mediagoblin_local.ini" \
CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery
ExecStart=/path/to/mediagoblin/bin/celeryd
PIDFile=/var/run/mediagoblin/mediagoblin-celeryd.pid

[Install]
WantedBy=multi-user.target

Invoke the Paste or Celery processes using systemctl. Eg,

$ sudo systemctl start|stop|restart gmg.service

$ sudo systemctl start|stop|restart gmg-celeryd.service

= Running on Dreamhost.com =

===Set up your python virtualenv===

dreamhost.com servers come with python 2.4 and 2.5 installed which is too old for mediagoblin. This means you need to compile and install a newer python (don't worry this is really not difficult on dreamhost servers). In order to be able to install python packages for a local user without touching system files, we need to setup a python ''virtualenv''. Fortunately, this is not too tricky either. If your server has python 2.6 or 2.7 installed already, you can skip the [[#Install a local python|python installation stuff]] and you'll only need to [[#Setup virtualenv to install local python packages|install virtualenv]].

====Install a local python====
# Download the latest python: http://python.org/ftp/python/XXX/Python-XXX.tar.bz2
# Unpack with <pre>tar xvjf Python-XXX.tar.bz2</pre>
# ''cd'' into the directory and compile and install python locally:
<blockquote><pre>./configure --prefix=$HOME/local
make
make install</pre></blockquote>
:This will install python (I used 2.7.3) into /home/<USERNAME>/local/bin/python. You might get warnings about some modules not being able to compile. :It was tcl/tk and bzip2 mainly, but things still worked out overall.
:
:You should now be able to invoke <tt>~/local/bin/python</tt> and fall into a shell of your new python (exit with ctrl-d). Congrats, you have now a new python 2.7 that you can use. However, you will need to be able to install additional packages as a user and this is what virtualenv allows.

====Setup virtualenv to install local python packages====
# Download the latest virtualenv: http://pypi.python.org/packages/source/v/virtualenv/XXX
# Install the virtualenv: <blockquote><pre>~/local/bin/python ~/virtualenv-1.8/virtualenv.py $HOME/local/virtenv</pre></blockquote>
:: You will now have: ~/local/virtenv/bin/python
# In ~/.bash_profile add: <blockquote><pre>PATH=~/local/virtenv/bin:~/local/bin:${PATH}</pre></blockquote>
:: so that your local python will be preferred.
:: Log out, log in again and test python" to see which version will be invoked. It should be the new python. Check "which easy_install" to see if the one in local/virtenv would be executed.
::
:: You have now 1) a local ''python'' installation that you can use, and 2) ''easy_install'' will also work with your local user installation. From now on you can e.g. locally install the nose testing framework (''easy_install nose'') and use it (''python -c "import nose"'').

===Install mediagoblin as a site package===

# Check out mediagoblin from git to e.g. ~/mediagoblin
# Install MediaGoblin and all dependencies for MediaGoblin with easy_install.
:* In the mediagoblin directory issue:
:<blockquote><pre>python setup.py</pre></blockquote>
:* You will also need to: easy_install lxml
:* Python-image was trickier to install: <pre>easy_install --find-links http://www.pythonware.com/products/pil/ Imaging</pre>
::Test by leaving the mediagoblin directory and see if you can import it: <pre>python -c "import mediagoblin"</pre> looking for error messages.

===Set up an WSGI environment on Dreamhost===

# Enable the mod_passenger setting for ruby/python in the domains web panel
# ''cd'' into the domain directory (e.g. ~/media.sspaeth.de for me)
# Copy mediagoblin.ini and paste.ini from the ~/mediagoblin directory here
# Create passenger_wsgi.py in your domain directory (e.g. ~/media.sspaeth.de for me):
<blockquote><pre>import sys, os
INTERP = "/home/<username>/local/virtenv/bin/python"
#INTERP is present twice so that the new python interpreter knows the actual executable path
if sys.executable != INTERP: os.execl(INTERP, INTERP, *sys.argv)

from paste.deploy import loadapp
application = loadapp('config:/home/mediagoblin/media.sspaeth.de/paste.ini')

#If in case of errors, all you get are mysterious error 500, you can set debug=true in paste.ini to see stack traces
# Otherwise, add this:
#from paste.exceptions.errormiddleware import ErrorMiddleware
#application = ErrorMiddleware(application, debug=True)</pre></blockquote>
# Set up the database by issueing: <pre>gmg dbupdate</pre>
# (optional but recommended) Serve the static files directly from the webserver.
:In paste.ini in this section: [composite:routing] comment out the static files:
#/mgoblin_static/ = mediagoblin_static
#/theme_static/ = theme_static
:and symlink the mediagobin/mediagoblin/static directoy to public/mgoblin_static
: and mediagoblin/mediagoblin/themes to public/theme_static
:
:so it is displayed directly. This step might be different depending on your web server configuration. There is no reason that static
:files need to go through all the python indirection when they can be served directly by nginx/apache/...
:
:In case you want to delete your git checkout after the installation (you don't need it, since you installed the mediagoblin package to ~/local/virtenv/lib/python2.7/mediagoblin... you should do the symlinking from there.
===Open Issues===
Please fill in the blanks if you find them out:
* How to enable logging to a file or standard html logs? Console output just disappears.
* How to set up things not using sqlite. What's wrong with mySQL?

===Troubleshooting===
* First of all: invoke <pre>python passenger_wsgi.py</pre> directly from the shell. This will tell you if there are import errors etc. If this command exits without any output on the console, you have already achieved a lot.
* In ''paste.ini'' set debug=true. This will show you python backtraces directly in the web page
* You will need to configure an smtp user/passwd/server in mediagoblin.ini, so you actually get the account creation token mailed out. Hint, the relevant settings are: <blockquote><pre>email_debug_mode = false
email_sender_address = postmaster@sspaeth.de
email_smtp_host = SMTP.DOMAIN.TLD
email_smtp_user = USERNAME
email_smtp_pass = WEIRDPASSWORD</pre></blockquote>
* In case you can upload media, but it does not appear. You don't have the celery server running. Add <pre>CELERY_ALWAYS_EAGER = true</pre> to the ''[celery]'' section in mediagoblin.ini

=Miscellaneous Hacks=
==Force translation==
There might some conditions under which you would prefer that MediaGoblin always return the same translation. If you are deploying with nginx and fastcgi, you can force MediaGoblin to return a specific translation regardless of browser preferences by passing a specific HTTP_ACCEPT_LANGUAGE fastcgi parameter in your location block. Example last line in location block:
<nowiki>
fastcgi_param HTTP_ACCEPT_LANGUAGE es; #force spanish translation
</nowiki>

==Create an admin user==

To create an admin user, first create the user you need either via the website or the command
<pre>
./bin/gmg adduser
</pre>

Then to turn it into an admin account use

<pre>
./bin/gmg makeadmin your_username
</pre>

Deployment

2014-06-23T14:58:25Z

Breton: /* uwsgi with configs */

This page could use a lot of work. For now, a few smaller deployment tips!

See also: http://docs.mediagoblin.org/deploying.html (some of which may belong here)

= uwsgi =

Possibly a cleaner way to do nginx configuration than the official nginx + fastcgi suggestions!

http://qalthos.github.io/blog/FOSS@RIT/deploying-mediagoblin-1-fastcgi-vs-uwsgi.html

= uwsgi with configs =

Below there are 3 configs: for nginx, for uwsgi version 1.2.3-debian and for supervisord, used to run celery.

In future, when newer versions of uwsgi appear in debian, supervisor can be removed.

All instructions below are for Debian 7. They should also work with rpm-based systems, but with other paths.

<pre>apt-get install uwsgi uwsgi-plugin-python nginx-full supervisor;</pre>

/etc/nginx/sites-available/mediagoblin:
<pre>
server {
server_name <domain>;

access_log /var/log/nginx/mg.access.log;
error_log /var/log/nginx/mg.error.log error;

#include global/common.conf;

client_max_body_size 100m;
add_header X-Content-Type-Options nosniff;

root /path/to/mediagoblin//user_dev/;

location /mgoblin_static/ {
alias /path/to/mediagoblin/mediagoblin/static/;
}
location /mgoblin_media/ {
alias /path/to/mediagoblin/user_dev/media/public/;
}
location /theme_static/ {
}
location /plugin_static/ {
}
location / {
uwsgi_pass unix:///tmp/mg.uwsgi.sock;
uwsgi_param SCRIPT_NAME "/";
include uwsgi_params;
}
}
</pre>

/etc/uwsgi/apps-available/mg.yaml:
<pre>
uwsgi:
uid: mediagoblin
gid: mediagoblin
socket: /tmp/mg.uwsgi.sock
chown-socket: www-data:www-data
plugins: python
home: /path/to/mediagoblin/
chdir: /path/to/mediagoblin/
ini-paste: /path/to/mediagoblin/paste_local.ini
</pre>

/etc/supervisor/conf.d/mediagoblin.conf:
<pre>
[program:celery]
command=/path/to/mediagoblin/bin/celery worker -l debug

; Set PYTHONPATH to the directory containing celeryconfig.py
environment=PYTHONPATH="/path/to/mediagoblin",MEDIAGOBLIN_CONFIG="/path/to/mediagoblin/mediagoblin_local.ini",CELERY_CONFIG_MODULE="mediagoblin.init.celery.from_celery"

directory=/path/to/mediagoblin/
user=mediagoblin
numprocs=1
; uncomment below to enable logs saving
;stdout_logfile=/some/logs/path/celeryd_stdout.log
;stderr_logfile=/some/logs/path/celeryd_stderr.log
autostart=true
autorestart=false
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
</pre>

After setting all configs do:
<pre>
ln -s /etc/nginx/sites-available/mediagoblin /etc/nginx/sites-enabled/
ln -s /etc/uwsgi/apps-available/mg.yaml /etc/uwsgi/apps-enabled/
</pre>

= FCGI script =

This works great with the apache FCGID config example in the next section :) in which case you should name it "mg.fcgi".

Before use, make sure you replace '/path/to/mediagoblin/bin/python' with a real path on your server, e.g. '/srv/www/myhomepage.com/mediagoblin/bin/python'. Also replace '/path/to/mediagoblin/paste.ini'.

If you encounter problems, try executing executing the script manually, e.g. <pre>./mg.fcgi</pre>

Script:

<pre>#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

from paste.deploy import loadapp
from flup.server.fcgi import WSGIServer

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
# import os
# os.environ['CELERY_ALWAYS_EAGER'] = 'true'

def launch_fcgi():
ccengine_wsgi_app = loadapp('config:' + CONFIG_PATH)
WSGIServer(ccengine_wsgi_app).run()

if __name__ == '__main__':
launch_fcgi()</pre>

= Apache 2 Config With fcgid =

Note that the libapache2-mod-fcgi in Debian is in the main section. libapache2-mod-fcgid can be used, but requires a slightly different configuration. For yum-based distributions, you may need to install mod_fcgid.

<VirtualHost *:80>
Options +ExecCGI

# Accept up to 16MB requests
FcgidMaxRequestLen 16777216

ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

ScriptAlias / /path/to/mediagoblin/mg.fcgi/
</VirtualHost>

= Apache 2 Config With mod_wsgi =

Apache can manage the MediaGoblin application itself with mod_wsgi. It requires slightly more configuration on the Apache side, but cuts out the FastCGI middleware. See their [http://code.google.com/p/modwsgi/wiki/ConfigurationGuidelines configuration guidelines] for advanced configuration.

<VirtualHost *:80>
ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

WSGIScriptAlias / /path/to/mediagoblin/wsgi.py
# Set user and group to whatever Apache runs as
WSGIDaemonProcess gmg user=apache group=apache processes=2 threads=10 \
umask=0007 inactivity-timeout=900 maximum-requests=1000 \
python-path=/path/to/mediagoblin/:/path/to/mediagoblin/lib/python-2.x/site-packages/
WSGIProcessGroup gmg
WSGIApplicationGroup %{GLOBAL}

<Directory "/path/to/mediagoblin/mediagoblin/static/">
Order allow,deny
Allow from all
</Directory>
<Directory "/path/to/mediagoblin/user_dev/media/public">
Order allow,deny
Allow from all
</Directory>
</VirtualHost>

You'll also need to copy this into /path/to/mediagoblin/wsgi.py:

#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

from paste.deploy import loadapp

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
#import os
#os.environ['CELERY_ALWAYS_EAGER'] = 'true'

application = loadapp('config:' + CONFIG_PATH)

= Apache Config Example =
This configuration example uses mod_fastcgi.

To install and enable mod_fastcgi on a Debian/Ubuntu based system:
<pre># apt-get install libapache2-mod-suexec libapache2-mod-fastcgi
# a2enmod suexec
# a2enmod fastcgi</pre>

Sample configuration:
<pre>
<VirtualHost *:80>
ServerName mediagoblin.yourdomain.tld
ServerAdmin webmaster@yourdoimain.tld
DocumentRoot /var/www/
# Custom log files
CustomLog /var/log/apache2/mediagobling_access.log combined
ErrorLog /var/log/apache2/mediagoblin_error.log

# Serve static and media files via alias
Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

# Rewrite all URLs to fcgi, except for static and media urls
RewriteEngine On
RewriteRule ^(mgoblin_static|mgoblin_media)($|/) - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ /mg.fcgi/$1 [QSA,L]

# Allow access to static and media directories
<Directory /path/to/mediagoblin/mediagoblin/static>
Order allow,deny
Allow from all
</Directory>
<Directory /path/to/mediagoblin/mediagoblin/user_dev/media/public>
Order allow,deny
Allow from all
</Directory>

# Connect to fcgi server
FastCGIExternalServer /var/www/mg.fcgi -host 127.0.0.1:26543
</VirtualHost>
</pre>
Then, you need to make sure mediagoblin is running in fcgi mode:
<pre>cd /path/to/mediagoblin
./lazyserver.sh --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543</pre>

Note: there may be several ways to improve this configuration

If it is too slow and you use the deflate module, you could try to use the following option : SetOutputFilter INFLATE

= Lighttpd =

This configuration example uses mod_fastcgi.
Make a symbolic link from <pre>/etc/lighttpd/conf-available/10-fastcgi.conf</pre> to <pre>/etc/lighttpd/conf-enabled/10-fastcgi.conf</pre>

<pre>
ln -s /etc/lighttpd/conf-available/10-fastcgi.conf /etc/lighttpd/conf-enabled/10-fastcgi.conf
</pre>

then edit your /etc/lighttpd/conf-enabled/10-fastcgi.conf file modifying it as below to serve mediagoblin on the /tube/ path of your webserver.

<pre>
## FastCGI programs have the same functionality as CGI programs,
## but are considerably faster through lower interpreter startup
## time and socketed communication
##
## Documentation: /usr/share/doc/lighttpd-doc/fastcgi.txt.gz
## http://www.lighttpd.net/documentation/fastcgi.html

server.modules += ( "mod_fastcgi" )

## Start an FastCGI server for mediagoblin (lazyserver.sh or other deployments scripts should be up and running)
fastcgi.server = (
# url at which the app should be accessible on the server
"/tube" =>
# mnemonical name of the backend
( "mg-local" => (
"host" => "127.0.0.1",
"port" => 6543,
"docroot" => "/path/to/mediagoblin-git-folder/mediagoblin",
"check-local" => "disable"
)
)

)
</pre>

Then open your /etc/lighttpd/lighttpd.conf and add the configuration to serve directly the static/public folders

<pre>
alias.url += (
"/mgoblin_static/" => "/path/to/mediagoblin-git-folder/mediagoblin/static/",
"/mgoblin_media/" => "/path/to/mediagoblin-git-folder/user_dev/media/public/",
"/theme_static/" => "/path/to/mediagoblin-git-folder/user_dev/theme_static/",
"/plugin_static/" => "/path/to/mediagoblin-git-folder/user_dev/plugin_static/"
)

</pre>

Done!

= OpenShift =

Thers's a blogpost saying how to [http://www.sud0.com/gnu-media-goblin-on-openshift.html install mediagoblin on openshift].

= Juju =

There is a juju [https://juju.ubuntu.com/Charms charm] available for deploying mediagoblin into EC2 or on your local box. [https://juju.ubuntu.com/ juju] is available in Ubuntu 11.10 and later, though it is recommended that you pull it from [https://launchpad.net/~juju/+archive/pkgs the juju PPA], which includes backported packages going back to Ubuntu 11.10. To use the juju charm, install juju and configure either the [https://juju.ubuntu.com/docs/provider-configuration-local.html local provider] or one of the cloud API providers, currently the [https://juju.ubuntu.com/docs/provider-configuration-ec2.html EC2 provider] is the best supported and works not only with Amazon Web Services but also OpenStack clouds. There is also a newer native OpenStack API provider which is known to support HP Cloud.
<pre>
# if you have not bootstrapped
juju bootstrap
mkdir ~/charms
bzr init-repo ~/charms/precise
bzr branch lp:~clint-fewbar/charms/precise/mediagoblin/trunk ~/charms/precise/mediagoblin
juju deploy --repository ~/charms local:mediagoblin
juju expose mediagoblin
</pre>

Currently the charm is volatile, deploying from trunk, and deploys a single-server version of MediaGoblin only. It will eventually relate to the existing juju charms for other supported data stores to allow one to scale out their MediaGoblin instance.

= Init scripts =

== Debian init scripts ==

Joar has some scripts for running celery and mediagoblin separately that are designed for Debian.

https://github.com/joar/mediagoblin-init-scripts

== Arch Linux init scripts ==

[http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts Jeremy Pope has written a nice blogpost] on how to add init scripts to deploy MediaGoblin with both the python paste http server and the celery deployments separated.

If you want a simpler setup and don't want to deploy celery separately, consider either turning CELERY_ALWAYS_EAGER to true in the paste init script described above, or check out [http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/ Chimo's guide].

== Generic, simple init script ==

This is a super stupidly simple init script that was used for mediagoblin.com... note that this has Celery running in always eager mode.

You will need to adjust the paths appropriately. This could probably be better written!

#! /bin/sh

## Stupidly simple mediagoblin init script.
#
# Written in 2012 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

PASTER=/srv/mediagoblin.com/bin/paster
PASTE_CONFIG=/srv/mediagoblin.com/paste.ini
OPTIONS="--pid-file=/tmp/mediagoblin.pid \
--log-file=/srv/mediagoblin.com/paster.log \
--server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543"

CELERY_ALWAYS_EAGER=true su -pc "$PASTER serve $PASTE_CONFIG $1 $OPTIONS" webadmin

= Running on Dreamhost.com =

===Set up your python virtualenv===

dreamhost.com servers come with python 2.4 and 2.5 installed which is too old for mediagoblin. This means you need to compile and install a newer python (don't worry this is really not difficult on dreamhost servers). In order to be able to install python packages for a local user without touching system files, we need to setup a python ''virtualenv''. Fortunately, this is not too tricky either. If your server has python 2.6 or 2.7 installed already, you can skip the [[#Install a local python|python installation stuff]] and you'll only need to [[#Setup virtualenv to install local python packages|install virtualenv]].

====Install a local python====
# Download the latest python: http://python.org/ftp/python/XXX/Python-XXX.tar.bz2
# Unpack with <pre>tar xvjf Python-XXX.tar.bz2</pre>
# ''cd'' into the directory and compile and install python locally:
<blockquote><pre>./configure --prefix=$HOME/local
make
make install</pre></blockquote>
:This will install python (I used 2.7.3) into /home/<USERNAME>/local/bin/python. You might get warnings about some modules not being able to compile. :It was tcl/tk and bzip2 mainly, but things still worked out overall.
:
:You should now be able to invoke <tt>~/local/bin/python</tt> and fall into a shell of your new python (exit with ctrl-d). Congrats, you have now a new python 2.7 that you can use. However, you will need to be able to install additional packages as a user and this is what virtualenv allows.

====Setup virtualenv to install local python packages====
# Download the latest virtualenv: http://pypi.python.org/packages/source/v/virtualenv/XXX
# Install the virtualenv: <blockquote><pre>~/local/bin/python ~/virtualenv-1.8/virtualenv.py $HOME/local/virtenv</pre></blockquote>
:: You will now have: ~/local/virtenv/bin/python
# In ~/.bash_profile add: <blockquote><pre>PATH=~/local/virtenv/bin:~/local/bin:${PATH}</pre></blockquote>
:: so that your local python will be preferred.
:: Log out, log in again and test python" to see which version will be invoked. It should be the new python. Check "which easy_install" to see if the one in local/virtenv would be executed.
::
:: You have now 1) a local ''python'' installation that you can use, and 2) ''easy_install'' will also work with your local user installation. From now on you can e.g. locally install the nose testing framework (''easy_install nose'') and use it (''python -c "import nose"'').

===Install mediagoblin as a site package===

# Check out mediagoblin from git to e.g. ~/mediagoblin
# Install MediaGoblin and all dependencies for MediaGoblin with easy_install.
:* In the mediagoblin directory issue:
:<blockquote><pre>python setup.py</pre></blockquote>
:* You will also need to: easy_install lxml
:* Python-image was trickier to install: <pre>easy_install --find-links http://www.pythonware.com/products/pil/ Imaging</pre>
::Test by leaving the mediagoblin directory and see if you can import it: <pre>python -c "import mediagoblin"</pre> looking for error messages.

===Set up an WSGI environment on Dreamhost===

# Enable the mod_passenger setting for ruby/python in the domains web panel
# ''cd'' into the domain directory (e.g. ~/media.sspaeth.de for me)
# Copy mediagoblin.ini and paste.ini from the ~/mediagoblin directory here
# Create passenger_wsgi.py in your domain directory (e.g. ~/media.sspaeth.de for me):
<blockquote><pre>import sys, os
INTERP = "/home/<username>/local/virtenv/bin/python"
#INTERP is present twice so that the new python interpreter knows the actual executable path
if sys.executable != INTERP: os.execl(INTERP, INTERP, *sys.argv)

from paste.deploy import loadapp
application = loadapp('config:/home/mediagoblin/media.sspaeth.de/paste.ini')

#If in case of errors, all you get are mysterious error 500, you can set debug=true in paste.ini to see stack traces
# Otherwise, add this:
#from paste.exceptions.errormiddleware import ErrorMiddleware
#application = ErrorMiddleware(application, debug=True)</pre></blockquote>
# Set up the database by issueing: <pre>gmg dbupdate</pre>
# (optional but recommended) Serve the static files directly from the webserver.
:In paste.ini in this section: [composite:routing] comment out the static files:
#/mgoblin_static/ = mediagoblin_static
#/theme_static/ = theme_static
:and symlink the mediagobin/mediagoblin/static directoy to public/mgoblin_static
: and mediagoblin/mediagoblin/themes to public/theme_static
:
:so it is displayed directly. This step might be different depending on your web server configuration. There is no reason that static
:files need to go through all the python indirection when they can be served directly by nginx/apache/...
:
:In case you want to delete your git checkout after the installation (you don't need it, since you installed the mediagoblin package to ~/local/virtenv/lib/python2.7/mediagoblin... you should do the symlinking from there.
===Open Issues===
Please fill in the blanks if you find them out:
* How to enable logging to a file or standard html logs? Console output just disappears.
* How to set up things not using sqlite. What's wrong with mySQL?

===Troubleshooting===
* First of all: invoke <pre>python passenger_wsgi.py</pre> directly from the shell. This will tell you if there are import errors etc. If this command exits without any output on the console, you have already achieved a lot.
* In ''paste.ini'' set debug=true. This will show you python backtraces directly in the web page
* You will need to configure an smtp user/passwd/server in mediagoblin.ini, so you actually get the account creation token mailed out. Hint, the relevant settings are: <blockquote><pre>email_debug_mode = false
email_sender_address = postmaster@sspaeth.de
email_smtp_host = SMTP.DOMAIN.TLD
email_smtp_user = USERNAME
email_smtp_pass = WEIRDPASSWORD</pre></blockquote>
* In case you can upload media, but it does not appear. You don't have the celery server running. Add <pre>CELERY_ALWAYS_EAGER = true</pre> to the ''[celery]'' section in mediagoblin.ini

=Miscellaneous Hacks=
==Force translation==
There might some conditions under which you would prefer that MediaGoblin always return the same translation. If you are deploying with nginx and fastcgi, you can force MediaGoblin to return a specific translation regardless of browser preferences by passing a specific HTTP_ACCEPT_LANGUAGE fastcgi parameter in your location block. Example last line in location block:
<nowiki>
fastcgi_param HTTP_ACCEPT_LANGUAGE es; #force spanish translation
</nowiki>

==Create an admin user==

To create an admin user, first create the user you need either via the website or the command
<pre>
./bin/gmg adduser
</pre>

Then to turn it into an admin account use

<pre>
./bin/gmg makeadmin your_username
</pre>

Deployment

2014-06-21T07:11:46Z

Breton: /* uwsgi with configs */

This page could use a lot of work. For now, a few smaller deployment tips!

See also: http://docs.mediagoblin.org/deploying.html (some of which may belong here)

= uwsgi =

Possibly a cleaner way to do nginx configuration than the official nginx + fastcgi suggestions!

http://qalthos.github.io/blog/FOSS@RIT/deploying-mediagoblin-1-fastcgi-vs-uwsgi.html

= uwsgi with configs =

Below there are 3 configs: for nginx, for uwsgi version 1.2.3-debian and for supervisord, used to run celery.

In future, when newer versions of uwsgi appear in debian, supervisor can be removed.

All instructions below are for Debian 6. They should also work with rpm-based systems, but with other paths.

<pre>apt-get install uwsgi uwsgi-plugin-python nginx-full supervisor;</pre>

/etc/nginx/sites-available/mediagoblin:
<pre>
server {
server_name <domain>;

access_log /var/log/nginx/mg.access.log;
error_log /var/log/nginx/mg.error.log error;

#include global/common.conf;

client_max_body_size 100m;
add_header X-Content-Type-Options nosniff;

root /path/to/mediagoblin//user_dev/;

location /mgoblin_static/ {
alias /path/to/mediagoblin/mediagoblin/static/;
}
location /mgoblin_media/ {
alias /path/to/mediagoblin/user_dev/media/public/;
}
location /theme_static/ {
}
location /plugin_static/ {
}
location / {
uwsgi_pass unix:///tmp/mg.uwsgi.sock;
uwsgi_param SCRIPT_NAME "/";
include uwsgi_params;
}
}
</pre>

/etc/uwsgi/apps-available/mg.yaml:
<pre>
uwsgi:
uid: mediagoblin
gid: mediagoblin
socket: /tmp/mg.uwsgi.sock
chown-socket: www-data:www-data
plugins: python
home: /path/to/mediagoblin/
chdir: /path/to/mediagoblin/
ini-paste: /path/to/mediagoblin/paste_local.ini
</pre>

/etc/supervisor/conf.d/mediagoblin.conf:
<pre>
[program:celery]
command=/path/to/mediagoblin/bin/celery worker -l debug

; Set PYTHONPATH to the directory containing celeryconfig.py
environment=PYTHONPATH="/path/to/mediagoblin",MEDIAGOBLIN_CONFIG="/path/to/mediagoblin/mediagoblin_local.ini",CELERY_CONFIG_MODULE="mediagoblin.init.celery.from_celery"

directory=/path/to/mediagoblin/
user=mediagoblin
numprocs=1
; uncomment below to enable logs saving
;stdout_logfile=/some/logs/path/celeryd_stdout.log
;stderr_logfile=/some/logs/path/celeryd_stderr.log
autostart=true
autorestart=false
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
</pre>

After setting all configs do:
<pre>
ln -s /etc/nginx/sites-available/mediagoblin /etc/nginx/sites-enabled/
ln -s /etc/uwsgi/apps-available/mg.yaml /etc/uwsgi/apps-enabled/
</pre>

= FCGI script =

This works great with the apache FCGID config example in the next section :) in which case you should name it "mg.fcgi".

Before use, make sure you replace '/path/to/mediagoblin/bin/python' with a real path on your server, e.g. '/srv/www/myhomepage.com/mediagoblin/bin/python'. Also replace '/path/to/mediagoblin/paste.ini'.

If you encounter problems, try executing executing the script manually, e.g. <pre>./mg.fcgi</pre>

Script:

<pre>#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

from paste.deploy import loadapp
from flup.server.fcgi import WSGIServer

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
# import os
# os.environ['CELERY_ALWAYS_EAGER'] = 'true'

def launch_fcgi():
ccengine_wsgi_app = loadapp('config:' + CONFIG_PATH)
WSGIServer(ccengine_wsgi_app).run()

if __name__ == '__main__':
launch_fcgi()</pre>

= Apache 2 Config With fcgid =

Note that the libapache2-mod-fcgi in Debian is in the main section. libapache2-mod-fcgid can be used, but requires a slightly different configuration. For yum-based distributions, you may need to install mod_fcgid.

<VirtualHost *:80>
Options +ExecCGI

# Accept up to 16MB requests
FcgidMaxRequestLen 16777216

ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

ScriptAlias / /path/to/mediagoblin/mg.fcgi/
</VirtualHost>

= Apache 2 Config With mod_wsgi =

Apache can manage the MediaGoblin application itself with mod_wsgi. It requires slightly more configuration on the Apache side, but cuts out the FastCGI middleware. See their [http://code.google.com/p/modwsgi/wiki/ConfigurationGuidelines configuration guidelines] for advanced configuration.

<VirtualHost *:80>
ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

WSGIScriptAlias / /path/to/mediagoblin/wsgi.py
# Set user and group to whatever Apache runs as
WSGIDaemonProcess gmg user=apache group=apache processes=2 threads=10 \
umask=0007 inactivity-timeout=900 maximum-requests=1000 \
python-path=/path/to/mediagoblin/:/path/to/mediagoblin/lib/python-2.x/site-packages/
WSGIProcessGroup gmg
WSGIApplicationGroup %{GLOBAL}

<Directory "/path/to/mediagoblin/mediagoblin/static/">
Order allow,deny
Allow from all
</Directory>
<Directory "/path/to/mediagoblin/user_dev/media/public">
Order allow,deny
Allow from all
</Directory>
</VirtualHost>

You'll also need to copy this into /path/to/mediagoblin/wsgi.py:

#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

from paste.deploy import loadapp

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
#import os
#os.environ['CELERY_ALWAYS_EAGER'] = 'true'

application = loadapp('config:' + CONFIG_PATH)

= Apache Config Example =
This configuration example uses mod_fastcgi.

To install and enable mod_fastcgi on a Debian/Ubuntu based system:
<pre># apt-get install libapache2-mod-suexec libapache2-mod-fastcgi
# a2enmod suexec
# a2enmod fastcgi</pre>

Sample configuration:
<pre>
<VirtualHost *:80>
ServerName mediagoblin.yourdomain.tld
ServerAdmin webmaster@yourdoimain.tld
DocumentRoot /var/www/
# Custom log files
CustomLog /var/log/apache2/mediagobling_access.log combined
ErrorLog /var/log/apache2/mediagoblin_error.log

# Serve static and media files via alias
Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

# Rewrite all URLs to fcgi, except for static and media urls
RewriteEngine On
RewriteRule ^(mgoblin_static|mgoblin_media)($|/) - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ /mg.fcgi/$1 [QSA,L]

# Allow access to static and media directories
<Directory /path/to/mediagoblin/mediagoblin/static>
Order allow,deny
Allow from all
</Directory>
<Directory /path/to/mediagoblin/mediagoblin/user_dev/media/public>
Order allow,deny
Allow from all
</Directory>

# Connect to fcgi server
FastCGIExternalServer /var/www/mg.fcgi -host 127.0.0.1:26543
</VirtualHost>
</pre>
Then, you need to make sure mediagoblin is running in fcgi mode:
<pre>cd /path/to/mediagoblin
./lazyserver.sh --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543</pre>

Note: there may be several ways to improve this configuration

If it is too slow and you use the deflate module, you could try to use the following option : SetOutputFilter INFLATE

= Lighttpd =

This configuration example uses mod_fastcgi.
Make a symbolic link from <pre>/etc/lighttpd/conf-available/10-fastcgi.conf</pre> to <pre>/etc/lighttpd/conf-enabled/10-fastcgi.conf</pre>

<pre>
ln -s /etc/lighttpd/conf-available/10-fastcgi.conf /etc/lighttpd/conf-enabled/10-fastcgi.conf
</pre>

then edit your /etc/lighttpd/conf-enabled/10-fastcgi.conf file modifying it as below to serve mediagoblin on the /tube/ path of your webserver.

<pre>
## FastCGI programs have the same functionality as CGI programs,
## but are considerably faster through lower interpreter startup
## time and socketed communication
##
## Documentation: /usr/share/doc/lighttpd-doc/fastcgi.txt.gz
## http://www.lighttpd.net/documentation/fastcgi.html

server.modules += ( "mod_fastcgi" )

## Start an FastCGI server for mediagoblin (lazyserver.sh or other deployments scripts should be up and running)
fastcgi.server = (
# url at which the app should be accessible on the server
"/tube" =>
# mnemonical name of the backend
( "mg-local" => (
"host" => "127.0.0.1",
"port" => 6543,
"docroot" => "/path/to/mediagoblin-git-folder/mediagoblin",
"check-local" => "disable"
)
)

)
</pre>

Then open your /etc/lighttpd/lighttpd.conf and add the configuration to serve directly the static/public folders

<pre>
alias.url += (
"/mgoblin_static/" => "/path/to/mediagoblin-git-folder/mediagoblin/static/",
"/mgoblin_media/" => "/path/to/mediagoblin-git-folder/user_dev/media/public/",
"/theme_static/" => "/path/to/mediagoblin-git-folder/user_dev/theme_static/",
"/plugin_static/" => "/path/to/mediagoblin-git-folder/user_dev/plugin_static/"
)

</pre>

Done!

= OpenShift =

Thers's a blogpost saying how to [http://www.sud0.com/gnu-media-goblin-on-openshift.html install mediagoblin on openshift].

= Juju =

There is a juju [https://juju.ubuntu.com/Charms charm] available for deploying mediagoblin into EC2 or on your local box. [https://juju.ubuntu.com/ juju] is available in Ubuntu 11.10 and later, though it is recommended that you pull it from [https://launchpad.net/~juju/+archive/pkgs the juju PPA], which includes backported packages going back to Ubuntu 11.10. To use the juju charm, install juju and configure either the [https://juju.ubuntu.com/docs/provider-configuration-local.html local provider] or one of the cloud API providers, currently the [https://juju.ubuntu.com/docs/provider-configuration-ec2.html EC2 provider] is the best supported and works not only with Amazon Web Services but also OpenStack clouds. There is also a newer native OpenStack API provider which is known to support HP Cloud.
<pre>
# if you have not bootstrapped
juju bootstrap
mkdir ~/charms
bzr init-repo ~/charms/precise
bzr branch lp:~clint-fewbar/charms/precise/mediagoblin/trunk ~/charms/precise/mediagoblin
juju deploy --repository ~/charms local:mediagoblin
juju expose mediagoblin
</pre>

Currently the charm is volatile, deploying from trunk, and deploys a single-server version of MediaGoblin only. It will eventually relate to the existing juju charms for other supported data stores to allow one to scale out their MediaGoblin instance.

= Init scripts =

== Debian init scripts ==

Joar has some scripts for running celery and mediagoblin separately that are designed for Debian.

https://github.com/joar/mediagoblin-init-scripts

== Arch Linux init scripts ==

[http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts Jeremy Pope has written a nice blogpost] on how to add init scripts to deploy MediaGoblin with both the python paste http server and the celery deployments separated.

If you want a simpler setup and don't want to deploy celery separately, consider either turning CELERY_ALWAYS_EAGER to true in the paste init script described above, or check out [http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/ Chimo's guide].

== Generic, simple init script ==

This is a super stupidly simple init script that was used for mediagoblin.com... note that this has Celery running in always eager mode.

You will need to adjust the paths appropriately. This could probably be better written!

#! /bin/sh

## Stupidly simple mediagoblin init script.
#
# Written in 2012 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

PASTER=/srv/mediagoblin.com/bin/paster
PASTE_CONFIG=/srv/mediagoblin.com/paste.ini
OPTIONS="--pid-file=/tmp/mediagoblin.pid \
--log-file=/srv/mediagoblin.com/paster.log \
--server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543"

CELERY_ALWAYS_EAGER=true su -pc "$PASTER serve $PASTE_CONFIG $1 $OPTIONS" webadmin

= Running on Dreamhost.com =

===Set up your python virtualenv===

dreamhost.com servers come with python 2.4 and 2.5 installed which is too old for mediagoblin. This means you need to compile and install a newer python (don't worry this is really not difficult on dreamhost servers). In order to be able to install python packages for a local user without touching system files, we need to setup a python ''virtualenv''. Fortunately, this is not too tricky either. If your server has python 2.6 or 2.7 installed already, you can skip the [[#Install a local python|python installation stuff]] and you'll only need to [[#Setup virtualenv to install local python packages|install virtualenv]].

====Install a local python====
# Download the latest python: http://python.org/ftp/python/XXX/Python-XXX.tar.bz2
# Unpack with <pre>tar xvjf Python-XXX.tar.bz2</pre>
# ''cd'' into the directory and compile and install python locally:
<blockquote><pre>./configure --prefix=$HOME/local
make
make install</pre></blockquote>
:This will install python (I used 2.7.3) into /home/<USERNAME>/local/bin/python. You might get warnings about some modules not being able to compile. :It was tcl/tk and bzip2 mainly, but things still worked out overall.
:
:You should now be able to invoke <tt>~/local/bin/python</tt> and fall into a shell of your new python (exit with ctrl-d). Congrats, you have now a new python 2.7 that you can use. However, you will need to be able to install additional packages as a user and this is what virtualenv allows.

====Setup virtualenv to install local python packages====
# Download the latest virtualenv: http://pypi.python.org/packages/source/v/virtualenv/XXX
# Install the virtualenv: <blockquote><pre>~/local/bin/python ~/virtualenv-1.8/virtualenv.py $HOME/local/virtenv</pre></blockquote>
:: You will now have: ~/local/virtenv/bin/python
# In ~/.bash_profile add: <blockquote><pre>PATH=~/local/virtenv/bin:~/local/bin:${PATH}</pre></blockquote>
:: so that your local python will be preferred.
:: Log out, log in again and test python" to see which version will be invoked. It should be the new python. Check "which easy_install" to see if the one in local/virtenv would be executed.
::
:: You have now 1) a local ''python'' installation that you can use, and 2) ''easy_install'' will also work with your local user installation. From now on you can e.g. locally install the nose testing framework (''easy_install nose'') and use it (''python -c "import nose"'').

===Install mediagoblin as a site package===

# Check out mediagoblin from git to e.g. ~/mediagoblin
# Install MediaGoblin and all dependencies for MediaGoblin with easy_install.
:* In the mediagoblin directory issue:
:<blockquote><pre>python setup.py</pre></blockquote>
:* You will also need to: easy_install lxml
:* Python-image was trickier to install: <pre>easy_install --find-links http://www.pythonware.com/products/pil/ Imaging</pre>
::Test by leaving the mediagoblin directory and see if you can import it: <pre>python -c "import mediagoblin"</pre> looking for error messages.

===Set up an WSGI environment on Dreamhost===

# Enable the mod_passenger setting for ruby/python in the domains web panel
# ''cd'' into the domain directory (e.g. ~/media.sspaeth.de for me)
# Copy mediagoblin.ini and paste.ini from the ~/mediagoblin directory here
# Create passenger_wsgi.py in your domain directory (e.g. ~/media.sspaeth.de for me):
<blockquote><pre>import sys, os
INTERP = "/home/<username>/local/virtenv/bin/python"
#INTERP is present twice so that the new python interpreter knows the actual executable path
if sys.executable != INTERP: os.execl(INTERP, INTERP, *sys.argv)

from paste.deploy import loadapp
application = loadapp('config:/home/mediagoblin/media.sspaeth.de/paste.ini')

#If in case of errors, all you get are mysterious error 500, you can set debug=true in paste.ini to see stack traces
# Otherwise, add this:
#from paste.exceptions.errormiddleware import ErrorMiddleware
#application = ErrorMiddleware(application, debug=True)</pre></blockquote>
# Set up the database by issueing: <pre>gmg dbupdate</pre>
# (optional but recommended) Serve the static files directly from the webserver.
:In paste.ini in this section: [composite:routing] comment out the static files:
#/mgoblin_static/ = mediagoblin_static
#/theme_static/ = theme_static
:and symlink the mediagobin/mediagoblin/static directoy to public/mgoblin_static
: and mediagoblin/mediagoblin/themes to public/theme_static
:
:so it is displayed directly. This step might be different depending on your web server configuration. There is no reason that static
:files need to go through all the python indirection when they can be served directly by nginx/apache/...
:
:In case you want to delete your git checkout after the installation (you don't need it, since you installed the mediagoblin package to ~/local/virtenv/lib/python2.7/mediagoblin... you should do the symlinking from there.
===Open Issues===
Please fill in the blanks if you find them out:
* How to enable logging to a file or standard html logs? Console output just disappears.
* How to set up things not using sqlite. What's wrong with mySQL?

===Troubleshooting===
* First of all: invoke <pre>python passenger_wsgi.py</pre> directly from the shell. This will tell you if there are import errors etc. If this command exits without any output on the console, you have already achieved a lot.
* In ''paste.ini'' set debug=true. This will show you python backtraces directly in the web page
* You will need to configure an smtp user/passwd/server in mediagoblin.ini, so you actually get the account creation token mailed out. Hint, the relevant settings are: <blockquote><pre>email_debug_mode = false
email_sender_address = postmaster@sspaeth.de
email_smtp_host = SMTP.DOMAIN.TLD
email_smtp_user = USERNAME
email_smtp_pass = WEIRDPASSWORD</pre></blockquote>
* In case you can upload media, but it does not appear. You don't have the celery server running. Add <pre>CELERY_ALWAYS_EAGER = true</pre> to the ''[celery]'' section in mediagoblin.ini

=Miscellaneous Hacks=
==Force translation==
There might some conditions under which you would prefer that MediaGoblin always return the same translation. If you are deploying with nginx and fastcgi, you can force MediaGoblin to return a specific translation regardless of browser preferences by passing a specific HTTP_ACCEPT_LANGUAGE fastcgi parameter in your location block. Example last line in location block:
<nowiki>
fastcgi_param HTTP_ACCEPT_LANGUAGE es; #force spanish translation
</nowiki>

==Create an admin user==

To create an admin user, first create the user you need either via the website or the command
<pre>
./bin/gmg adduser
</pre>

Then to turn it into an admin account use

<pre>
./bin/gmg makeadmin your_username
</pre>

Deployment

2014-06-21T07:10:21Z

Breton: /* uwsgi with configs */

This page could use a lot of work. For now, a few smaller deployment tips!

See also: http://docs.mediagoblin.org/deploying.html (some of which may belong here)

= uwsgi =

Possibly a cleaner way to do nginx configuration than the official nginx + fastcgi suggestions!

http://qalthos.github.io/blog/FOSS@RIT/deploying-mediagoblin-1-fastcgi-vs-uwsgi.html

= uwsgi with configs =

Below there are 3 configs: for nginx, for uwsgi version 1.2.3-debian and for supervisord, used to run celery.

In future, when newer versions of uwsgi appear in debian, supervisor can be removed.

All instructions below are for Debian 6. They should also work with rpm-based systems, but with other paths.

apt-get install uwsgi uwsgi-plugin-python nginx-full supervisor;

/etc/nginx/sites-available/mediagoblin:
<pre>
server {
server_name <domain>;

access_log /var/log/nginx/mg.access.log;
error_log /var/log/nginx/mg.error.log error;

#include global/common.conf;

client_max_body_size 100m;
add_header X-Content-Type-Options nosniff;

root /path/to/mediagoblin//user_dev/;

location /mgoblin_static/ {
alias /path/to/mediagoblin/mediagoblin/static/;
}
location /mgoblin_media/ {
alias /path/to/mediagoblin/user_dev/media/public/;
}
location /theme_static/ {
}
location /plugin_static/ {
}
location / {
uwsgi_pass unix:///tmp/mg.uwsgi.sock;
uwsgi_param SCRIPT_NAME "/";
include uwsgi_params;
}
}
</pre>

/etc/uwsgi/apps-available/mg.yaml:
<pre>
uwsgi:
uid: mediagoblin
gid: mediagoblin
socket: /tmp/mg.uwsgi.sock
chown-socket: www-data:www-data
plugins: python
home: /path/to/mediagoblin/
chdir: /path/to/mediagoblin/
ini-paste: /path/to/mediagoblin/paste_local.ini
</pre>

/etc/supervisor/conf.d/mediagoblin.conf:
<pre>
[program:celery]
command=/path/to/mediagoblin/bin/celery worker -l debug

; Set PYTHONPATH to the directory containing celeryconfig.py
environment=PYTHONPATH="/path/to/mediagoblin",MEDIAGOBLIN_CONFIG="/path/to/mediagoblin/mediagoblin_local.ini",CELERY_CONFIG_MODULE="mediagoblin.init.celery.from_celery"

directory=/path/to/mediagoblin/
user=mediagoblin
numprocs=1
; uncomment below to enable logs saving
;stdout_logfile=/some/logs/path/celeryd_stdout.log
;stderr_logfile=/some/logs/path/celeryd_stderr.log
autostart=true
autorestart=false
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
</pre>

After setting all configs do:
<pre>
ln -s /etc/nginx/sites-available/mediagoblin /etc/nginx/sites-enabled/
ln -s /etc/uwsgi/apps-available/mg.yaml /etc/uwsgi/apps-enabled/mg.yaml
</pre>

= FCGI script =

This works great with the apache FCGID config example in the next section :) in which case you should name it "mg.fcgi".

Before use, make sure you replace '/path/to/mediagoblin/bin/python' with a real path on your server, e.g. '/srv/www/myhomepage.com/mediagoblin/bin/python'. Also replace '/path/to/mediagoblin/paste.ini'.

If you encounter problems, try executing executing the script manually, e.g. <pre>./mg.fcgi</pre>

Script:

<pre>#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

from paste.deploy import loadapp
from flup.server.fcgi import WSGIServer

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
# import os
# os.environ['CELERY_ALWAYS_EAGER'] = 'true'

def launch_fcgi():
ccengine_wsgi_app = loadapp('config:' + CONFIG_PATH)
WSGIServer(ccengine_wsgi_app).run()

if __name__ == '__main__':
launch_fcgi()</pre>

= Apache 2 Config With fcgid =

Note that the libapache2-mod-fcgi in Debian is in the main section. libapache2-mod-fcgid can be used, but requires a slightly different configuration. For yum-based distributions, you may need to install mod_fcgid.

<VirtualHost *:80>
Options +ExecCGI

# Accept up to 16MB requests
FcgidMaxRequestLen 16777216

ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

ScriptAlias / /path/to/mediagoblin/mg.fcgi/
</VirtualHost>

= Apache 2 Config With mod_wsgi =

Apache can manage the MediaGoblin application itself with mod_wsgi. It requires slightly more configuration on the Apache side, but cuts out the FastCGI middleware. See their [http://code.google.com/p/modwsgi/wiki/ConfigurationGuidelines configuration guidelines] for advanced configuration.

<VirtualHost *:80>
ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

WSGIScriptAlias / /path/to/mediagoblin/wsgi.py
# Set user and group to whatever Apache runs as
WSGIDaemonProcess gmg user=apache group=apache processes=2 threads=10 \
umask=0007 inactivity-timeout=900 maximum-requests=1000 \
python-path=/path/to/mediagoblin/:/path/to/mediagoblin/lib/python-2.x/site-packages/
WSGIProcessGroup gmg
WSGIApplicationGroup %{GLOBAL}

<Directory "/path/to/mediagoblin/mediagoblin/static/">
Order allow,deny
Allow from all
</Directory>
<Directory "/path/to/mediagoblin/user_dev/media/public">
Order allow,deny
Allow from all
</Directory>
</VirtualHost>

You'll also need to copy this into /path/to/mediagoblin/wsgi.py:

#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

from paste.deploy import loadapp

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
#import os
#os.environ['CELERY_ALWAYS_EAGER'] = 'true'

application = loadapp('config:' + CONFIG_PATH)

= Apache Config Example =
This configuration example uses mod_fastcgi.

To install and enable mod_fastcgi on a Debian/Ubuntu based system:
<pre># apt-get install libapache2-mod-suexec libapache2-mod-fastcgi
# a2enmod suexec
# a2enmod fastcgi</pre>

Sample configuration:
<pre>
<VirtualHost *:80>
ServerName mediagoblin.yourdomain.tld
ServerAdmin webmaster@yourdoimain.tld
DocumentRoot /var/www/
# Custom log files
CustomLog /var/log/apache2/mediagobling_access.log combined
ErrorLog /var/log/apache2/mediagoblin_error.log

# Serve static and media files via alias
Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

# Rewrite all URLs to fcgi, except for static and media urls
RewriteEngine On
RewriteRule ^(mgoblin_static|mgoblin_media)($|/) - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ /mg.fcgi/$1 [QSA,L]

# Allow access to static and media directories
<Directory /path/to/mediagoblin/mediagoblin/static>
Order allow,deny
Allow from all
</Directory>
<Directory /path/to/mediagoblin/mediagoblin/user_dev/media/public>
Order allow,deny
Allow from all
</Directory>

# Connect to fcgi server
FastCGIExternalServer /var/www/mg.fcgi -host 127.0.0.1:26543
</VirtualHost>
</pre>
Then, you need to make sure mediagoblin is running in fcgi mode:
<pre>cd /path/to/mediagoblin
./lazyserver.sh --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543</pre>

Note: there may be several ways to improve this configuration

If it is too slow and you use the deflate module, you could try to use the following option : SetOutputFilter INFLATE

= Lighttpd =

This configuration example uses mod_fastcgi.
Make a symbolic link from <pre>/etc/lighttpd/conf-available/10-fastcgi.conf</pre> to <pre>/etc/lighttpd/conf-enabled/10-fastcgi.conf</pre>

<pre>
ln -s /etc/lighttpd/conf-available/10-fastcgi.conf /etc/lighttpd/conf-enabled/10-fastcgi.conf
</pre>

then edit your /etc/lighttpd/conf-enabled/10-fastcgi.conf file modifying it as below to serve mediagoblin on the /tube/ path of your webserver.

<pre>
## FastCGI programs have the same functionality as CGI programs,
## but are considerably faster through lower interpreter startup
## time and socketed communication
##
## Documentation: /usr/share/doc/lighttpd-doc/fastcgi.txt.gz
## http://www.lighttpd.net/documentation/fastcgi.html

server.modules += ( "mod_fastcgi" )

## Start an FastCGI server for mediagoblin (lazyserver.sh or other deployments scripts should be up and running)
fastcgi.server = (
# url at which the app should be accessible on the server
"/tube" =>
# mnemonical name of the backend
( "mg-local" => (
"host" => "127.0.0.1",
"port" => 6543,
"docroot" => "/path/to/mediagoblin-git-folder/mediagoblin",
"check-local" => "disable"
)
)

)
</pre>

Then open your /etc/lighttpd/lighttpd.conf and add the configuration to serve directly the static/public folders

<pre>
alias.url += (
"/mgoblin_static/" => "/path/to/mediagoblin-git-folder/mediagoblin/static/",
"/mgoblin_media/" => "/path/to/mediagoblin-git-folder/user_dev/media/public/",
"/theme_static/" => "/path/to/mediagoblin-git-folder/user_dev/theme_static/",
"/plugin_static/" => "/path/to/mediagoblin-git-folder/user_dev/plugin_static/"
)

</pre>

Done!

= OpenShift =

Thers's a blogpost saying how to [http://www.sud0.com/gnu-media-goblin-on-openshift.html install mediagoblin on openshift].

= Juju =

There is a juju [https://juju.ubuntu.com/Charms charm] available for deploying mediagoblin into EC2 or on your local box. [https://juju.ubuntu.com/ juju] is available in Ubuntu 11.10 and later, though it is recommended that you pull it from [https://launchpad.net/~juju/+archive/pkgs the juju PPA], which includes backported packages going back to Ubuntu 11.10. To use the juju charm, install juju and configure either the [https://juju.ubuntu.com/docs/provider-configuration-local.html local provider] or one of the cloud API providers, currently the [https://juju.ubuntu.com/docs/provider-configuration-ec2.html EC2 provider] is the best supported and works not only with Amazon Web Services but also OpenStack clouds. There is also a newer native OpenStack API provider which is known to support HP Cloud.
<pre>
# if you have not bootstrapped
juju bootstrap
mkdir ~/charms
bzr init-repo ~/charms/precise
bzr branch lp:~clint-fewbar/charms/precise/mediagoblin/trunk ~/charms/precise/mediagoblin
juju deploy --repository ~/charms local:mediagoblin
juju expose mediagoblin
</pre>

Currently the charm is volatile, deploying from trunk, and deploys a single-server version of MediaGoblin only. It will eventually relate to the existing juju charms for other supported data stores to allow one to scale out their MediaGoblin instance.

= Init scripts =

== Debian init scripts ==

Joar has some scripts for running celery and mediagoblin separately that are designed for Debian.

https://github.com/joar/mediagoblin-init-scripts

== Arch Linux init scripts ==

[http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts Jeremy Pope has written a nice blogpost] on how to add init scripts to deploy MediaGoblin with both the python paste http server and the celery deployments separated.

If you want a simpler setup and don't want to deploy celery separately, consider either turning CELERY_ALWAYS_EAGER to true in the paste init script described above, or check out [http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/ Chimo's guide].

== Generic, simple init script ==

This is a super stupidly simple init script that was used for mediagoblin.com... note that this has Celery running in always eager mode.

You will need to adjust the paths appropriately. This could probably be better written!

#! /bin/sh

## Stupidly simple mediagoblin init script.
#
# Written in 2012 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

PASTER=/srv/mediagoblin.com/bin/paster
PASTE_CONFIG=/srv/mediagoblin.com/paste.ini
OPTIONS="--pid-file=/tmp/mediagoblin.pid \
--log-file=/srv/mediagoblin.com/paster.log \
--server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543"

CELERY_ALWAYS_EAGER=true su -pc "$PASTER serve $PASTE_CONFIG $1 $OPTIONS" webadmin

= Running on Dreamhost.com =

===Set up your python virtualenv===

dreamhost.com servers come with python 2.4 and 2.5 installed which is too old for mediagoblin. This means you need to compile and install a newer python (don't worry this is really not difficult on dreamhost servers). In order to be able to install python packages for a local user without touching system files, we need to setup a python ''virtualenv''. Fortunately, this is not too tricky either. If your server has python 2.6 or 2.7 installed already, you can skip the [[#Install a local python|python installation stuff]] and you'll only need to [[#Setup virtualenv to install local python packages|install virtualenv]].

====Install a local python====
# Download the latest python: http://python.org/ftp/python/XXX/Python-XXX.tar.bz2
# Unpack with <pre>tar xvjf Python-XXX.tar.bz2</pre>
# ''cd'' into the directory and compile and install python locally:
<blockquote><pre>./configure --prefix=$HOME/local
make
make install</pre></blockquote>
:This will install python (I used 2.7.3) into /home/<USERNAME>/local/bin/python. You might get warnings about some modules not being able to compile. :It was tcl/tk and bzip2 mainly, but things still worked out overall.
:
:You should now be able to invoke <tt>~/local/bin/python</tt> and fall into a shell of your new python (exit with ctrl-d). Congrats, you have now a new python 2.7 that you can use. However, you will need to be able to install additional packages as a user and this is what virtualenv allows.

====Setup virtualenv to install local python packages====
# Download the latest virtualenv: http://pypi.python.org/packages/source/v/virtualenv/XXX
# Install the virtualenv: <blockquote><pre>~/local/bin/python ~/virtualenv-1.8/virtualenv.py $HOME/local/virtenv</pre></blockquote>
:: You will now have: ~/local/virtenv/bin/python
# In ~/.bash_profile add: <blockquote><pre>PATH=~/local/virtenv/bin:~/local/bin:${PATH}</pre></blockquote>
:: so that your local python will be preferred.
:: Log out, log in again and test python" to see which version will be invoked. It should be the new python. Check "which easy_install" to see if the one in local/virtenv would be executed.
::
:: You have now 1) a local ''python'' installation that you can use, and 2) ''easy_install'' will also work with your local user installation. From now on you can e.g. locally install the nose testing framework (''easy_install nose'') and use it (''python -c "import nose"'').

===Install mediagoblin as a site package===

# Check out mediagoblin from git to e.g. ~/mediagoblin
# Install MediaGoblin and all dependencies for MediaGoblin with easy_install.
:* In the mediagoblin directory issue:
:<blockquote><pre>python setup.py</pre></blockquote>
:* You will also need to: easy_install lxml
:* Python-image was trickier to install: <pre>easy_install --find-links http://www.pythonware.com/products/pil/ Imaging</pre>
::Test by leaving the mediagoblin directory and see if you can import it: <pre>python -c "import mediagoblin"</pre> looking for error messages.

===Set up an WSGI environment on Dreamhost===

# Enable the mod_passenger setting for ruby/python in the domains web panel
# ''cd'' into the domain directory (e.g. ~/media.sspaeth.de for me)
# Copy mediagoblin.ini and paste.ini from the ~/mediagoblin directory here
# Create passenger_wsgi.py in your domain directory (e.g. ~/media.sspaeth.de for me):
<blockquote><pre>import sys, os
INTERP = "/home/<username>/local/virtenv/bin/python"
#INTERP is present twice so that the new python interpreter knows the actual executable path
if sys.executable != INTERP: os.execl(INTERP, INTERP, *sys.argv)

from paste.deploy import loadapp
application = loadapp('config:/home/mediagoblin/media.sspaeth.de/paste.ini')

#If in case of errors, all you get are mysterious error 500, you can set debug=true in paste.ini to see stack traces
# Otherwise, add this:
#from paste.exceptions.errormiddleware import ErrorMiddleware
#application = ErrorMiddleware(application, debug=True)</pre></blockquote>
# Set up the database by issueing: <pre>gmg dbupdate</pre>
# (optional but recommended) Serve the static files directly from the webserver.
:In paste.ini in this section: [composite:routing] comment out the static files:
#/mgoblin_static/ = mediagoblin_static
#/theme_static/ = theme_static
:and symlink the mediagobin/mediagoblin/static directoy to public/mgoblin_static
: and mediagoblin/mediagoblin/themes to public/theme_static
:
:so it is displayed directly. This step might be different depending on your web server configuration. There is no reason that static
:files need to go through all the python indirection when they can be served directly by nginx/apache/...
:
:In case you want to delete your git checkout after the installation (you don't need it, since you installed the mediagoblin package to ~/local/virtenv/lib/python2.7/mediagoblin... you should do the symlinking from there.
===Open Issues===
Please fill in the blanks if you find them out:
* How to enable logging to a file or standard html logs? Console output just disappears.
* How to set up things not using sqlite. What's wrong with mySQL?

===Troubleshooting===
* First of all: invoke <pre>python passenger_wsgi.py</pre> directly from the shell. This will tell you if there are import errors etc. If this command exits without any output on the console, you have already achieved a lot.
* In ''paste.ini'' set debug=true. This will show you python backtraces directly in the web page
* You will need to configure an smtp user/passwd/server in mediagoblin.ini, so you actually get the account creation token mailed out. Hint, the relevant settings are: <blockquote><pre>email_debug_mode = false
email_sender_address = postmaster@sspaeth.de
email_smtp_host = SMTP.DOMAIN.TLD
email_smtp_user = USERNAME
email_smtp_pass = WEIRDPASSWORD</pre></blockquote>
* In case you can upload media, but it does not appear. You don't have the celery server running. Add <pre>CELERY_ALWAYS_EAGER = true</pre> to the ''[celery]'' section in mediagoblin.ini

=Miscellaneous Hacks=
==Force translation==
There might some conditions under which you would prefer that MediaGoblin always return the same translation. If you are deploying with nginx and fastcgi, you can force MediaGoblin to return a specific translation regardless of browser preferences by passing a specific HTTP_ACCEPT_LANGUAGE fastcgi parameter in your location block. Example last line in location block:
<nowiki>
fastcgi_param HTTP_ACCEPT_LANGUAGE es; #force spanish translation
</nowiki>

==Create an admin user==

To create an admin user, first create the user you need either via the website or the command
<pre>
./bin/gmg adduser
</pre>

Then to turn it into an admin account use

<pre>
./bin/gmg makeadmin your_username
</pre>

Deployment

2014-06-21T07:03:16Z

Breton:

This page could use a lot of work. For now, a few smaller deployment tips!

See also: http://docs.mediagoblin.org/deploying.html (some of which may belong here)

= uwsgi =

Possibly a cleaner way to do nginx configuration than the official nginx + fastcgi suggestions!

http://qalthos.github.io/blog/FOSS@RIT/deploying-mediagoblin-1-fastcgi-vs-uwsgi.html

= uwsgi with configs =

Below there are 3 configs: for nginx, for uwsgi version 1.2.3-debian and for supervisord, used to run celery

<pre>
server {
server_name <domain>;

access_log /var/log/nginx/mg.access.log;
error_log /var/log/nginx/mg.error.log error;

#include global/common.conf;

client_max_body_size 100m;
add_header X-Content-Type-Options nosniff;

root /path/to/mediagoblin//user_dev/;

location /mgoblin_static/ {
alias /path/to/mediagoblin/mediagoblin/static/;
}
location /mgoblin_media/ {
alias /path/to/mediagoblin/user_dev/media/public/;
}
location /theme_static/ {
}
location /plugin_static/ {
}
location / {
uwsgi_pass unix:///tmp/mg.uwsgi.sock;
uwsgi_param SCRIPT_NAME "/";
include uwsgi_params;
}
}
</pre>

mg.yaml:

<pre>
uwsgi:
uid: mediagoblin
gid: mediagoblin
socket: /tmp/mg.uwsgi.sock
chown-socket: www-data:www-data
plugins: python
home: /path/to/mediagoblin/
chdir: /path/to/mediagoblin/
ini-paste: /path/to/mediagoblin/paste_local.ini
</pre>

/etc/supervisor/conf.d/mediagoblin.conf:

<pre>
[program:celery]
command=/home/mediagoblin/bretons-mediagoblin/bin/celery worker -l debug

; Set PYTHONPATH to the directory containing celeryconfig.py
environment=PYTHONPATH="/path/to/mediagoblin",MEDIAGOBLIN_CONFIG="/path/to/mediagoblin/mediagoblin_local.ini",CELERY_CONFIG_MODULE="mediagoblin.init.celery.from_celery"

directory=/path/to/mediagoblin/
user=mediagoblin
numprocs=1
; uncomment below to enable logs saving
;stdout_logfile=/some/logs/path/celeryd_stdout.log
;stderr_logfile=/some/logs/path/celeryd_stderr.log
autostart=true
autorestart=false
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
</pre>

= FCGI script =

This works great with the apache FCGID config example in the next section :) in which case you should name it "mg.fcgi".

Before use, make sure you replace '/path/to/mediagoblin/bin/python' with a real path on your server, e.g. '/srv/www/myhomepage.com/mediagoblin/bin/python'. Also replace '/path/to/mediagoblin/paste.ini'.

If you encounter problems, try executing executing the script manually, e.g. <pre>./mg.fcgi</pre>

Script:

<pre>#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

from paste.deploy import loadapp
from flup.server.fcgi import WSGIServer

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
# import os
# os.environ['CELERY_ALWAYS_EAGER'] = 'true'

def launch_fcgi():
ccengine_wsgi_app = loadapp('config:' + CONFIG_PATH)
WSGIServer(ccengine_wsgi_app).run()

if __name__ == '__main__':
launch_fcgi()</pre>

= Apache 2 Config With fcgid =

Note that the libapache2-mod-fcgi in Debian is in the main section. libapache2-mod-fcgid can be used, but requires a slightly different configuration. For yum-based distributions, you may need to install mod_fcgid.

<VirtualHost *:80>
Options +ExecCGI

# Accept up to 16MB requests
FcgidMaxRequestLen 16777216

ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

ScriptAlias / /path/to/mediagoblin/mg.fcgi/
</VirtualHost>

= Apache 2 Config With mod_wsgi =

Apache can manage the MediaGoblin application itself with mod_wsgi. It requires slightly more configuration on the Apache side, but cuts out the FastCGI middleware. See their [http://code.google.com/p/modwsgi/wiki/ConfigurationGuidelines configuration guidelines] for advanced configuration.

<VirtualHost *:80>
ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

WSGIScriptAlias / /path/to/mediagoblin/wsgi.py
# Set user and group to whatever Apache runs as
WSGIDaemonProcess gmg user=apache group=apache processes=2 threads=10 \
umask=0007 inactivity-timeout=900 maximum-requests=1000 \
python-path=/path/to/mediagoblin/:/path/to/mediagoblin/lib/python-2.x/site-packages/
WSGIProcessGroup gmg
WSGIApplicationGroup %{GLOBAL}

<Directory "/path/to/mediagoblin/mediagoblin/static/">
Order allow,deny
Allow from all
</Directory>
<Directory "/path/to/mediagoblin/user_dev/media/public">
Order allow,deny
Allow from all
</Directory>
</VirtualHost>

You'll also need to copy this into /path/to/mediagoblin/wsgi.py:

#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

from paste.deploy import loadapp

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
#import os
#os.environ['CELERY_ALWAYS_EAGER'] = 'true'

application = loadapp('config:' + CONFIG_PATH)

= Apache Config Example =
This configuration example uses mod_fastcgi.

To install and enable mod_fastcgi on a Debian/Ubuntu based system:
<pre># apt-get install libapache2-mod-suexec libapache2-mod-fastcgi
# a2enmod suexec
# a2enmod fastcgi</pre>

Sample configuration:
<pre>
<VirtualHost *:80>
ServerName mediagoblin.yourdomain.tld
ServerAdmin webmaster@yourdoimain.tld
DocumentRoot /var/www/
# Custom log files
CustomLog /var/log/apache2/mediagobling_access.log combined
ErrorLog /var/log/apache2/mediagoblin_error.log

# Serve static and media files via alias
Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

# Rewrite all URLs to fcgi, except for static and media urls
RewriteEngine On
RewriteRule ^(mgoblin_static|mgoblin_media)($|/) - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ /mg.fcgi/$1 [QSA,L]

# Allow access to static and media directories
<Directory /path/to/mediagoblin/mediagoblin/static>
Order allow,deny
Allow from all
</Directory>
<Directory /path/to/mediagoblin/mediagoblin/user_dev/media/public>
Order allow,deny
Allow from all
</Directory>

# Connect to fcgi server
FastCGIExternalServer /var/www/mg.fcgi -host 127.0.0.1:26543
</VirtualHost>
</pre>
Then, you need to make sure mediagoblin is running in fcgi mode:
<pre>cd /path/to/mediagoblin
./lazyserver.sh --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543</pre>

Note: there may be several ways to improve this configuration

If it is too slow and you use the deflate module, you could try to use the following option : SetOutputFilter INFLATE

= Lighttpd =

This configuration example uses mod_fastcgi.
Make a symbolic link from <pre>/etc/lighttpd/conf-available/10-fastcgi.conf</pre> to <pre>/etc/lighttpd/conf-enabled/10-fastcgi.conf</pre>

<pre>
ln -s /etc/lighttpd/conf-available/10-fastcgi.conf /etc/lighttpd/conf-enabled/10-fastcgi.conf
</pre>

then edit your /etc/lighttpd/conf-enabled/10-fastcgi.conf file modifying it as below to serve mediagoblin on the /tube/ path of your webserver.

<pre>
## FastCGI programs have the same functionality as CGI programs,
## but are considerably faster through lower interpreter startup
## time and socketed communication
##
## Documentation: /usr/share/doc/lighttpd-doc/fastcgi.txt.gz
## http://www.lighttpd.net/documentation/fastcgi.html

server.modules += ( "mod_fastcgi" )

## Start an FastCGI server for mediagoblin (lazyserver.sh or other deployments scripts should be up and running)
fastcgi.server = (
# url at which the app should be accessible on the server
"/tube" =>
# mnemonical name of the backend
( "mg-local" => (
"host" => "127.0.0.1",
"port" => 6543,
"docroot" => "/path/to/mediagoblin-git-folder/mediagoblin",
"check-local" => "disable"
)
)

)
</pre>

Then open your /etc/lighttpd/lighttpd.conf and add the configuration to serve directly the static/public folders

<pre>
alias.url += (
"/mgoblin_static/" => "/path/to/mediagoblin-git-folder/mediagoblin/static/",
"/mgoblin_media/" => "/path/to/mediagoblin-git-folder/user_dev/media/public/",
"/theme_static/" => "/path/to/mediagoblin-git-folder/user_dev/theme_static/",
"/plugin_static/" => "/path/to/mediagoblin-git-folder/user_dev/plugin_static/"
)

</pre>

Done!

= OpenShift =

Thers's a blogpost saying how to [http://www.sud0.com/gnu-media-goblin-on-openshift.html install mediagoblin on openshift].

= Juju =

There is a juju [https://juju.ubuntu.com/Charms charm] available for deploying mediagoblin into EC2 or on your local box. [https://juju.ubuntu.com/ juju] is available in Ubuntu 11.10 and later, though it is recommended that you pull it from [https://launchpad.net/~juju/+archive/pkgs the juju PPA], which includes backported packages going back to Ubuntu 11.10. To use the juju charm, install juju and configure either the [https://juju.ubuntu.com/docs/provider-configuration-local.html local provider] or one of the cloud API providers, currently the [https://juju.ubuntu.com/docs/provider-configuration-ec2.html EC2 provider] is the best supported and works not only with Amazon Web Services but also OpenStack clouds. There is also a newer native OpenStack API provider which is known to support HP Cloud.
<pre>
# if you have not bootstrapped
juju bootstrap
mkdir ~/charms
bzr init-repo ~/charms/precise
bzr branch lp:~clint-fewbar/charms/precise/mediagoblin/trunk ~/charms/precise/mediagoblin
juju deploy --repository ~/charms local:mediagoblin
juju expose mediagoblin
</pre>

Currently the charm is volatile, deploying from trunk, and deploys a single-server version of MediaGoblin only. It will eventually relate to the existing juju charms for other supported data stores to allow one to scale out their MediaGoblin instance.

= Init scripts =

== Debian init scripts ==

Joar has some scripts for running celery and mediagoblin separately that are designed for Debian.

https://github.com/joar/mediagoblin-init-scripts

== Arch Linux init scripts ==

[http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts Jeremy Pope has written a nice blogpost] on how to add init scripts to deploy MediaGoblin with both the python paste http server and the celery deployments separated.

If you want a simpler setup and don't want to deploy celery separately, consider either turning CELERY_ALWAYS_EAGER to true in the paste init script described above, or check out [http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/ Chimo's guide].

== Generic, simple init script ==

This is a super stupidly simple init script that was used for mediagoblin.com... note that this has Celery running in always eager mode.

You will need to adjust the paths appropriately. This could probably be better written!

#! /bin/sh

## Stupidly simple mediagoblin init script.
#
# Written in 2012 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

PASTER=/srv/mediagoblin.com/bin/paster
PASTE_CONFIG=/srv/mediagoblin.com/paste.ini
OPTIONS="--pid-file=/tmp/mediagoblin.pid \
--log-file=/srv/mediagoblin.com/paster.log \
--server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543"

CELERY_ALWAYS_EAGER=true su -pc "$PASTER serve $PASTE_CONFIG $1 $OPTIONS" webadmin

= Running on Dreamhost.com =

===Set up your python virtualenv===

dreamhost.com servers come with python 2.4 and 2.5 installed which is too old for mediagoblin. This means you need to compile and install a newer python (don't worry this is really not difficult on dreamhost servers). In order to be able to install python packages for a local user without touching system files, we need to setup a python ''virtualenv''. Fortunately, this is not too tricky either. If your server has python 2.6 or 2.7 installed already, you can skip the [[#Install a local python|python installation stuff]] and you'll only need to [[#Setup virtualenv to install local python packages|install virtualenv]].

====Install a local python====
# Download the latest python: http://python.org/ftp/python/XXX/Python-XXX.tar.bz2
# Unpack with <pre>tar xvjf Python-XXX.tar.bz2</pre>
# ''cd'' into the directory and compile and install python locally:
<blockquote><pre>./configure --prefix=$HOME/local
make
make install</pre></blockquote>
:This will install python (I used 2.7.3) into /home/<USERNAME>/local/bin/python. You might get warnings about some modules not being able to compile. :It was tcl/tk and bzip2 mainly, but things still worked out overall.
:
:You should now be able to invoke <tt>~/local/bin/python</tt> and fall into a shell of your new python (exit with ctrl-d). Congrats, you have now a new python 2.7 that you can use. However, you will need to be able to install additional packages as a user and this is what virtualenv allows.

====Setup virtualenv to install local python packages====
# Download the latest virtualenv: http://pypi.python.org/packages/source/v/virtualenv/XXX
# Install the virtualenv: <blockquote><pre>~/local/bin/python ~/virtualenv-1.8/virtualenv.py $HOME/local/virtenv</pre></blockquote>
:: You will now have: ~/local/virtenv/bin/python
# In ~/.bash_profile add: <blockquote><pre>PATH=~/local/virtenv/bin:~/local/bin:${PATH}</pre></blockquote>
:: so that your local python will be preferred.
:: Log out, log in again and test python" to see which version will be invoked. It should be the new python. Check "which easy_install" to see if the one in local/virtenv would be executed.
::
:: You have now 1) a local ''python'' installation that you can use, and 2) ''easy_install'' will also work with your local user installation. From now on you can e.g. locally install the nose testing framework (''easy_install nose'') and use it (''python -c "import nose"'').

===Install mediagoblin as a site package===

# Check out mediagoblin from git to e.g. ~/mediagoblin
# Install MediaGoblin and all dependencies for MediaGoblin with easy_install.
:* In the mediagoblin directory issue:
:<blockquote><pre>python setup.py</pre></blockquote>
:* You will also need to: easy_install lxml
:* Python-image was trickier to install: <pre>easy_install --find-links http://www.pythonware.com/products/pil/ Imaging</pre>
::Test by leaving the mediagoblin directory and see if you can import it: <pre>python -c "import mediagoblin"</pre> looking for error messages.

===Set up an WSGI environment on Dreamhost===

# Enable the mod_passenger setting for ruby/python in the domains web panel
# ''cd'' into the domain directory (e.g. ~/media.sspaeth.de for me)
# Copy mediagoblin.ini and paste.ini from the ~/mediagoblin directory here
# Create passenger_wsgi.py in your domain directory (e.g. ~/media.sspaeth.de for me):
<blockquote><pre>import sys, os
INTERP = "/home/<username>/local/virtenv/bin/python"
#INTERP is present twice so that the new python interpreter knows the actual executable path
if sys.executable != INTERP: os.execl(INTERP, INTERP, *sys.argv)

from paste.deploy import loadapp
application = loadapp('config:/home/mediagoblin/media.sspaeth.de/paste.ini')

#If in case of errors, all you get are mysterious error 500, you can set debug=true in paste.ini to see stack traces
# Otherwise, add this:
#from paste.exceptions.errormiddleware import ErrorMiddleware
#application = ErrorMiddleware(application, debug=True)</pre></blockquote>
# Set up the database by issueing: <pre>gmg dbupdate</pre>
# (optional but recommended) Serve the static files directly from the webserver.
:In paste.ini in this section: [composite:routing] comment out the static files:
#/mgoblin_static/ = mediagoblin_static
#/theme_static/ = theme_static
:and symlink the mediagobin/mediagoblin/static directoy to public/mgoblin_static
: and mediagoblin/mediagoblin/themes to public/theme_static
:
:so it is displayed directly. This step might be different depending on your web server configuration. There is no reason that static
:files need to go through all the python indirection when they can be served directly by nginx/apache/...
:
:In case you want to delete your git checkout after the installation (you don't need it, since you installed the mediagoblin package to ~/local/virtenv/lib/python2.7/mediagoblin... you should do the symlinking from there.
===Open Issues===
Please fill in the blanks if you find them out:
* How to enable logging to a file or standard html logs? Console output just disappears.
* How to set up things not using sqlite. What's wrong with mySQL?

===Troubleshooting===
* First of all: invoke <pre>python passenger_wsgi.py</pre> directly from the shell. This will tell you if there are import errors etc. If this command exits without any output on the console, you have already achieved a lot.
* In ''paste.ini'' set debug=true. This will show you python backtraces directly in the web page
* You will need to configure an smtp user/passwd/server in mediagoblin.ini, so you actually get the account creation token mailed out. Hint, the relevant settings are: <blockquote><pre>email_debug_mode = false
email_sender_address = postmaster@sspaeth.de
email_smtp_host = SMTP.DOMAIN.TLD
email_smtp_user = USERNAME
email_smtp_pass = WEIRDPASSWORD</pre></blockquote>
* In case you can upload media, but it does not appear. You don't have the celery server running. Add <pre>CELERY_ALWAYS_EAGER = true</pre> to the ''[celery]'' section in mediagoblin.ini

=Miscellaneous Hacks=
==Force translation==
There might some conditions under which you would prefer that MediaGoblin always return the same translation. If you are deploying with nginx and fastcgi, you can force MediaGoblin to return a specific translation regardless of browser preferences by passing a specific HTTP_ACCEPT_LANGUAGE fastcgi parameter in your location block. Example last line in location block:
<nowiki>
fastcgi_param HTTP_ACCEPT_LANGUAGE es; #force spanish translation
</nowiki>

==Create an admin user==

To create an admin user, first create the user you need either via the website or the command
<pre>
./bin/gmg adduser
</pre>

Then to turn it into an admin account use

<pre>
./bin/gmg makeadmin your_username
</pre>

User:Breton/GSoC2014

2014-04-06T19:00:09Z

Breton: /* Write migrations */

==About me==

My name is Boris Bobrov. I am a fourth-year student in Moscow State University, Tashkent branch, applied math and informatics faculty.

You can contact me via:
* jabber: breton@jabber.ru
* email: breton@cynicmansion.ru
* irc: breton in #mediagoblin

===Experience===
* 3.5 years of Python
* 3 years of Django
* 3.5 years of C in university
* 1.5 years of C++ in university
* 4 years of javascript (though I don't write too much in it); with jQuery and pure
* 6 years of HTML and CSS
* 1.5 year of Scheme and 12 months of Common Lisp
* basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
* a couple of FreeBSD kernel modules

===I use===
* Debian as main OS
* git (mercurial where required)
* vim

I am quite new to Mediagoblin community, but you may already know me by http://issues.mediagoblin.org/ticket/858 for the start

==Python 3 port==

===The goal===

Have Mediagoblin running on python3 and python2 with a single codebase.

===Rough plan===

====Requirements====
In bug https://issues.mediagoblin.org/ticket/813 there is a table (with my little edit):

{| class="wikitable"
|-
!Library
!py3?
!notes
|-
| setuptools || X || Distribute
|-
| PasteScript || ?? || not *technically* necessary;
|-
| wtforms || X ||
|-
| py-bcrypt || X ||
|-
| pytest>=2.3.1 || X ||
|-
| pytest-xdist || X ||
|-
| werkzeug>=0.7 || X ||
|-
| celery>=3.0 || X ||
|-
| kombu || X ||
|-
| jinja2 || X ||
|-
| sphinx || X ||
|-
| Babel<1.0 || || Babel 0.9.6 does not support py3
|-
| argparse || X ||
|-
| webtest<2 || X ||
|-
| ConfigObj || X || officially supported from now
|-
| Markdown || X ||
|-
| sqlalchemy>=0.8.0 || X ||
|-
| sqlalchemy-migrate || || unmaintained anyhow :\
|-
| mock || X ||
|-
| itsdangerous || X ||
|-
| pytz || X ||
|-
| six || X ||
|-
| oauthlib==0.5.0 || X ||
|-
| unidecode || X ||
|}

sqlalchemy-migrate is suggested to change to Alembic.

Something need to be done with Babel. '''TODO'''.

====Version considerations====

We should support is 3.2. 3.1 is too old and is present in old Debian. RHEL people install the latest version anyway. 3.2 is available in current Debian.
Though 3.3 has some nice features that would make the migration easier, u'' strings for example.

The goal is to have a nice clean codebase that runs python 2.6, 2.7, 3.3. If some time will be left, 3.2 will be supported.

====Caveats====

Article http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ describes some caveats, they should be checked:

# Metaclass Syntax Changes -- metaclasses are not used in gmg
# Dictionaries. Mostly describes how to write dictionary-like objects. I think it's not interesting in our case, though TODO.
# General Iterator Changes -- not used.
# Transformation Codecs -- are used in some parts. Also, it differs in 3.2 and 3.3.
# Filesystem and file IO access -- something happens in gmg, not sure yet, what.
# The last and the biggest -- String Handling

====Plan====

# sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
# adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
# do something about Paste. The whole Paste as well as pastescript is python3-incompatible.
# follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
# write migrations
# if time left, backport to 3.2

=====Sort out versions of dependencies=====
# Babel < 1.0 does not support py3. Maybe switch to newer versions of babel?
# TODO, some stuff didn't work while I was trying to convert to py3.

=====Adopt tox=====

=====Do something about Paste=====

=====Start porting code to python3=====
# There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
# http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
# Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
# The conversion will look like this: 3.3 -> 2.7 -> 2.6 [-> 3.2] [-> 3.4?].
# To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.

=====Write migrations=====
* Switch to Alembic
* It still doesn't support hacks around SQLite migrations.
* But there is a workaround that looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite

=====Backport to 3.2=====
* As said on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, "Supporting 3.2 is possible if you are willing to wrap all your strings in function calls which I don't recommend doing for aesthetic and performance reasons".
* Mostly we use hardcore unicode strings in tests. Maybe code aesthetics is not so important there?

==What I do and don't know==
# Paste. I have no idea what it is for.
# sqlalchemy-migrate. Never used it. Though I have worked with django's south.
# No experience in migrating to Python 3 and no proper experience in Python 3.

Also see [[/Melange]]

User:Breton/GSoC2014

2014-03-20T21:27:34Z

Breton:

==About me==

My name is Boris Bobrov. I am a fourth-year student in Moscow State University, Tashkent branch, applied math and informatics faculty.

You can contact me via:
* jabber: breton@jabber.ru
* email: breton@cynicmansion.ru
* irc: breton in #mediagoblin

===Experience===
* 3.5 years of Python
* 3 years of Django
* 3.5 years of C in university
* 1.5 years of C++ in university
* 4 years of javascript (though I don't write too much in it); with jQuery and pure
* 6 years of HTML and CSS
* 1.5 year of Scheme and 12 months of Common Lisp
* basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
* a couple of FreeBSD kernel modules

===I use===
* Debian as main OS
* git (mercurial where required)
* vim

I am quite new to Mediagoblin community, but you may already know me by http://issues.mediagoblin.org/ticket/858 for the start

==Python 3 port==

===The goal===

Have Mediagoblin running on python3 and python2 with a single codebase.

===Rough plan===

====Requirements====
In bug https://issues.mediagoblin.org/ticket/813 there is a table (with my little edit):

{| class="wikitable"
|-
!Library
!py3?
!notes
|-
| setuptools || X || Distribute
|-
| PasteScript || ?? || not *technically* necessary;
|-
| wtforms || X ||
|-
| py-bcrypt || X ||
|-
| pytest>=2.3.1 || X ||
|-
| pytest-xdist || X ||
|-
| werkzeug>=0.7 || X ||
|-
| celery>=3.0 || X ||
|-
| kombu || X ||
|-
| jinja2 || X ||
|-
| sphinx || X ||
|-
| Babel<1.0 || || Babel 0.9.6 does not support py3
|-
| argparse || X ||
|-
| webtest<2 || X ||
|-
| ConfigObj || X || officially supported from now
|-
| Markdown || X ||
|-
| sqlalchemy>=0.8.0 || X ||
|-
| sqlalchemy-migrate || || unmaintained anyhow :\
|-
| mock || X ||
|-
| itsdangerous || X ||
|-
| pytz || X ||
|-
| six || X ||
|-
| oauthlib==0.5.0 || X ||
|-
| unidecode || X ||
|}

sqlalchemy-migrate is suggested to change to Alembic.

Something need to be done with Babel. '''TODO'''.

====Version considerations====

We should support is 3.2. 3.1 is too old and is present in old Debian. RHEL people install the latest version anyway. 3.2 is available in current Debian.
Though 3.3 has some nice features that would make the migration easier, u'' strings for example.

The goal is to have a nice clean codebase that runs python 2.6, 2.7, 3.3. If some time will be left, 3.2 will be supported.

====Caveats====

Article http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ describes some caveats, they should be checked:

# Metaclass Syntax Changes -- metaclasses are not used in gmg
# Dictionaries. Mostly describes how to write dictionary-like objects. I think it's not interesting in our case, though TODO.
# General Iterator Changes -- not used.
# Transformation Codecs -- are used in some parts. Also, it differs in 3.2 and 3.3.
# Filesystem and file IO access -- something happens in gmg, not sure yet, what.
# The last and the biggest -- String Handling

====Plan====

# sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
# adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
# do something about Paste. The whole Paste as well as pastescript is python3-incompatible.
# follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
# write migrations
# if time left, backport to 3.2

=====Sort out versions of dependencies=====
# Babel < 1.0 does not support py3. Maybe switch to newer versions of babel?
# TODO, some stuff didn't work while I was trying to convert to py3.

=====Adopt tox=====

=====Do something about Paste=====

=====Start porting code to python3=====
# There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
# http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
# Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
# The conversion will look like this: 3.3 -> 2.7 -> 2.6 [-> 3.2] [-> 3.4?].
# To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.

=====Write migrations=====
* Switch to Alembic
* It still doesn't support hacks around SQLite migrations.
* But there is a workaround which looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite

=====Backport to 3.2=====
* As said on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, "Supporting 3.2 is possible if you are willing to wrap all your strings in function calls which I don't recommend doing for aesthetic and performance reasons".
* Mostly we use hardcore unicode strings in tests. Maybe code aesthetics is not so important there?

==What I do and don't know==
# Paste. I have no idea what it is for.
# sqlalchemy-migrate. Never used it. Though I have worked with django's south.
# No experience in migrating to Python 3 and no proper experience in Python 3.

Also see [[/Melange]]

User:Breton/GSoC2014/Melange

2014-03-20T21:26:31Z

Breton: Created page with "This application was submitted to Melange with compliance to GNU template. Pasting it here. <pre> Your name Boris Bobrov Your email address breton@cynicmansion.ru T..."

This application was submitted to Melange with compliance to GNU template. Pasting it here.

<pre>
Your name
Boris Bobrov

Your email address
breton@cynicmansion.ru

The name of the project
Mediagoblin: Python 3 port + switch to Alembic

Summary
Port all of MediaGoblin's code to python 3: https://issues.mediagoblin.org/ticket/813
Currently MediaGoblin uses sqlalchemy-migrate, but it's fairly outdated and unmaintained.
Port the migration infrastructure to using Alembic instead. Which is also python3-compatible.

Benefits
Python 3 is a new modern language that gaines popularity.
Also sqlalchemy-migrate used there is old and should be changed to a new shiny Alembic

Deliverables
All code will be python3.3-compatible, it means, that there the whole codebase will work with python 2.6, 2.7, 3.3+, maybe 3.2.
Minimal requirements' versions will be determined and the code will be ported to use them.
Alembic will be switched to sqlalchemy-migrate. Alembic is a database-schema migration tool.

Plan
A rough plan:
1. do something about Paste. Paste as well as pastescript is python3-incompatible.
- Drop it!
2. sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
3. adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
4. follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
- There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
- http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
- Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
- The conversion will look like this: 3.3 -> 2.7 -> 2.6 [-> 3.2] [-> 3.4?].
- To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.
5. Write migrations
- Switch to Alembic
- It still doesn't support hacks around SQLite migrations.
- But there is a workaround which looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite
6. if time left, backport to 3.2
A little script involving 2to3 shows that these types of changes are required:
import, set_literal, input, idioms, print, dict, ws_comma (sorted asc).
Timeline:
Bonding period: getting to know architecture better, reading requirements' changelogs, experimenting.
19 May -- 27 May: drop Paste, drop sqlalchemy-migrate, getting dependencies, fix tests
28 May -- 1 June: adopting tox, making current tests pass and getting failures on python 3.
2 June -- 15 June: porting
15 June -- 27 June (mid-term): if everything is done, porting to python 3.2. Else finishing what remains. Or start switching to Alembic.
30 June -- 20 Jule: integrating Alembic
31 Jule -- 11 August: backporting old migrations to Alembic

Communication
We are going to chat in email, mailing lists and IRC. My nick in IRC is breton, I already hang out in #mediagoblin on freenode.
I use git, mediagoblin is stored in git, we shall work using it.

Qualification
Experience:
3.5 years of Python
3 years of Django
3.5 years of C in university
1.5 years of C++ in university
4 years of javascript (though I don't write too much in it); with jQuery and pure
6 years of HTML and CSS
1.5 year of Scheme and 12 months of Common Lisp
basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
a couple of FreeBSD kernel modules
I use
Debian as main OS
git (mercurial where required)
vim
I liked the project because it's:
1. MediaGoblin. I like it's idea and what people do there;
2. Python;
3. Python 3. I don't have too much experience in Python 3, and I want to a) learn it better and b) learn how to port to it.
You can find my github repo here: https://github.com/bretonium/
I don't know some of the tools used there, such as Paste. I don't think it's a problem, because one of the subtasks is to drop it.
</pre>

User:Breton/GSoC2014

2014-03-20T20:58:53Z

Breton: /* Write migrations */

==About me==

My name is Boris Bobrov. I am a fourth-year student in Moscow State University, Tashkent branch, applied math and informatics faculty.

You can contact me via:
* jabber: breton@jabber.ru
* email: breton@cynicmansion.ru
* irc: breton in #mediagoblin

===Experience===
* 3.5 years of Python
* 3 years of Django
* 3.5 years of C in university
* 1.5 years of C++ in university
* 4 years of javascript (though I don't write too much in it); with jQuery and pure
* 6 years of HTML and CSS
* 1.5 year of Scheme and 12 months of Common Lisp
* basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
* a couple of FreeBSD kernel modules

===I use===
* Debian as main OS
* git (mercurial where required)
* vim

I am quite new to Mediagoblin community, but you may already know me by http://issues.mediagoblin.org/ticket/858 for the start

==Python 3 port==

===The goal===

Have Mediagoblin running on python3 and python2 with a single codebase.

===Rough plan===

====Requirements====
In bug https://issues.mediagoblin.org/ticket/813 there is a table (with my little edit):

{| class="wikitable"
|-
!Library
!py3?
!notes
|-
| setuptools || X || Distribute
|-
| PasteScript || ?? || not *technically* necessary;
|-
| wtforms || X ||
|-
| py-bcrypt || X ||
|-
| pytest>=2.3.1 || X ||
|-
| pytest-xdist || X ||
|-
| werkzeug>=0.7 || X ||
|-
| celery>=3.0 || X ||
|-
| kombu || X ||
|-
| jinja2 || X ||
|-
| sphinx || X ||
|-
| Babel<1.0 || || Babel 0.9.6 does not support py3
|-
| argparse || X ||
|-
| webtest<2 || X ||
|-
| ConfigObj || X || officially supported from now
|-
| Markdown || X ||
|-
| sqlalchemy>=0.8.0 || X ||
|-
| sqlalchemy-migrate || || unmaintained anyhow :\
|-
| mock || X ||
|-
| itsdangerous || X ||
|-
| pytz || X ||
|-
| six || X ||
|-
| oauthlib==0.5.0 || X ||
|-
| unidecode || X ||
|}

sqlalchemy-migrate is suggested to change to Alembic.

Something need to be done with Babel. '''TODO'''.

====Version considerations====

We should support is 3.2. 3.1 is too old and is present in old Debian. RHEL people install the latest version anyway. 3.2 is available in current Debian.
Though 3.3 has some nice features that would make the migration easier, u'' strings for example.

The goal is to have a nice clean codebase that runs python 2.6, 2.7, 3.3. If some time will be left, 3.2 will be supported.

====Caveats====

Article http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ describes some caveats, they should be checked:

# Metaclass Syntax Changes -- metaclasses are not used in gmg
# Dictionaries. Mostly describes how to write dictionary-like objects. I think it's not interesting in our case, though TODO.
# General Iterator Changes -- not used.
# Transformation Codecs -- are used in some parts. Also, it differs in 3.2 and 3.3.
# Filesystem and file IO access -- something happens in gmg, not sure yet, what.
# The last and the biggest -- String Handling

====Plan====

# sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
# adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
# do something about Paste. The whole Paste as well as pastescript is python3-incompatible.
# follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
# write migrations
# if time left, backport to 3.2

=====Sort out versions of dependencies=====
# Babel < 1.0 does not support py3. Maybe switch to newer versions of babel?
# TODO, some stuff didn't work while I was trying to convert to py3.

=====Adopt tox=====

=====Do something about Paste=====

=====Start porting code to python3=====
# There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
# http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
# Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
# The conversion will look like this: 3.3 -> 2.7 -> 2.6 [-> 3.2] [-> 3.4?].
# To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.

=====Write migrations=====
* Switch to Alembic
* It still doesn't support hacks around SQLite migrations.
* But there is a workaround which looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite

=====Backport to 3.2=====
* As said on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, "Supporting 3.2 is possible if you are willing to wrap all your strings in function calls which I don't recommend doing for aesthetic and performance reasons".
* Mostly we use hardcore unicode strings in tests. Maybe code aesthetics is not so important there?

==What I do and don't know==
# Paste. I have no idea what it is for.
# sqlalchemy-migrate. Never used it. Though I have worked with django's south.
# No experience in migrating to Python 3 and no proper experience in Python 3.

User:Breton/GSoC2014

2014-03-19T00:02:26Z

Breton:

==About me==

My name is Boris Bobrov. I am a fourth-year student in Moscow State University, Tashkent branch, applied math and informatics faculty.

You can contact me via:
* jabber: breton@jabber.ru
* email: breton@cynicmansion.ru
* irc: breton in #mediagoblin

===Experience===
* 3.5 years of Python
* 3 years of Django
* 3.5 years of C in university
* 1.5 years of C++ in university
* 4 years of javascript (though I don't write too much in it); with jQuery and pure
* 6 years of HTML and CSS
* 1.5 year of Scheme and 12 months of Common Lisp
* basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
* a couple of FreeBSD kernel modules

===I use===
* Debian as main OS
* git (mercurial where required)
* vim

I am quite new to Mediagoblin community, but you may already know me by http://issues.mediagoblin.org/ticket/858 for the start

==Python 3 port==

===The goal===

Have Mediagoblin running on python3 and python2 with a single codebase.

===Rough plan===

====Requirements====
In bug https://issues.mediagoblin.org/ticket/813 there is a table (with my little edit):

{| class="wikitable"
|-
!Library
!py3?
!notes
|-
| setuptools || X || Distribute
|-
| PasteScript || ?? || not *technically* necessary;
|-
| wtforms || X ||
|-
| py-bcrypt || X ||
|-
| pytest>=2.3.1 || X ||
|-
| pytest-xdist || X ||
|-
| werkzeug>=0.7 || X ||
|-
| celery>=3.0 || X ||
|-
| kombu || X ||
|-
| jinja2 || X ||
|-
| sphinx || X ||
|-
| Babel<1.0 || || Babel 0.9.6 does not support py3
|-
| argparse || X ||
|-
| webtest<2 || X ||
|-
| ConfigObj || X || officially supported from now
|-
| Markdown || X ||
|-
| sqlalchemy>=0.8.0 || X ||
|-
| sqlalchemy-migrate || || unmaintained anyhow :\
|-
| mock || X ||
|-
| itsdangerous || X ||
|-
| pytz || X ||
|-
| six || X ||
|-
| oauthlib==0.5.0 || X ||
|-
| unidecode || X ||
|}

sqlalchemy-migrate is suggested to change to Alembic.

Something need to be done with Babel. '''TODO'''.

====Version considerations====

We should support is 3.2. 3.1 is too old and is present in old Debian. RHEL people install the latest version anyway. 3.2 is available in current Debian.
Though 3.3 has some nice features that would make the migration easier, u'' strings for example.

The goal is to have a nice clean codebase that runs python 2.6, 2.7, 3.3. If some time will be left, 3.2 will be supported.

====Caveats====

Article http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ describes some caveats, they should be checked:

# Metaclass Syntax Changes -- metaclasses are not used in gmg
# Dictionaries. Mostly describes how to write dictionary-like objects. I think it's not interesting in our case, though TODO.
# General Iterator Changes -- not used.
# Transformation Codecs -- are used in some parts. Also, it differs in 3.2 and 3.3.
# Filesystem and file IO access -- something happens in gmg, not sure yet, what.
# The last and the biggest -- String Handling

====Plan====

# sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
# adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
# do something about Paste. The whole Paste as well as pastescript is python3-incompatible.
# follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
# write migrations
# if time left, backport to 3.2

=====Sort out versions of dependencies=====
# Babel < 1.0 does not support py3. Maybe switch to newer versions of babel?
# TODO, some stuff didn't work while I was trying to convert to py3.

=====Adopt tox=====

=====Do something about Paste=====

=====Start porting code to python3=====
# There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
# http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
# Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
# The conversion will look like this: 3.3 -> 2.7 -> 2.6 [-> 3.2] [-> 3.4?].
# To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.

=====Write migrations=====
* Maybe switch to Alembic
* It still doesn't support hacks around SQLite migrations.
* But there is a workaround which looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite

=====Backport to 3.2=====
* As said on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, "Supporting 3.2 is possible if you are willing to wrap all your strings in function calls which I don't recommend doing for aesthetic and performance reasons".
* Mostly we use hardcore unicode strings in tests. Maybe code aesthetics is not so important there?

==What I do and don't know==
# Paste. I have no idea what it is for.
# sqlalchemy-migrate. Never used it. Though I have worked with django's south.
# No experience in migrating to Python 3 and no proper experience in Python 3.

User:Breton/GSoC2014

2014-03-19T00:00:30Z

Breton: fixed lists

==About me==

My name is Boris Bobrov. I am a fourth-year student in Moscow State University, Tashkent branch, applied math and informatics faculty.

You can contact me via:
* jabber: breton@jabber.ru
* email: breton@cynicmansion.ru
* irc: breton in #mediagoblin

===Experience===
* 3.5 years of Python
* 3 years of Django
* 3.5 years of C in university
* 1.5 years of C++ in university
* 4 years of javascript (though I don't write too much in it); with jQuery and pure
* 6 years of HTML and CSS
* 1.5 year of Scheme and 12 months of Common Lisp
* basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
* a couple of FreeBSD kernel modules

===I use===
* Debian as main OS
* git (mercurial where required)
* vim

I am quite new to Mediagoblin community, but you may already know me by http://issues.mediagoblin.org/ticket/858 for the start

==Python 3 port==

===The goal===

Have Mediagoblin running on python3 and python2 with a single codebase.

===Rough plan===

====Requirements====
In bug https://issues.mediagoblin.org/ticket/813 there is a table (with my little edit):

{| class="wikitable"
|-
!Library
!py3?
!notes
|-
| setuptools || X || Distribute
|-
| PasteScript || ?? || not *technically* necessary;
|-
| wtforms || X ||
|-
| py-bcrypt || X ||
|-
| pytest>=2.3.1 || X ||
|-
| pytest-xdist || X ||
|-
| werkzeug>=0.7 || X ||
|-
| celery>=3.0 || X ||
|-
| kombu || X ||
|-
| jinja2 || X ||
|-
| sphinx || X ||
|-
| Babel<1.0 || || Babel 0.9.6 does not support py3
|-
| argparse || X ||
|-
| webtest<2 || X ||
|-
| ConfigObj || X || officially supported from now
|-
| Markdown || X ||
|-
| sqlalchemy>=0.8.0 || X ||
|-
| sqlalchemy-migrate || || unmaintained anyhow :\
|-
| mock || X ||
|-
| itsdangerous || X ||
|-
| pytz || X ||
|-
| six || X ||
|-
| oauthlib==0.5.0 || X ||
|-
| unidecode || X ||
|}

sqlalchemy-migrate is suggested to change to Alembic.

Something need to be done with Babel. '''TODO'''.

====Version considerations====

We should support is 3.2. 3.1 is too old and is present in old Debian. RHEL people install the latest version anyway. 3.2 is available in current Debian.
Though 3.3 has some nice features that would make the migration easier, u'' strings for example.

The goal is to have a nice clean codebase that runs python 2.6, 2.7, 3.3. If some time will be left, 3.2 will be supported.

====Caveats====

Article http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ describes some caveats, they should be checked:

# Metaclass Syntax Changes -- metaclasses are not used in gmg
# Dictionaries. Mostly describes how to write dictionary-like objects. I think it's not interesting in our case, though TODO.
# General Iterator Changes -- not used.
# Transformation Codecs -- are used in some parts. Also, it differs in 3.2 and 3.3.
# Filesystem and file IO access -- something happens in gmg, not sure yet, what.
# The last and the biggest -- String Handling

====Plan====

# sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
# adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
# do something about Paste. The whole Paste as well as pastescript is python3-incompatible.
# follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
# write migrations
# if time left, backport to 3.2

=====Sort out versions of dependencies=====
# Babel < 1.0 does not support py3. Maybe switch to newer versions of babel?
# TODO, some stuff didn't work while I was trying to convert to py3.

=====Adopt tox=====

=====Do something about Paste=====

=====Start porting code to python3=====
# There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
# http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
# Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
# The conversion will look like this: 3.3 -> 2.7 -> 2.6 [-> 3.2] [-> 3.4?].
# To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.

=====Write migrations=====
* Maybe switch to Alembic
* It still doesn't support hacks around SQLite migrations.
* But there is a workaround which looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite

=====Backport to 3.2=====
* As said on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, "Supporting 3.2 is possible if you are willing to wrap all your strings in function calls which I don't recommend doing for aesthetic and performance reasons".
* Mostly we use hardcore unicode strings in tests. Maybe code aesthetics is not so important there?

==What I do and don't know==
# Paste. I have no idea what it is for.
# sqlalchemy-migrate. Never used it. Though I have worked with django's south.

User:Breton/GSoC2014

2014-03-18T23:56:14Z

Breton:

==About me==

My name is Boris Bobrov. I am a fourth-year student in Moscow State University, Tashkent branch, applied math and informatics faculty.

You can contact me via:
- jabber: breton@jabber.ru
- email: breton@cynicmansion.ru
- irc: breton in #mediagoblin

===Experience===
- 3.5 years of Python
- 3 years of Django
- 3.5 years of C in university
- 1.5 years of C++ in university
- 4 years of javascript (though I don't write too much in it); with jQuery and pure
- 6 years of HTML and CSS
- 1.5 year of Scheme and 12 months of Common Lisp
- basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
- a couple of FreeBSD kernel modules

===I use===
- Debian as main OS
- git (mercurial where required)
- vim

I am quite new to Mediagoblin community, but you may already know me by http://issues.mediagoblin.org/ticket/858 for the start

==Python 3 port==

===The goal===

Have Mediagoblin running on python3 and python2 with a single codebase.

===Rough plan===

====Requirements====
In bug https://issues.mediagoblin.org/ticket/813 there is a table (with my little edit):

{| class="wikitable"
|-
!Library
!py3?
!notes
|-
| setuptools || X || Distribute
|-
| PasteScript || ?? || not *technically* necessary;
|-
| wtforms || X ||
|-
| py-bcrypt || X ||
|-
| pytest>=2.3.1 || X ||
|-
| pytest-xdist || X ||
|-
| werkzeug>=0.7 || X ||
|-
| celery>=3.0 || X ||
|-
| kombu || X ||
|-
| jinja2 || X ||
|-
| sphinx || X ||
|-
| Babel<1.0 || || Babel 0.9.6 does not support py3
|-
| argparse || X ||
|-
| webtest<2 || X ||
|-
| ConfigObj || X || officially supported from now
|-
| Markdown || X ||
|-
| sqlalchemy>=0.8.0 || X ||
|-
| sqlalchemy-migrate || || unmaintained anyhow :\
|-
| mock || X ||
|-
| itsdangerous || X ||
|-
| pytz || X ||
|-
| six || X ||
|-
| oauthlib==0.5.0 || X ||
|-
| unidecode || X ||
|}

sqlalchemy-migrate is suggested to change to Alembic.

Something need to be done with Babel. '''TODO'''.

====Version considerations====

We should support is 3.2. 3.1 is too old and is present in old Debian. RHEL people install the latest version anyway. 3.2 is available in current Debian.
Though 3.3 has some nice features that would make the migration easier, u'' strings for example.

The goal is to have a nice clean codebase that runs python 2.6, 2.7, 3.3. If some time will be left, 3.2 will be supported.

====Caveats====

Article http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ describes some caveats, they should be checked:

# Metaclass Syntax Changes -- metaclasses are not used in gmg
# Dictionaries. Mostly describes how to write dictionary-like objects. I think it's not interesting in our case, though TODO.
# General Iterator Changes -- not used.
# Transformation Codecs -- are used in some parts. Also, it differs in 3.2 and 3.3.
# Filesystem and file IO access -- something happens in gmg, not sure yet, what.
# The last and the biggest -- String Handling

====Plan====

# sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
# adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
# do something about Paste. The whole Paste as well as pastescript is python3-incompatible.
# follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
# write migrations
# if time left, backport to 3.2

=====Sort out versions of dependencies=====
# Babel < 1.0 does not support py3. Maybe switch to newer versions of babel?
# TODO, some stuff didn't work while I was trying to convert to py3.

=====Adopt tox=====

=====Do something about Paste=====

=====Start porting code to python3=====
# There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
# http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
# Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
# The conversion will look like this: 3.3 -> 2.7 -> 2.6 [-> 3.2] [-> 3.4?].
# To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.

=====Write migrations=====
- Maybe switch to Alembic
- It still doesn't support hacks around SQLite migrations.
- But there is a workaround which looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite

=====Backport to 3.2=====
- As said on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, "Supporting 3.2 is possible if you are willing to wrap all your strings in function calls which I don't recommend doing for aesthetic and performance reasons".
- Mostly we use hardcore unicode strings in tests. Maybe code aesthetics is not so important there?

==What I do and don't know==
# Paste. I have no idea what it is for.
# sqlalchemy-migrate. Never used it. Though I have worked with django's south.

User:Breton/GSoC2014

2014-03-18T23:54:27Z

Breton: styling and some additions

My name is Boris Bobrov. I am a fourth-year student in Moscow State University, Tashkent branch, applied math and informatics faculty.

==About me==
===Experience===
- 3.5 years of Python
- 3 years of Django
- 3.5 years of C in university
- 1.5 years of C++ in university
- 4 years of javascript (though I don't write too much in it); with jQuery and pure
- 6 years of HTML and CSS
- 1.5 year of Scheme and 12 months of Common Lisp
- basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
- a couple of FreeBSD kernel modules

===I use===
- Debian as main OS
- git (mercurial where required)
- vim

I am quite new to Mediagoblin community, but you may already know me by http://issues.mediagoblin.org/ticket/858 for the start

==Python 3 port==

===The goal===

Have Mediagoblin running on python3 and python2 with a single codebase.

===Rough plan===

====Requirements====
In bug https://issues.mediagoblin.org/ticket/813 there is a table (with my little edit):

{| class="wikitable"
|-
!Library
!py3?
!notes
|-
| setuptools || X || Distribute
|-
| PasteScript || ?? || not *technically* necessary;
|-
| wtforms || X ||
|-
| py-bcrypt || X ||
|-
| pytest>=2.3.1 || X ||
|-
| pytest-xdist || X ||
|-
| werkzeug>=0.7 || X ||
|-
| celery>=3.0 || X ||
|-
| kombu || X ||
|-
| jinja2 || X ||
|-
| sphinx || X ||
|-
| Babel<1.0 || || Babel 0.9.6 does not support py3
|-
| argparse || X ||
|-
| webtest<2 || X ||
|-
| ConfigObj || X || officially supported from now
|-
| Markdown || X ||
|-
| sqlalchemy>=0.8.0 || X ||
|-
| sqlalchemy-migrate || || unmaintained anyhow :\
|-
| mock || X ||
|-
| itsdangerous || X ||
|-
| pytz || X ||
|-
| six || X ||
|-
| oauthlib==0.5.0 || X ||
|-
| unidecode || X ||
|}

sqlalchemy-migrate is suggested to change to Alembic.

Something need to be done with Babel. '''TODO'''.

====Version considerations====

We should support is 3.2. 3.1 is too old and is present in old Debian. RHEL people install the latest version anyway. 3.2 is available in current Debian.
Though 3.3 has some nice features that would make the migration easier, u'' strings for example.

The goal is to have a nice clean codebase that runs python 2.6, 2.7, 3.3. If some time will be left, 3.2 will be supported.

====Caveats====

Article http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ describes some caveats, they should be checked:

# Metaclass Syntax Changes -- metaclasses are not used in gmg
# Dictionaries. Mostly describes how to write dictionary-like objects. I think it's not interesting in our case, though TODO.
# General Iterator Changes -- not used.
# Transformation Codecs -- are used in some parts. Also, it differs in 3.2 and 3.3.
# Filesystem and file IO access -- something happens in gmg, not sure yet, what.
# The last and the biggest -- String Handling

====Plan====

# sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
# adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
# do something about Paste. The whole Paste as well as pastescript is python3-incompatible.
# follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
# write migrations
# if time left, backport to 3.2

=====Sort out versions of dependencies=====
# Babel < 1.0 does not support py3. Maybe switch to newer versions of babel?
# TODO, some stuff didn't work while I was trying to convert to py3.

=====Adopt tox=====

=====Do something about Paste=====

=====Start porting code to python3=====
# There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
# http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
# Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
# The conversion will look like this: 3.3 -> 2.7 -> 2.6 [-> 3.2] [-> 3.4?].
# To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.

=====Write migrations=====
- Maybe switch to Alembic
- It still doesn't support hacks around SQLite migrations.
- But there is a workaround which looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite

=====Backport to 3.2=====
- As said on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, "Supporting 3.2 is possible if you are willing to wrap all your strings in function calls which I don't recommend doing for aesthetic and performance reasons".
- Mostly we use hardcore unicode strings in tests. Maybe code aesthetics is not so important there?

==What I do and don't know==
# Paste. I have no idea what it is for.
# sqlalchemy-migrate. Never used it. Though I have worked with django's south.

User:Breton/GSoC2014

2014-03-18T23:31:07Z

Breton: application unstyled

My name is Boris Bobrov. I am a fourth-year student in Moscow State University, Tashkent branch, applied math and informatics faculty.

Experience:
- 3.5 years of Python
- 3 years of Django
- 3.5 years of C in university
- 1.5 years of C++ in university
- 4 years of javascript (though I don't write too much in it); with jQuery and pure
- 6 years of HTML and CSS
- 1.5 year of Scheme and 12 months of Common Lisp
- basic knowledge of system administration (set up nginx, uwsgi etc, familiar with cron, shell scripting)
- a couple of FreeBSD kernel modules

I use:
- Debian as main OS
- git (mercurial where required)
- vim

I am quite new to Mediagoblin community, but you may already know me by http://issues.mediagoblin.org/ticket/858 for the start

Python 3 port.

The goal is to have Mediagoblin running on python3 and a /HackingHowto for those who want it too.

In bug https://issues.mediagoblin.org/ticket/813 there is a table (with my little edit):
| Library | py3? | notes |
|--------------------+------+-------------------------------------|
| setuptools | X | Distribute |
| PasteScript | ?? | not *technically* necessary; |
| wtforms | X | |
| py-bcrypt | X | |
| pytest>=2.3.1 | X | |
| pytest-xdist | X | |
| werkzeug>=0.7 | X | |
| celery>=3.0 | X | |
| kombu | X | |
| jinja2 | X | |
| sphinx | X | |
| Babel<1.0 | | Babel 0.9.6 does not support py3 |
| argparse | X | |
| webtest<2 | X | |
| ConfigObj | X | officially supported from now |
| Markdown | X | |
| sqlalchemy>=0.8.0 | X | |
| sqlalchemy-migrate | | unmaintained anyhow :\ |
| mock | X | |
| itsdangerous | X | |
| pytz | X | |
| six | X | |
| oauthlib==0.5.0 | X | |
| unidecode | X | |

sqlalchemy-migrate is suggested to change to Alembic, which runs python3. TODO: tests with sqlite

Something need to be done with Babel. TODO.

We should support is 3.2. 3.1 is too old and is present in old Debian. RHEL people install the latest version anyway. 3.2 is available in current Debian.
Though 3.3 has some nice features that would make the migration easier, u'' strings for example.

The goal is to have a nice clean codebase that runs python 2.6, 2.7, 3.3. If some time will be left, 3.2 will be supported.

Article http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/ describes some caveats, they should be checked:

1. Metaclass Syntax Changes -- metaclasses are not used in gmg
2. Dictionaries. Mostly describes how to write dictionary-like objects. I think it's not interesting in our case, though TODO.
3. General Iterator Changes -- not used.
4. Transformation Codecs -- are used in some parts. Also, it differs in 3.2 and 3.3.
5. Filesystem and file IO access -- something happens in gmg, not sure yet, what.
6. The last and the biggest -- String Handling

Plan.

0. sort out versions of dependencies. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel? Etc.
1. adopt tox -- https://bitbucket.org/hpk42/tox, use it for different Python versions and interpreters.
2. do something about Paste. The whole paste, as well as pastescript is python3-incompatible.
3. follow suggestion on #813: "Start porting MediaGoblin's code to python 3 generally. Ignore migrations, just support MediaGoblin "starting with the latest database schema", or even easier, set up your database on the python 2 branch before you git checkout the python 3 branch. ;)"
4. write migrations
5. if time left, backport to 3.2

Sort out versions of dependencies.
1. Babel < 1.0 does not support py3. Maybe switch to newer versions of babel?
X. TODO, some stuff didn't work while I was trying to convert to py3.

Adopt tox.

Do something about Paste

Start porting code to python3 generally.
1. There is an option to use "six". I suggest not to use it, for code cleanness. If some compatibility code will be required, it will be shipped in _compat module.
2. http://python3porting.com/noconv.html suggests to convert to python3 and then convert back to python2. I agree with that and will do so.
3. Hold up to ideas listed on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, like "Test for 2.x not 3.x".
4. The conversion will look like this: 3.3 -> 2.7 -> 2.6 -> 3.2 (-> 3.4?).
5. To minimize the number of errors need to be fixed during an iteration, the conversion will be done by a number of fixes. A fix is a line from "2to3 -l". Every "fix" will be tested in order mentioned above, after it gives 0 errors, another one will be applied.

Write migrations.
Maybe switch to Alembic. It still doesn't support hacks around SQLite migrations.
But there is a workaround which looks small and which we could use perhaps -- https://github.com/klugjohannes/alembic-sqlite

Backport to 3.2.
As said on http://lucumr.pocoo.org/2013/5/21/porting-to-python-3-redux/, "Supporting 3.2 is possible if you are willing to wrap all your strings in function calls which I don't recommend doing for aesthetic and performance reasons".
Mostly we use hardcore unicode strings in tests. Maybe code aesthetics is not so important there?