GNU MediaGoblin Wiki - User contributions [en]

Main Page

2014-09-01T12:02:46Z

Moggers87: Undo revision 1568 by EpifaniChamberl (talk)

= Want to Join the MediaGoblin Community? =

We’re really glad that you want to join the MediaGoblin community!

There are a variety of ways to help and support MediaGoblin and to join the team. If you want to code, great, if not, even better! MediaGoblin interested contributors in many different roles: users, system administrators, technical writers, testers, evangelists, UI/UX and graphics designers, cheerleaders, and dreamers.

We observe the [https://www.djangoproject.com/conduct/ Django code of conduct]. Be welcoming, friendly, and patient!

This wiki covers a variety of ways that you can get involved with MediaGoblin as well as instructions on how to get started.

== Hang out with the MediaGoblin folk ==

MediaGoblin has a mailing list and an IRC channel where we hang out. See [http://mediagoblin.org/pages/join.html our join page] for links.

Please drop by and say “Hi!” And, if you’re looking for something to do, just ask---there’s always work to be done.

== Take Part in the Monthly Meetings ==

Each month is a [[:Meeting|Meeting]]. You can take part and help decide on the future of MediaGoblin. Or just be around and see what's happening live!

=How Can you help ?=

First and foremost, for many (but not all) types of contributions you may want to set up a local instance. To learn how to do this, see the [[HackingHowto]] page. (If you're not familiar with command line level things, there are still ways you can help below!)

== File Bugs / Triage Bugs ==

Issue reports are critical for all projects. Identified bugs give developers a basis for beginning work, and providing an idea of what features and issues are most important to users and the overall usability of the software. If you identify errors, flaws, unexpected behaviors, or deficits that impede use, file a bug.

* [[File Bugs]] -- notes on filing new bugs/issues/feature requests
* [[Feature Ideas]] -- notes on possible features
* [[Triage Bugs]] -- notes on triaging
* [[BugTriageDay]] -- every other Thursday is bug triage day where anyone can help out triaging bugs

== Send Encouragement / Spread the Word ==

Sometimes, a nice word, simple encouragement, and interest in the work we’re doing is enough to inspire a tizzy of productive work. Just a bit more interest and encouragement can even make the difference between a complete feature and limited functionality; between a completed milestone and lost momentum.

Similarly, MediaGoblin, and the movement for free network services, is always in need of encouragement. Use free network services, understand the principals behind the movement, be able to articulate the benefits of free network services and the problems with psudo-free applications that don’t respect the users’ freedom.

Write a blog post, post a status update, drop by the listserv or join #mediagoblin on freenode.net and let us know. See [http://mediagoblin.org/join/ our join page] for links.

== Write Documentation / Edit Documentation ==

* [[Documentation quick start]] - How to contribute to the documentation effort.
* [[ManualStandards]] - covers the standards for writing the user manual (forthcoming.)

== Test MediaGoblin ==

Do you have access to the web? Do you like sharing your opinions? If so, we need your help to test MediaGoblin! Testers play around with the current test instance, note what operating system and browser they use (notes on multiple set-ups are also helpful) and take some notes. That's it! It's a very important task that doesn't require any special knowledge and you're done in under an hour. Ready to help?

* [[User Experience]] - user experience testing. Includes link to an instance you can try!
* [[UnitTests|Unit Tests]] - all about the unit tests
* [[Manual_Functional_Testing|Manual Functional Testing]] - a great way to get to know MediaGoblin.

== Translate MediaGoblin ==

If you know English and another language and feel comfortable translating elements of the interface or even the documentation, we’d love to have help translating the software and resources.
Translating MediaGoblin is very easy with a web interface, so there is no programming knowledge required at all.

* [[Translations]] - How to translate stuff or update the translations

== Become a User ==

Coming soon!

We’re building MediaGoblin for us and for you but really you’re one of us and I am you and we are we and MediaGoblin is the walrus.

We're planning to launch our own public instance of MediaGoblin in the near future--probably in the September/October 2011 time frame. When we do, sign up for an account, use the service and relish in the thought that this service comes with a heaping side of Freedom and you can salt and pepper it to your liking.

== Help Others ==

Have you spent time with MediaGoblin? If so, your experience and wisdom are invaluable and you’re the best person we can think of to help other users with their questions.

Hang out on the IRC channel and help answer new peoples' questions. See [http://mediagoblin.org/join/ our join page] for links.

== Run your own MediaGoblin Instance ==

Are there things about our instance you want to change? Are there things about other instances you wish were different? Want to test upcoming changes? Want to create patches to implement things you need? That’s great—you can run your own instance!

The primary documentation for this is at [http://docs.mediagoblin.org http://docs.mediagoblin.org] but here are some additional tips:

* [[Configure_MediaGoblin|Configuration]] - Learn about MediaGoblin configuration files and file options.
* [[Deployment]] - General deployment advice
* [[Scaling Down]] - Minimizing MediaGoblin's resource requirements
* [[Virtual Machine Hosting]] - Deploy your own publicly available MediaGoblin server using [http://aws.amazon.com/free/?utm_source=adwords&utm_medium=cpc&utm_campaign=CPC_Google_AWS_ec2&utm_content=TextV01_PP_V01_EC2&trk=CPC_Google_AWS_ec2 Amazon's free EC2 tier].

= Technical project documentation =

The technical docs, that are more finished and the ones that are better maintained near the code (so they stay up to date) are in the more technical chapters of the [http://docs.mediagoblin.org/ main documentation].
* [[Storage]] - How MediaGoblin's internal storage system works.
* [[Processing]] - What happens after you submit your image/video/etc? Processing! More about that.
* [https://gitorious.org/mediagoblin/mediagoblin/blobs/master/extlib/README External Library Policy] - covers use of external libraries
* [[User:Cwebber/braindumps]] - Chris Webber's braindumps (you can help refactoring these into real sections of the site!)
* [[Multiple media support]] - Design plan for multiple media support

== Write Code / Fix Code ==

MedaGoblin development is premised on the idea that the entire interface for the platform be completely theme-able. If you have a design or theming background, consider developing themes for MediaGoblin. New themes help test the theming system, provide attractive and appealing interfaces for prospective users. If you want to start a new theme but don’t know where to start, touch base with the development community on the list or in the IRC channel for more information.

If you are a coder and you would like to write code, the repository is hosted on gitorious. Clone or fork the repository and start poking around. Become familiar with this manual for an overview of how the software works and is used. Consider the contributor wiki for more information about the project, our preferred methods, and guides for developing MediaGoblin. We even have tips on becoming a coder and we’re willing to help!

* [[HackingHowto|Hacking]] - notes on making and sending in code contributions
** [[BeginnersCorner|Beginner's Corner]] - resources for those who are new to Python or Git.
** ''Started from an older version of the Hacking Howto? We switched from buildout->virtualenv, so look at [[Moving from buildout to virtualenv]] for information on how to move over.''
* [[Git workflow]] - How to go about submitting patches via git.
* [[Code review tips]] - Tips on how to go about doing local code review
* [[Templating]] - How our templating structure is set up
* [[Code overview]] - Overview of the structure of the codebase

== Create a Theme ==

See [http://docs.mediagoblin.org/siteadmin/theming.html the theming docs]

== Write a plugin ==

If you start, you'll find some basic documentation [http://docs.mediagoblin.org/#part-4-developer-s-zone| in the docs site]

You've made a plugin ? List it here :

[[Available_Plugins|Available plugins]]

You've written a plugin ? You got some tips to share ? A tutorial idea ? Please do :

[[PluginsTips|Plugins Tips]]

= Misc =

== Android client ==

See [[Android Client]]

= Inner workings of the secret sanctum =

* [[IRCBot]] - covers our irc bot
* [[ReleaseProcess|Release Process]] - covers the release process
* [[Update the website]] - Learn how to update mediagoblin.org!

=FAQ=

[[GMG FAQ]]

Deployment

2014-07-20T14:15:53Z

Moggers87: /* Apache 2 Config With mod_wsgi */ whoops, don't run gmg as apache!

This page could use a lot of work. For now, a few smaller deployment tips!

See also: http://docs.mediagoblin.org/deploying.html (some of which may belong here)

= uwsgi =

Possibly a cleaner way to do nginx configuration than the official nginx + fastcgi suggestions!

http://qalthos.github.io/blog/FOSS@RIT/deploying-mediagoblin-1-fastcgi-vs-uwsgi.html

= uwsgi with configs =

Below there are 3 configs: for nginx, for uwsgi version 1.2.3-debian and for supervisord, used to run celery.

In future, when newer versions of uwsgi appear in debian, supervisor can be removed.

All instructions below are for Debian 7. They should also work with rpm-based systems, but with other paths.

<pre>apt-get install uwsgi uwsgi-plugin-python nginx-full supervisor;</pre>

/etc/nginx/sites-available/mediagoblin:
<pre>
server {
server_name <domain>;

access_log /var/log/nginx/mg.access.log;
error_log /var/log/nginx/mg.error.log error;

#include global/common.conf;

client_max_body_size 100m;
add_header X-Content-Type-Options nosniff;

root /path/to/mediagoblin//user_dev/;

location /mgoblin_static/ {
alias /path/to/mediagoblin/mediagoblin/static/;
}
location /mgoblin_media/ {
alias /path/to/mediagoblin/user_dev/media/public/;
}
location /theme_static/ {
}
location /plugin_static/ {
}
location / {
uwsgi_pass unix:///tmp/mg.uwsgi.sock;
uwsgi_param SCRIPT_NAME "/";
include uwsgi_params;
}
}
</pre>

/etc/uwsgi/apps-available/mg.yaml:
<pre>
uwsgi:
uid: mediagoblin
gid: mediagoblin
socket: /tmp/mg.uwsgi.sock
chown-socket: www-data:www-data
plugins: python
home: /path/to/mediagoblin/
chdir: /path/to/mediagoblin/
ini-paste: /path/to/mediagoblin/paste_local.ini
</pre>

/etc/supervisor/conf.d/mediagoblin.conf:
<pre>
[program:celery]
command=/path/to/mediagoblin/bin/celery worker -l debug

; Set PYTHONPATH to the directory containing celeryconfig.py
environment=PYTHONPATH="/path/to/mediagoblin",MEDIAGOBLIN_CONFIG="/path/to/mediagoblin/mediagoblin_local.ini",CELERY_CONFIG_MODULE="mediagoblin.init.celery.from_celery"

directory=/path/to/mediagoblin/
user=mediagoblin
numprocs=1
; uncomment below to enable logs saving
;stdout_logfile=/some/logs/path/celeryd_stdout.log
;stderr_logfile=/some/logs/path/celeryd_stderr.log
autostart=true
autorestart=false
startsecs=10

; Need to wait for currently executing tasks to finish at shutdown.
; Increase this if you have very long running tasks.
stopwaitsecs = 600
</pre>

After setting all configs do:
<pre>
ln -s /etc/nginx/sites-available/mediagoblin /etc/nginx/sites-enabled/
ln -s /etc/uwsgi/apps-available/mg.yaml /etc/uwsgi/apps-enabled/
</pre>

= FCGI script =

This works great with the apache FCGID config example in the next section :) in which case you should name it "mg.fcgi".

Before use, make sure you replace '/path/to/mediagoblin/bin/python' with a real path on your server, e.g. '/srv/www/myhomepage.com/mediagoblin/bin/python'. Also replace '/path/to/mediagoblin/paste.ini'.

If you encounter problems, try executing executing the script manually, e.g. <pre>./mg.fcgi</pre>

Script:

<pre>#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

from paste.deploy import loadapp
from flup.server.fcgi import WSGIServer

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
# import os
# os.environ['CELERY_ALWAYS_EAGER'] = 'true'

def launch_fcgi():
ccengine_wsgi_app = loadapp('config:' + CONFIG_PATH)
WSGIServer(ccengine_wsgi_app).run()

if __name__ == '__main__':
launch_fcgi()</pre>

= Apache 2 Config With fcgid =

Note that the libapache2-mod-fcgi in Debian is in the main section. libapache2-mod-fcgid can be used, but requires a slightly different configuration. For yum-based distributions, you may need to install mod_fcgid.

<VirtualHost *:80>
Options +ExecCGI

# Accept up to 16MB requests
FcgidMaxRequestLen 16777216

ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

ScriptAlias / /path/to/mediagoblin/mg.fcgi/
</VirtualHost>

= Apache 2 Config With mod_wsgi =

Apache can manage the MediaGoblin application itself with mod_wsgi. It requires slightly more configuration on the Apache side, but cuts out the FastCGI middleware. See their [http://code.google.com/p/modwsgi/wiki/ConfigurationGuidelines configuration guidelines] for advanced configuration.

<VirtualHost *:80>
ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

WSGIScriptAlias / /path/to/mediagoblin/wsgi.py
# Set user and group to whatever user you used to install mediagoblin
# Remember to change python-path too!
WSGIDaemonProcess gmg user=mediagoblin group=mediagoblin processes=2 threads=10 \
umask=0007 inactivity-timeout=900 maximum-requests=1000 \
python-path=/path/to/mediagoblin/:/path/to/mediagoblin/lib/python-2.x/site-packages/
WSGIProcessGroup gmg
WSGIApplicationGroup %{GLOBAL}

<Directory "/path/to/mediagoblin/mediagoblin/static/">
Order allow,deny
Allow from all
</Directory>
<Directory "/path/to/mediagoblin/user_dev/media/public">
Order allow,deny
Allow from all
</Directory>
</VirtualHost>

You'll also need to copy this into /path/to/mediagoblin/wsgi.py:

#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

from paste.deploy import loadapp

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
#import os
#os.environ['CELERY_ALWAYS_EAGER'] = 'true'

application = loadapp('config:' + CONFIG_PATH)

= Apache Config Example =
This configuration example uses mod_fastcgi.

To install and enable mod_fastcgi on a Debian/Ubuntu based system:
<pre># apt-get install libapache2-mod-suexec libapache2-mod-fastcgi
# a2enmod suexec
# a2enmod fastcgi</pre>

Sample configuration:
<pre>
<VirtualHost *:80>
ServerName mediagoblin.yourdomain.tld
ServerAdmin webmaster@yourdoimain.tld
DocumentRoot /var/www/
# Custom log files
CustomLog /var/log/apache2/mediagobling_access.log combined
ErrorLog /var/log/apache2/mediagoblin_error.log

# Serve static and media files via alias
Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

# Rewrite all URLs to fcgi, except for static and media urls
RewriteEngine On
RewriteRule ^(mgoblin_static|mgoblin_media)($|/) - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ /mg.fcgi/$1 [QSA,L]

# Allow access to static and media directories
<Directory /path/to/mediagoblin/mediagoblin/static>
Order allow,deny
Allow from all
</Directory>
<Directory /path/to/mediagoblin/mediagoblin/user_dev/media/public>
Order allow,deny
Allow from all
</Directory>

# Connect to fcgi server
FastCGIExternalServer /var/www/mg.fcgi -host 127.0.0.1:26543
</VirtualHost>
</pre>
Then, you need to make sure mediagoblin is running in fcgi mode:
<pre>cd /path/to/mediagoblin
./lazyserver.sh --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543</pre>

Note: there may be several ways to improve this configuration

If it is too slow and you use the deflate module, you could try to use the following option : SetOutputFilter INFLATE

= Lighttpd =

This configuration example uses mod_fastcgi.
Make a symbolic link from <pre>/etc/lighttpd/conf-available/10-fastcgi.conf</pre> to <pre>/etc/lighttpd/conf-enabled/10-fastcgi.conf</pre>

<pre>
ln -s /etc/lighttpd/conf-available/10-fastcgi.conf /etc/lighttpd/conf-enabled/10-fastcgi.conf
</pre>

then edit your /etc/lighttpd/conf-enabled/10-fastcgi.conf file modifying it as below to serve mediagoblin on the /tube/ path of your webserver.

<pre>
## FastCGI programs have the same functionality as CGI programs,
## but are considerably faster through lower interpreter startup
## time and socketed communication
##
## Documentation: /usr/share/doc/lighttpd-doc/fastcgi.txt.gz
## http://www.lighttpd.net/documentation/fastcgi.html

server.modules += ( "mod_fastcgi" )

## Start an FastCGI server for mediagoblin (lazyserver.sh or other deployments scripts should be up and running)
fastcgi.server = (
# url at which the app should be accessible on the server
"/tube" =>
# mnemonical name of the backend
( "mg-local" => (
"host" => "127.0.0.1",
"port" => 6543,
"docroot" => "/path/to/mediagoblin-git-folder/mediagoblin",
"check-local" => "disable"
)
)

)
</pre>

Then open your /etc/lighttpd/lighttpd.conf and add the configuration to serve directly the static/public folders

<pre>
alias.url += (
"/mgoblin_static/" => "/path/to/mediagoblin-git-folder/mediagoblin/static/",
"/mgoblin_media/" => "/path/to/mediagoblin-git-folder/user_dev/media/public/",
"/theme_static/" => "/path/to/mediagoblin-git-folder/user_dev/theme_static/",
"/plugin_static/" => "/path/to/mediagoblin-git-folder/user_dev/plugin_static/"
)

</pre>

Done!

= OpenShift =

Thers's a blogpost saying how to [http://www.sud0.com/gnu-media-goblin-on-openshift.html install mediagoblin on openshift].

= Juju =

There is a juju [https://juju.ubuntu.com/Charms charm] available for deploying mediagoblin into EC2 or on your local box. [https://juju.ubuntu.com/ juju] is available in Ubuntu 11.10 and later, though it is recommended that you pull it from [https://launchpad.net/~juju/+archive/pkgs the juju PPA], which includes backported packages going back to Ubuntu 11.10. To use the juju charm, install juju and configure either the [https://juju.ubuntu.com/docs/provider-configuration-local.html local provider] or one of the cloud API providers, currently the [https://juju.ubuntu.com/docs/provider-configuration-ec2.html EC2 provider] is the best supported and works not only with Amazon Web Services but also OpenStack clouds. There is also a newer native OpenStack API provider which is known to support HP Cloud.
<pre>
# if you have not bootstrapped
juju bootstrap
mkdir ~/charms
bzr init-repo ~/charms/precise
bzr branch lp:~clint-fewbar/charms/precise/mediagoblin/trunk ~/charms/precise/mediagoblin
juju deploy --repository ~/charms local:mediagoblin
juju expose mediagoblin
</pre>

Currently the charm is volatile, deploying from trunk, and deploys a single-server version of MediaGoblin only. It will eventually relate to the existing juju charms for other supported data stores to allow one to scale out their MediaGoblin instance.

= Init scripts =

== Debian init scripts ==

Joar has some scripts for running celery and mediagoblin separately that are designed for Debian.

https://github.com/joar/mediagoblin-init-scripts

== Arch Linux init scripts ==

[http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts Jeremy Pope has written a nice blogpost] on how to add init scripts to deploy MediaGoblin with both the python paste http server and the celery deployments separated.

If you want a simpler setup and don't want to deploy celery separately, consider either turning CELERY_ALWAYS_EAGER to true in the paste init script described above, or check out [http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/ Chimo's guide].

== Generic, simple init script ==

This is a super stupidly simple init script that was used for mediagoblin.com... note that this has Celery running in always eager mode.

You will need to adjust the paths appropriately. This could probably be better written!

#! /bin/sh

## Stupidly simple mediagoblin init script.
#
# Written in 2012 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

PASTER=/srv/mediagoblin.com/bin/paster
PASTE_CONFIG=/srv/mediagoblin.com/paste.ini
OPTIONS="--pid-file=/tmp/mediagoblin.pid \
--log-file=/srv/mediagoblin.com/paster.log \
--server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543"

CELERY_ALWAYS_EAGER=true su -pc "$PASTER serve $PASTE_CONFIG $1 $OPTIONS" webadmin

= Running on Dreamhost.com =

===Set up your python virtualenv===

dreamhost.com servers come with python 2.4 and 2.5 installed which is too old for mediagoblin. This means you need to compile and install a newer python (don't worry this is really not difficult on dreamhost servers). In order to be able to install python packages for a local user without touching system files, we need to setup a python ''virtualenv''. Fortunately, this is not too tricky either. If your server has python 2.6 or 2.7 installed already, you can skip the [[#Install a local python|python installation stuff]] and you'll only need to [[#Setup virtualenv to install local python packages|install virtualenv]].

====Install a local python====
# Download the latest python: http://python.org/ftp/python/XXX/Python-XXX.tar.bz2
# Unpack with <pre>tar xvjf Python-XXX.tar.bz2</pre>
# ''cd'' into the directory and compile and install python locally:
<blockquote><pre>./configure --prefix=$HOME/local
make
make install</pre></blockquote>
:This will install python (I used 2.7.3) into /home/<USERNAME>/local/bin/python. You might get warnings about some modules not being able to compile. :It was tcl/tk and bzip2 mainly, but things still worked out overall.
:
:You should now be able to invoke <tt>~/local/bin/python</tt> and fall into a shell of your new python (exit with ctrl-d). Congrats, you have now a new python 2.7 that you can use. However, you will need to be able to install additional packages as a user and this is what virtualenv allows.

====Setup virtualenv to install local python packages====
# Download the latest virtualenv: http://pypi.python.org/packages/source/v/virtualenv/XXX
# Install the virtualenv: <blockquote><pre>~/local/bin/python ~/virtualenv-1.8/virtualenv.py $HOME/local/virtenv</pre></blockquote>
:: You will now have: ~/local/virtenv/bin/python
# In ~/.bash_profile add: <blockquote><pre>PATH=~/local/virtenv/bin:~/local/bin:${PATH}</pre></blockquote>
:: so that your local python will be preferred.
:: Log out, log in again and test python" to see which version will be invoked. It should be the new python. Check "which easy_install" to see if the one in local/virtenv would be executed.
::
:: You have now 1) a local ''python'' installation that you can use, and 2) ''easy_install'' will also work with your local user installation. From now on you can e.g. locally install the nose testing framework (''easy_install nose'') and use it (''python -c "import nose"'').

===Install mediagoblin as a site package===

# Check out mediagoblin from git to e.g. ~/mediagoblin
# Install MediaGoblin and all dependencies for MediaGoblin with easy_install.
:* In the mediagoblin directory issue:
:<blockquote><pre>python setup.py</pre></blockquote>
:* You will also need to: easy_install lxml
:* Python-image was trickier to install: <pre>easy_install --find-links http://www.pythonware.com/products/pil/ Imaging</pre>
::Test by leaving the mediagoblin directory and see if you can import it: <pre>python -c "import mediagoblin"</pre> looking for error messages.

===Set up an WSGI environment on Dreamhost===

# Enable the mod_passenger setting for ruby/python in the domains web panel
# ''cd'' into the domain directory (e.g. ~/media.sspaeth.de for me)
# Copy mediagoblin.ini and paste.ini from the ~/mediagoblin directory here
# Create passenger_wsgi.py in your domain directory (e.g. ~/media.sspaeth.de for me):
<blockquote><pre>import sys, os
INTERP = "/home/<username>/local/virtenv/bin/python"
#INTERP is present twice so that the new python interpreter knows the actual executable path
if sys.executable != INTERP: os.execl(INTERP, INTERP, *sys.argv)

from paste.deploy import loadapp
application = loadapp('config:/home/mediagoblin/media.sspaeth.de/paste.ini')

#If in case of errors, all you get are mysterious error 500, you can set debug=true in paste.ini to see stack traces
# Otherwise, add this:
#from paste.exceptions.errormiddleware import ErrorMiddleware
#application = ErrorMiddleware(application, debug=True)</pre></blockquote>
# Set up the database by issueing: <pre>gmg dbupdate</pre>
# (optional but recommended) Serve the static files directly from the webserver.
:In paste.ini in this section: [composite:routing] comment out the static files:
#/mgoblin_static/ = mediagoblin_static
#/theme_static/ = theme_static
:and symlink the mediagobin/mediagoblin/static directoy to public/mgoblin_static
: and mediagoblin/mediagoblin/themes to public/theme_static
:
:so it is displayed directly. This step might be different depending on your web server configuration. There is no reason that static
:files need to go through all the python indirection when they can be served directly by nginx/apache/...
:
:In case you want to delete your git checkout after the installation (you don't need it, since you installed the mediagoblin package to ~/local/virtenv/lib/python2.7/mediagoblin... you should do the symlinking from there.
===Open Issues===
Please fill in the blanks if you find them out:
* How to enable logging to a file or standard html logs? Console output just disappears.
* How to set up things not using sqlite. What's wrong with mySQL?

===Troubleshooting===
* First of all: invoke <pre>python passenger_wsgi.py</pre> directly from the shell. This will tell you if there are import errors etc. If this command exits without any output on the console, you have already achieved a lot.
* In ''paste.ini'' set debug=true. This will show you python backtraces directly in the web page
* You will need to configure an smtp user/passwd/server in mediagoblin.ini, so you actually get the account creation token mailed out. Hint, the relevant settings are: <blockquote><pre>email_debug_mode = false
email_sender_address = postmaster@sspaeth.de
email_smtp_host = SMTP.DOMAIN.TLD
email_smtp_user = USERNAME
email_smtp_pass = WEIRDPASSWORD</pre></blockquote>
* In case you can upload media, but it does not appear. You don't have the celery server running. Add <pre>CELERY_ALWAYS_EAGER = true</pre> to the ''[celery]'' section in mediagoblin.ini

=Miscellaneous Hacks=
==Force translation==
There might some conditions under which you would prefer that MediaGoblin always return the same translation. If you are deploying with nginx and fastcgi, you can force MediaGoblin to return a specific translation regardless of browser preferences by passing a specific HTTP_ACCEPT_LANGUAGE fastcgi parameter in your location block. Example last line in location block:
<nowiki>
fastcgi_param HTTP_ACCEPT_LANGUAGE es; #force spanish translation
</nowiki>

==Create an admin user==

To create an admin user, first create the user you need either via the website or the command
<pre>
./bin/gmg adduser
</pre>

Then to turn it into an admin account use

<pre>
./bin/gmg makeadmin your_username
</pre>

Deployment

2014-03-22T23:11:23Z

Moggers87: Added mod_wsgi example

This page could use a lot of work. For now, a few smaller deployment tips!

See also: http://docs.mediagoblin.org/deploying.html (some of which may belong here)

= uwsgi =

Possibly a cleaner way to do nginx configuration than the official nginx + fastcgi suggestions!

http://qalthos.github.io/blog/FOSS@RIT/deploying-mediagoblin-1-fastcgi-vs-uwsgi.html

= FCGI script =

This works great with the apache FCGID config example in the next section :) in which case you should name it "mg.fcgi".

Before use, make sure you replace '/path/to/mediagoblin/bin/python' with a real path on your server, e.g. '/srv/www/myhomepage.com/mediagoblin/bin/python'. Also replace '/path/to/mediagoblin/paste.ini'.

If you encounter problems, try executing executing the script manually, e.g. <pre>./mg.fcgi</pre>

Script:

<pre>#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

from paste.deploy import loadapp
from flup.server.fcgi import WSGIServer

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
# import os
# os.environ['CELERY_ALWAYS_EAGER'] = 'true'

def launch_fcgi():
ccengine_wsgi_app = loadapp('config:' + CONFIG_PATH)
WSGIServer(ccengine_wsgi_app).run()

if __name__ == '__main__':
launch_fcgi()</pre>

= Apache 2 Config With fcgid =

Note that the libapache2-mod-fcgi in Debian is in the main section. libapache2-mod-fcgid can be used, but requires a slightly different configuration.

<VirtualHost *:80>
Options +ExecCGI

# Accept up to 16MB requests
FcgidMaxRequestLen 16777216

ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

ScriptAlias / /path/to/mediagoblin/mg.fcgi/
</VirtualHost>

= Apache 2 Config With mod_wsgi =

Apache can manage the MediaGoblin application itself with mod_wsgi. It requires slightly more configuration on the Apache side, but cuts out the FastCGI middleware. See their [http://code.google.com/p/modwsgi/wiki/ConfigurationGuidelines configuration guidelines] for advanced configuration.

<VirtualHost *:80>
ServerName mediagoblin.example.org

Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

WSGIScriptAlias / /path/to/mediagoblin/wsgi.py
# Set user and group to whatever Apache runs as
WSGIDaemonProcess gmg user=apache group=apache processes=2 threads=10 \
umask=0007 inactivity-timeout=900 maximum-requests=1000 \
python-path=/path/to/mediagoblin/:/path/to/mediagoblin/lib/python-2.x/site-packages/
WSGIProcessGroup gmg
WSGIApplicationGroup %{GLOBAL}

<Directory "/path/to/mediagoblin/mediagoblin/static/">
Order allow,deny
Allow from all
</Directory>
<Directory "/path/to/mediagoblin/user_dev/media/public">
Order allow,deny
Allow from all
</Directory>
</VirtualHost>

You'll also need to copy this into /path/to/mediagoblin/wsgi.py:

#!/path/to/mediagoblin/bin/python

# Written in 2011 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

from paste.deploy import loadapp

CONFIG_PATH = '/path/to/mediagoblin/paste.ini'

## Uncomment this to run celery in "always eager" mode... ie, you don't have
## to run a separate process, but submissions wait till processing finishes
#import os
#os.environ['CELERY_ALWAYS_EAGER'] = 'true'

application = loadapp('config:' + CONFIG_PATH)

= Apache Config Example =
This configuration example uses mod_fastcgi.

To install and enable mod_fastcgi on a Debian/Ubuntu based system:
<pre># apt-get install libapache2-mod-suexec libapache2-mod-fastcgi
# a2enmod suexec
# a2enmod fastcgi</pre>

Sample configuration:
<pre>
<VirtualHost *:80>
ServerName mediagoblin.yourdomain.tld
ServerAdmin webmaster@yourdoimain.tld
DocumentRoot /var/www/
# Custom log files
CustomLog /var/log/apache2/mediagobling_access.log combined
ErrorLog /var/log/apache2/mediagoblin_error.log

# Serve static and media files via alias
Alias /mgoblin_static/ /path/to/mediagoblin/mediagoblin/static/
Alias /mgoblin_media/ /path/to/mediagoblin/user_dev/media/public/

# Rewrite all URLs to fcgi, except for static and media urls
RewriteEngine On
RewriteRule ^(mgoblin_static|mgoblin_media)($|/) - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^/(.*)$ /mg.fcgi/$1 [QSA,L]

# Allow access to static and media directories
<Directory /path/to/mediagoblin/mediagoblin/static>
Order allow,deny
Allow from all
</Directory>
<Directory /path/to/mediagoblin/mediagoblin/user_dev/media/public>
Order allow,deny
Allow from all
</Directory>

# Connect to fcgi server
FastCGIExternalServer /var/www/mg.fcgi -host 127.0.0.1:26543
</VirtualHost>
</pre>
Then, you need to make sure mediagoblin is running in fcgi mode:
<pre>cd /path/to/mediagoblin
./lazyserver.sh --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543</pre>

Note: there may be several ways to improve this configuration

If it is too slow and you use the deflate module, you could try to use the following option : SetOutputFilter INFLATE

= Lighttpd =

This configuration example uses mod_fastcgi.
Make a symbolic link from <pre>/etc/lighttpd/conf-available/10-fastcgi.conf</pre> to <pre>/etc/lighttpd/conf-enabled/10-fastcgi.conf</pre>

<pre>
ln -s /etc/lighttpd/conf-available/10-fastcgi.conf /etc/lighttpd/conf-enabled/10-fastcgi.conf
</pre>

then edit your /etc/lighttpd/conf-enabled/10-fastcgi.conf file modifying it as below to serve mediagoblin on the /tube/ path of your webserver.

<pre>
## FastCGI programs have the same functionality as CGI programs,
## but are considerably faster through lower interpreter startup
## time and socketed communication
##
## Documentation: /usr/share/doc/lighttpd-doc/fastcgi.txt.gz
## http://www.lighttpd.net/documentation/fastcgi.html

server.modules += ( "mod_fastcgi" )

## Start an FastCGI server for mediagoblin (lazyserver.sh or other deployments scripts should be up and running)
fastcgi.server = (
# url at which the app should be accessible on the server
"/tube" =>
# mnemonical name of the backend
( "mg-local" => (
"host" => "127.0.0.1",
"port" => 6543,
"docroot" => "/path/to/mediagoblin-git-folder/mediagoblin",
"check-local" => "disable"
)
)

)
</pre>

Then open your /etc/lighttpd/lighttpd.conf and add the configuration to serve directly the static/public folders

<pre>
alias.url += (
"/mgoblin_static/" => "/path/to/mediagoblin-git-folder/mediagoblin/static/",
"/mgoblin_media/" => "/path/to/mediagoblin-git-folder/user_dev/media/public/",
"/theme_static/" => "/path/to/mediagoblin-git-folder/user_dev/theme_static/",
"/plugin_static/" => "/path/to/mediagoblin-git-folder/user_dev/plugin_static/"
)

</pre>

Done!

= OpenShift =

Thers's a blogpost saying how to [http://www.sud0.com/gnu-media-goblin-on-openshift.html install mediagoblin on openshift].

= Juju =

There is a juju [https://juju.ubuntu.com/Charms charm] available for deploying mediagoblin into EC2 or on your local box. [https://juju.ubuntu.com/ juju] is available in Ubuntu 11.10 and later, though it is recommended that you pull it from [https://launchpad.net/~juju/+archive/pkgs the juju PPA], which includes backported packages going back to Ubuntu 11.10. To use the juju charm, install juju and configure either the [https://juju.ubuntu.com/docs/provider-configuration-local.html local provider] or one of the cloud API providers, currently the [https://juju.ubuntu.com/docs/provider-configuration-ec2.html EC2 provider] is the best supported and works not only with Amazon Web Services but also OpenStack clouds. There is also a newer native OpenStack API provider which is known to support HP Cloud.
<pre>
# if you have not bootstrapped
juju bootstrap
mkdir ~/charms
bzr init-repo ~/charms/precise
bzr branch lp:~clint-fewbar/charms/precise/mediagoblin/trunk ~/charms/precise/mediagoblin
juju deploy --repository ~/charms local:mediagoblin
juju expose mediagoblin
</pre>

Currently the charm is volatile, deploying from trunk, and deploys a single-server version of MediaGoblin only. It will eventually relate to the existing juju charms for other supported data stores to allow one to scale out their MediaGoblin instance.

= Init scripts =

== Debian init scripts ==

Joar has some scripts for running celery and mediagoblin separately that are designed for Debian.

https://github.com/joar/mediagoblin-init-scripts

== Arch Linux init scripts ==

[http://whird.jpope.org/2012/04/14/mediagoblin-archlinux-rcd-scripts Jeremy Pope has written a nice blogpost] on how to add init scripts to deploy MediaGoblin with both the python paste http server and the celery deployments separated.

If you want a simpler setup and don't want to deploy celery separately, consider either turning CELERY_ALWAYS_EAGER to true in the paste init script described above, or check out [http://chimo.chromic.org/2012/03/01/mediagoblin-init-script-on-archlinux/ Chimo's guide].

== Generic, simple init script ==

This is a super stupidly simple init script that was used for mediagoblin.com... note that this has Celery running in always eager mode.

You will need to adjust the paths appropriately. This could probably be better written!

#! /bin/sh

## Stupidly simple mediagoblin init script.
#
# Written in 2012 by Christopher Allan Webber
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the
# public domain worldwide. This software is distributed without any
# warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

PASTER=/srv/mediagoblin.com/bin/paster
PASTE_CONFIG=/srv/mediagoblin.com/paste.ini
OPTIONS="--pid-file=/tmp/mediagoblin.pid \
--log-file=/srv/mediagoblin.com/paster.log \
--server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543"

CELERY_ALWAYS_EAGER=true su -pc "$PASTER serve $PASTE_CONFIG $1 $OPTIONS" webadmin

= Running on Dreamhost.com =

===Set up your python virtualenv===

dreamhost.com servers come with python 2.4 and 2.5 installed which is too old for mediagoblin. This means you need to compile and install a newer python (don't worry this is really not difficult on dreamhost servers). In order to be able to install python packages for a local user without touching system files, we need to setup a python ''virtualenv''. Fortunately, this is not too tricky either. If your server has python 2.6 or 2.7 installed already, you can skip the [[#Install a local python|python installation stuff]] and you'll only need to [[#Setup virtualenv to install local python packages|install virtualenv]].

====Install a local python====
# Download the latest python: http://python.org/ftp/python/XXX/Python-XXX.tar.bz2
# Unpack with <pre>tar xvjf Python-XXX.tar.bz2</pre>
# ''cd'' into the directory and compile and install python locally:
<blockquote><pre>./configure --prefix=$HOME/local
make
make install</pre></blockquote>
:This will install python (I used 2.7.3) into /home/<USERNAME>/local/bin/python. You might get warnings about some modules not being able to compile. :It was tcl/tk and bzip2 mainly, but things still worked out overall.
:
:You should now be able to invoke <tt>~/local/bin/python</tt> and fall into a shell of your new python (exit with ctrl-d). Congrats, you have now a new python 2.7 that you can use. However, you will need to be able to install additional packages as a user and this is what virtualenv allows.

====Setup virtualenv to install local python packages====
# Download the latest virtualenv: http://pypi.python.org/packages/source/v/virtualenv/XXX
# Install the virtualenv: <blockquote><pre>~/local/bin/python ~/virtualenv-1.8/virtualenv.py $HOME/local/virtenv</pre></blockquote>
:: You will now have: ~/local/virtenv/bin/python
# In ~/.bash_profile add: <blockquote><pre>PATH=~/local/virtenv/bin:~/local/bin:${PATH}</pre></blockquote>
:: so that your local python will be preferred.
:: Log out, log in again and test python" to see which version will be invoked. It should be the new python. Check "which easy_install" to see if the one in local/virtenv would be executed.
::
:: You have now 1) a local ''python'' installation that you can use, and 2) ''easy_install'' will also work with your local user installation. From now on you can e.g. locally install the nose testing framework (''easy_install nose'') and use it (''python -c "import nose"'').

===Install mediagoblin as a site package===

# Check out mediagoblin from git to e.g. ~/mediagoblin
# Install MediaGoblin and all dependencies for MediaGoblin with easy_install.
:* In the mediagoblin directory issue:
:<blockquote><pre>python setup.py</pre></blockquote>
:* You will also need to: easy_install lxml
:* Python-image was trickier to install: <pre>easy_install --find-links http://www.pythonware.com/products/pil/ Imaging</pre>
::Test by leaving the mediagoblin directory and see if you can import it: <pre>python -c "import mediagoblin"</pre> looking for error messages.

===Set up an WSGI environment on Dreamhost===

# Enable the mod_passenger setting for ruby/python in the domains web panel
# ''cd'' into the domain directory (e.g. ~/media.sspaeth.de for me)
# Copy mediagoblin.ini and paste.ini from the ~/mediagoblin directory here
# Create passenger_wsgi.py in your domain directory (e.g. ~/media.sspaeth.de for me):
<blockquote><pre>import sys, os
INTERP = "/home/<username>/local/virtenv/bin/python"
#INTERP is present twice so that the new python interpreter knows the actual executable path
if sys.executable != INTERP: os.execl(INTERP, INTERP, *sys.argv)

from paste.deploy import loadapp
application = loadapp('config:/home/mediagoblin/media.sspaeth.de/paste.ini')

#If in case of errors, all you get are mysterious error 500, you can set debug=true in paste.ini to see stack traces
# Otherwise, add this:
#from paste.exceptions.errormiddleware import ErrorMiddleware
#application = ErrorMiddleware(application, debug=True)</pre></blockquote>
# Set up the database by issueing: <pre>gmg dbupdate</pre>
# (optional but recommended) Serve the static files directly from the webserver.
:In paste.ini in this section: [composite:routing] comment out the static files:
#/mgoblin_static/ = mediagoblin_static
#/theme_static/ = theme_static
:and symlink the mediagobin/mediagoblin/static directoy to public/mgoblin_static
: and mediagoblin/mediagoblin/themes to public/theme_static
:
:so it is displayed directly. This step might be different depending on your web server configuration. There is no reason that static
:files need to go through all the python indirection when they can be served directly by nginx/apache/...
:
:In case you want to delete your git checkout after the installation (you don't need it, since you installed the mediagoblin package to ~/local/virtenv/lib/python2.7/mediagoblin... you should do the symlinking from there.
===Open Issues===
Please fill in the blanks if you find them out:
* How to enable logging to a file or standard html logs? Console output just disappears.
* How to set up things not using sqlite. What's wrong with mySQL?

===Troubleshooting===
* First of all: invoke <pre>python passenger_wsgi.py</pre> directly from the shell. This will tell you if there are import errors etc. If this command exits without any output on the console, you have already achieved a lot.
* In ''paste.ini'' set debug=true. This will show you python backtraces directly in the web page
* You will need to configure an smtp user/passwd/server in mediagoblin.ini, so you actually get the account creation token mailed out. Hint, the relevant settings are: <blockquote><pre>email_debug_mode = false
email_sender_address = postmaster@sspaeth.de
email_smtp_host = SMTP.DOMAIN.TLD
email_smtp_user = USERNAME
email_smtp_pass = WEIRDPASSWORD</pre></blockquote>
* In case you can upload media, but it does not appear. You don't have the celery server running. Add <pre>CELERY_ALWAYS_EAGER = true</pre> to the ''[celery]'' section in mediagoblin.ini

=Miscellaneous Hacks=
==Force translation==
There might some conditions under which you would prefer that MediaGoblin always return the same translation. If you are deploying with nginx and fastcgi, you can force MediaGoblin to return a specific translation regardless of browser preferences by passing a specific HTTP_ACCEPT_LANGUAGE fastcgi parameter in your location block. Example last line in location block:
<nowiki>
fastcgi_param HTTP_ACCEPT_LANGUAGE es; #force spanish translation
</nowiki>