HackingHowto
Note: This page is marked for rewriting. We know it could be better! If you get confused, please ask for help on IRC.
Hacking HOWTO
So you want to hack on GNU MediaGoblin?
First thing to do is check out the web site where we list all the project infrastructure including:
- the IRC channel
- the mailing list
- the issue tracker
Additionally, we have information on how to get involved, who to talk to, what needs to be worked on, and other things besides!
Second thing to do is take a look at :ref:`codebase-chapter` where we've started documenting how GNU MediaGoblin is built and how to add new things.
Third you'll need to get the requirements.
Fourth, you'll need to build a development environment. We use buildout, but if you want to use virtualenv, there's a set of mediocre not-very-supported steps in the evil wiki.
Getting requirements
First, you need to have the following installed before you can build an environment for hacking on GNU MediaGoblin:
- Python 2.6 or 2.7 - http://www.python.org/ (You'll need Python as well as the dev files for building modules.)
- python-lxml - http://lxml.de/
- git - http://git-scm.com/
- MongoDB - http://www.mongodb.org/
- Python Imaging Library (PIL) - http://www.pythonware.com/products/pil/
If you're running Debian GNU/Linux or a Debian-derived distribution such as Debian, Mint, or Ubuntu 10.10+, running the following should install these requirements:
sudo apt-get install mongodb git-core python python-dev python-lxml python-imaging
On Fedora:
yum install mongodb-server python-paste-deploy python-paste-script git-core python python-devel python-lxml python-imaging
On Mac OS X Lion:
Download the Newest Python.
Git is already installed.
Install MongoDB from these instructions: http://www.mongodb.org/display/DOCS/Quickstart+OS+X
Python-lxml: http://muffinresearch.co.uk/archives/2009/03/05/install-lxml-on-osx/ with sudo
Python Imaging Library (PIL): http://code.google.com/appengine/docs/python/images/installingPIL.html#mac
Libjpeg & Libpng: http://ethan.tira-thompson.org/Mac_OS_X_Ports.html Combo Installer
You can help:
If you have instructions for other GNU/Linux distributions or Mac OS X to set up requirements, let us know!
How to set up and maintain an environment for hacking with buildout
Requirements
No additional requirements.
Create a development environment
After installing the requirements, follow these steps:
- Clone the repository: git clone git://gitorious.org/mediagoblin/mediagoblin.git
- Bootstrap and run buildout: cd mediagoblin and then one of the following:
- python bootstrap.py && ./bin/buildout, OR
- python bootstrap.py --distribute && ./bin/buildout
Why would you want one over the other? If the first doesn't work, then try the second.
That's it! Using this method, buildout should create a user_dev directory, in which certain things will be stored (media, beaker session stuff, etc). You can change this, but for development purposes this default should be fine.
If you have problems, please let us know!
Updating for dependency changes
While hacking on GNU MediaGoblin over time, you'll eventually have to update your development environment because the dependencies have changed.
To do that, run:
./bin/buildout && ./bin/gmg migrate.
Updating for code changes
You don't need to do anything---code changes are automatically available.
Deleting your buildout
At some point, you may want to delete your buildout. Perhaps it's to start over. Perhaps it's to test building development environments with buildout.
To do this, run:
rm -rf bin develop-eggs eggs mediagoblin.egg-info parts user_dev.
Running Mongo Database
Startup the Database: ./mongodb-xxxxxxx/bin/mongod.
Running the server
If you want to get things running quickly and without hassle, just run:
./lazyserver.sh.
This will start up a python server where you can begin playing with
mediagoblin. It will also run celery in "always eager" mode so you
don't have to start a separate process for it.
By default, the instance is not sending out confirmation mails. Instead they are redirected to the standard output (the console) of lazyserver.sh.
You can change this behavior setting email_debug_mode to false in mediagoblin.ini
This is fine in development, but if you want to actually run celery
separately for testing (or deployment purposes), you'll want to run
the server independently:
./bin/paster serve paste.ini --reload
Running celeryd
If you aren't using ./lazyserver.sh or otherwise aren't running celery in always eager mode, you'll need to do this if you want your media to process and actually show up. It's probably a good idea in development to have the web server (above) running in one terminal and celeryd in another window.
Run:
CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd.
Running the test suite
Run:
./runtests.sh
Running a shell
If you want a shell with your database pre-setup and an instantiated application ready and at your fingertips....
Run:
./bin/gmg shell
Troubleshooting
pymongo.errors.AutoReconnect: could not find master/primary
If you see this:
pymongo.errors.AutoReconnect: could not find master/primary
then make sure mongodb is installed and running.
If it's installed, check the mongodb log. On my machine, that's /var/log/mongodb/mongodb.log. If you see something like:
old lock file: /var/lib/mongodb/mongod.lock. probably means...
in that case you might have had an unclean shutdown. Try:
sudo mongod --repair
If that didn't work, just delete the lock file and relaunch mongodb.
Anyway, then start the mongodb server in whatever way is appropriate for your distro / OS.
Wiping your user data
You can completely wipe all data from the instance by doing:
gmg wipealldata
Note:
Unless you're doing development and working on and testing creating a new instance, you will probably never have to do this. Will plans to do this work and thus he documented it.
Quickstart for Django programmers
We're not using Django, but the codebase is very Django-like in its structure.
- routing.py is like urls.py in Django
- models.py has mongokit ORM definitions
- views.py is where the views go
We're using MongoDB. Basically, instead of a relational database with tables, you have a big JSON structure which acts a lot like a Python dict.
YouCanHelp
If there are other things that you think would help orient someone new to GNU MediaGoblin but coming from Django, let us know!
Bite-sized bugs to start with
Now you should visit our latest list of bite-sized issues because squishing bugs is messy fun. If you're interested in other things to work on, or need help getting started on a bug, let us know on the mailing list or on the IRC channel.