Difference between revisions of "HackingHowto"

From GNU MediaGoblin Wiki
Jump to navigation Jump to search
(Switched hackinghowto over to SQLAlchemy descriptions)
m (tweaking vertical white space)
Line 30: Line 30:
 
mediagoblin in your own virtualenv setup... the necessary adjustments
 
mediagoblin in your own virtualenv setup... the necessary adjustments
 
should be obvious.
 
should be obvious.
  +
   
 
== Getting requirements ==
 
== Getting requirements ==
Line 43: Line 44:
 
* virtualenv - http://www.virtualenv.org/
 
* virtualenv - http://www.virtualenv.org/
 
* Python GStreamer Bindings - http://gstreamer.freedesktop.org/modules/gst-python.html
 
* Python GStreamer Bindings - http://gstreamer.freedesktop.org/modules/gst-python.html
  +
   
 
=== GNU/Linux ===
 
=== GNU/Linux ===
Line 53: Line 55:
   
 
{{Cmd|sudo apt-get install git-core python python-dev python-lxml python-imaging python-virtualenv python-gst0.10 libjpeg8-dev}}
 
{{Cmd|sudo apt-get install git-core python python-dev python-lxml python-imaging python-virtualenv python-gst0.10 libjpeg8-dev}}
  +
   
 
==== Fedora / RedHat(?) ====
 
==== Fedora / RedHat(?) ====
Line 59: Line 62:
   
 
{{Cmd|yum install python-paste-deploy python-paste-script git-core python python-devel python-lxml python-imaging python-virtualenv gstreamer-python}}
 
{{Cmd|yum install python-paste-deploy python-paste-script git-core python python-devel python-lxml python-imaging python-virtualenv gstreamer-python}}
  +
   
 
=== Mac OS X ===
 
=== Mac OS X ===
Line 80: Line 84:
 
If you have instructions for other GNU/Linux distributions, Windows, or Mac OS X to set
 
If you have instructions for other GNU/Linux distributions, Windows, or Mac OS X to set
 
up requirements, let us know!
 
up requirements, let us know!
  +
   
 
== How to set up and maintain an environment for hacking with virtualenv ==
 
== How to set up and maintain an environment for hacking with virtualenv ==
Line 86: Line 91:
   
 
No additional requirements.
 
No additional requirements.
  +
   
 
'''Create a development environment'''
 
'''Create a development environment'''
Line 104: Line 110:
   
 
If you have problems, please [http://mediagoblin.org/join/ let us know]!
 
If you have problems, please [http://mediagoblin.org/join/ let us know]!
  +
  +
  +
== Updating an existing environment ==
   
 
'''Updating for dependency changes'''
 
'''Updating for dependency changes'''
Line 119: Line 128:
   
 
{{Cmd|git pull -u}}
 
{{Cmd|git pull -u}}
  +
   
 
== Running the server ==
 
== Running the server ==
Line 142: Line 152:
   
 
{{Cmd|./bin/paster serve paste.ini --reload}}
 
{{Cmd|./bin/paster serve paste.ini --reload}}
  +
   
 
== Running celeryd ==
 
== Running celeryd ==
Line 154: Line 165:
   
 
{{Cmd|<nowiki>CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd</nowiki>}}
 
{{Cmd|<nowiki>CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd</nowiki>}}
  +
   
 
== Running the test suite ==
 
== Running the test suite ==
Line 181: Line 193:
   
 
... note that we don't "officially" support pillow at this time.
 
... note that we don't "officially" support pillow at this time.
  +
   
 
== Wiping your user data ==
 
== Wiping your user data ==

Revision as of 20:34, 20 April 2012

Hacking HOWTO

So you want to hack on GNU MediaGoblin?

First thing to do is check out the web site where we list all the project infrastructure including:

  • the IRC channel
  • the mailing list
  • the issue tracker

Additionally, we have information on how to get involved, who to talk to, what needs to be worked on, and other things besides!

Second thing to do is take a look at codebase chapter where we've started documenting how GNU MediaGoblin is built and how to add new things. If you're planning on contributing in python, you should be aware of PEP-8, the official Python style guide, which we follow.

Third you'll need to get the requirements.

Fourth, you'll need to build a development environment. We use an in-package checkout of virtualenv. This isn't the convenional way to install virtualenv (normally you don't install virtualenv inside the package itself) but we've found that it's significantly easier for newcomers who aren't already familiar with virtualenv. If you *are* already familiar with virtualenv, feel free to just install mediagoblin in your own virtualenv setup... the necessary adjustments should be obvious.


Getting requirements

First, you need to have the following installed before you can build an environment for hacking on GNU MediaGoblin:


GNU/Linux

Debian and derivatives

If you're running Debian GNU/Linux or a Debian-derived distribution such as Debian, Mint, or Ubuntu 10.10+, running the following should install these requirements:

sudo apt-get install git-core python python-dev python-lxml python-imaging python-virtualenv python-gst0.10 libjpeg8-dev


Fedora / RedHat(?)

On Fedora:

yum install python-paste-deploy python-paste-script git-core python python-devel python-lxml python-imaging python-virtualenv gstreamer-python


Mac OS X

On Mac OS X Lion:

Download the Newest Python.

Git is already installed.

Python-lxml: http://muffinresearch.co.uk/archives/2009/03/05/install-lxml-on-osx/ with sudo

Python Imaging Library (PIL): http://code.google.com/appengine/docs/python/images/installingPIL.html#mac

Libjpeg & Libpng: http://ethan.tira-thompson.org/Mac_OS_X_Ports.html Combo Installer


You can help:

If you have instructions for other GNU/Linux distributions, Windows, or Mac OS X to set up requirements, let us know!


How to set up and maintain an environment for hacking with virtualenv

Requirements

No additional requirements.


Create a development environment

After installing the requirements, follow these steps:

  • Clone the repository: git clone git://gitorious.org/mediagoblin/mediagoblin.git
  • Change directories to your new checkout: cd mediagoblin
  • Set up the in-package virtualenv:
 (virtualenv --system-site-packages . || virtualenv .) && ./bin/python setup.py develop
  • Init the database:
 ./bin/gmg dbupdate

That's it!

(If you have troubles in the remaining steps, consider try installing virtualenv with one of the flags --setuptools, --distribute or possibly --no-site-packages. Additionally, if your system has python3.X as the default, you might need to do virtualenv --python=python2.7 or --python=python2.6)

If you have problems, please let us know!


Updating an existing environment

Updating for dependency changes

While hacking on GNU MediaGoblin over time, you'll eventually have to update your development environment because the dependencies have changed.

To do that, run:

./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate


Updating for code changes

git pull -u


Running the server

If you want to get things running quickly and without hassle, just run:

./lazyserver.sh


This will start up a python server where you can begin playing with mediagoblin, listening on 127.0.0.1:6543. It will also run celery in "always eager" mode so you don't have to start a separate process for it.

By default, the instance is not sending out confirmation mails. Instead they are redirected to the standard output (the console) of lazyserver.sh.

You can change this behavior setting email_debug_mode to false in mediagoblin.ini


This is fine in development, but if you want to actually run celery separately for testing (or deployment purposes), you'll want to run the server independently:

./bin/paster serve paste.ini --reload


Running celeryd

If you aren't using ./lazyserver.sh or otherwise aren't running celery in always eager mode, you'll need to do this if you want your media to process and actually show up. It's probably a good idea in development to have the web server (above) running in one terminal and celeryd in another window.

Run:

CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd


Running the test suite

Run:

./runtests.sh


Running a shell

If you want a shell with your database pre-setup and an instantiated application ready and at your fingertips....

Run:

./bin/gmg shell


Troubleshooting

Problems with PIL

Some users have claimed they've had trouble with PIL working nicely on their system. You could try installing Pillow:

./bin/pip install pillow

... note that we don't "officially" support pillow at this time.


Wiping your user data

You can completely wipe all data from the instance by doing:

gmg wipealldata

Note:

Unless you're doing development and working on and testing creating a new instance, you will probably never have to do this. Will plans to do this work and thus he documented it.


Quickstart for Django programmers

We're not using Django, but the codebase is very Django-like in its structure.

  • routing.py is like urls.py in Django
  • models.py has SQLAlchemy ORM definitions
  • views.py is where the views go

We're using SQLAlchemy, which is semi-similar to the Django ORM, but not really because you can get a lot more fine-grained. The SQLAlchemy ORM tutorial is a great place to start.


YouCanHelp

If there are other things that you think would help orient someone new to GNU MediaGoblin but coming from Django, let us know!


Bite-sized bugs to start with

Now you should visit our latest list of bite-sized issues because squishing bugs is messy fun. If you're interested in other things to work on, or need help getting started on a bug, let us know on the mailing list or on the IRC channel.