HackingHowto: Difference between revisions

From GNU MediaGoblin Wiki
Jump to navigation Jump to search
No edit summary
m (Fixing broken link for MediaGoblin Join page)
 
(103 intermediate revisions by 28 users not shown)
Line 4: Line 4:
== So you want to hack on GNU MediaGoblin? ==
== So you want to hack on GNU MediaGoblin? ==


First thing to do is check out the [http://mediagoblin.org/join/ web site] where we list all the project
First thing to do is check out the [https://mediagoblin.org/pages/join.html web site] where we list all the project
infrastructure including:
infrastructure including:


Line 14: Line 14:
to, what needs to be worked on, and other things besides!
to, what needs to be worked on, and other things besides!


Second thing to do is take a look at :ref:`codebase-chapter` where
Second thing to do is take a look at [http://docs.mediagoblin.org/devel/codebase.html codebase chapter] where
we've started documenting how GNU MediaGoblin is built and how to add
we've started documenting how GNU MediaGoblin is built and how to add
new things. If you're planning on contributing in python, you should be aware
new things.
of [http://www.python.org/dev/peps/pep-0008/ PEP-8], the official Python style guide,
which we follow.


Third you'll need to get the requirements.
Third you'll need to get the requirements.


Fourth, you'll need to build a development environment. We use buildout,
Fourth, you'll need to build a development environment. We use an
in-package checkout of virtualenv. This isn't the convenional way to
but if you want to use virtualenv, there's a set of mediocre not-very-supported
install virtualenv (normally you don't install virtualenv inside the
steps in the [https://gitorious.org/mediagoblin/pages/Home evil wiki].
package itself) but we've found that it's significantly easier for
newcomers who aren't already familiar with virtualenv. If you *are*
already familiar with virtualenv, feel free to just install
mediagoblin in your own virtualenv setup... the necessary adjustments
should be obvious.


== Python 2 vs Python 3 ==

GNU MediaGoblin was originally written for Python 2, and much of the documentation on the wiki reflects that. However, since release v0.9.0, GNU MediaGoblin has attempted to provide support for Python 3. Unfortunately, not everything is working for Python 3 environments yet. Below is a non-exhaustive list of things that might not work if you set up GNU MediaGoblin a Python 3 environment.

=== Python 3 Issues ===

* fcgi - none of the flup packages ([https://pypi.python.org/pypi/flup flup], [https://pypi.python.org/pypi/flup-py3 flup-py3], [https://pypi.python.org/pypi/flup6 flup6]) appear to have proper support for Python 3.5.
* audio file spectrographs - [https://pypi.python.org/pypi/scikits.audiolab scikits.audiolab] [https://issues.mediagoblin.org/ticket/5467 appears to not work] in Python 3.
* Rackspace storage - the [https://pypi.python.org/pypi/python-cloudfiles cloudfiles] plugin may not work with Python 3.


== Getting requirements ==
== Getting requirements ==
Line 30: Line 46:
an environment for hacking on GNU MediaGoblin:
an environment for hacking on GNU MediaGoblin:


* Python 2.6 or 2.7 - http://www.python.org/ (You'll need Python as well as the dev files for building modules.)
* Python 2.7 - http://www.python.org/ (You'll need Python as well as the dev files for building modules.)
* python-lxml - http://lxml.de/
* python-lxml - http://lxml.de/
* git - http://git-scm.com/
* git - http://git-scm.com/
* MongoDB - http://www.mongodb.org/
* SQLAlchemy 0.7.0 or higher - http://www.sqlalchemy.org/
* Pillow - http://python-pillow.org/
* virtualenv - http://www.virtualenv.org/
* Python GStreamer Bindings - http://gstreamer.freedesktop.org/modules/gst-python.html
* Node.js
* RabbitMQ server

=== GNU/Linux ===

==== Debian and derivatives ====


If you're running Debian GNU/Linux or a Debian-derived distribution
If you're running Debian GNU/Linux or a Debian-derived distribution
Line 39: Line 64:
requirements:
requirements:


sudo apt-get install mongodb git-core python python-dev \
{{Cmd|sudo apt-get install git-core python python-dev python-lxml python-imaging python-virtualenv python-gst-1.0 libjpeg8-dev autoconf nodejs npm nodejs-legacy rabbitmq-server}}

python-lxml
==== Debian GNU/Linux jessie ====

{{Cmd|sudo apt-get install -y git-core python python-dev python-lxml python-imaging python-virtualenv libjpeg-dev autoconf nodejs npm nodejs-legacy python-gst-1.0 gstreamer1.0-plugins-base gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-libav python-numpy python-scipy libsndfile1-dev libasound2-dev libgstreamer-plugins-base1.0-dev rabbitmq-server}}

==== Fedora / RedHat(?) / CentOS ====

===== On Fedora: =====

{{Cmd|yum install python-paste-deploy python-paste-script git-core python python-devel python-lxml python-imaging python-virtualenv gstreamer-python nodejs}}

===== On CentOS7: =====

'''epel-release''' is the prerequisite for most of the below listed packages

{{Cmd|sudo yum install epel-release}}

{{Cmd|sudo yum update}}

{{Cmd|sudo yum install python-paste-deploy python-paste-script \ git-core python python-devel python-lxml python-imaging \ python-virtualenv npm automake nginx gcc}}

for video to work correctly (as per [https://mediagoblin.readthedocs.io/en/stable/siteadmin/media-types.html?highlight=video#video Enabling Media-Types Video]) you may choose to install AWEL 3rd party repo which provides with the latest gstreamer bindings and more:

* [http://awel.domblogger.net/7/media/x86_64/repoview/awel-media-release.html AWEL Repo],[https://media.librelamp.com/gst AWEL Gstreamer]

once installed:

{{Cmd|sudo yum install gstreamer1 gstreamer1-libav gstreamer1-plugins-base gstreamer1-plugins-base-tools gstreamer1-plugins-good gstreamer1-plugins-ugly gstreamer1-plugins-bad gstreamer1-python ffmpeg }}

==== openSUSE ====

This instructions might be incomplete.
Try on openSUSE 13.2 or 13.1:

{{Cmd|zypper install python-devel python-gstreamer-0_10 git-core python python-lxml python-Pillow python-virtualenv npm}}

==== ArchLinux / Parabola ====

The following command should work (<del>not tested on a new ArchLinux / Parabola install</del>. tested, it works):

{{Cmd|pacman -S git python2 python2-lxml python2-pillow python2-virtualenv gstreamer0.10-python}}

=== Mac OS X ===

==== Mac OS X Lion ====

Download the Newest Python.

Git is already installed.

* Note for Pillow and lxml, you can: pip install Pillow lxml

Python-lxml: http://muffinresearch.co.uk/archives/2009/03/05/install-lxml-on-osx/ with sudo

Pillow: http://pillow.readthedocs.org/en/3.0.x/installation.html#os-x-installation

Libjpeg & Libpng: http://ethan.tira-thompson.com/Mac_OS_X_Ports.html Combo Installer

==== Mac OS X Snow Leopard ====

# You will probably want to install MacPorts this will give you access to many free software packages in the same manner to apt-get and yum: https://www.macports.org/install.php
# Ensure you install Git and the command line tools: https://help.github.com/articles/set-up-git#platform-mac
# Once both of those are installed type this in your terminal and enter your password when prompted for it {{Cmd|sudo port install python27 py27-lxml py27-sqlalchemy py27-Pillow py27-virtualenv py27-gst-python py27-pastescript}}

=== Microsoft Windows ===

''Thanks wctype!''


==== Getting requirements ====
On Fedora:


* Python 2.7 - [http://www.python.org/download/ Download] <!-- http://www.python.org/ftp/python/2.7.3/python-2.7.3.msi -->
yum install mongodb-server python-paste-deploy python-paste-script \
* git - [https://github.com/msysgit/git/downloads Download] <!-- https://github.com/downloads/msysgit/git/Git-1.7.11-preview20120620.exe -->
git-core python python-devel python-lxml
* python-lxml - [http://pypi.python.org/pypi/lxml/2.3.5#downloads Tarball] [http://www.lfd.uci.edu/~gohlke/pythonlibs/#pil Binaries] <!-- http://pypi.python.org/packages/source/l/lxml/lxml-2.3.5.tar.gz, http://www.lfd.uci.edu/~gohlke/pythonlibs/z8sp4uqu/lxml-2.3.5.win32-py2.7.exe -->
* Pillow - [http://pillow.readthedocs.org/en/3.0.x/installation.html#windows-installation Download] <!-- http://effbot.org/downloads/PIL-1.1.7.win32-py2.7.exe] -->
* virtualenv - [http://pypi.python.org/pypi/virtualenvwrapper-win/1.0.8#downloads Download] <!-- http://pypi.python.org/packages/source/v/virtualenvwrapper-win/virtualenvwrapper-win-1.0.8.zip -->
* OSSBuild project provides reasonably up-to-date binaries of GStreamer - [https://code.google.com/p/ossbuild/downloads/list Download] <!-- http://ossbuild.googlecode.com/files/GStreamer-WinBuilds-GPL-x86.msi -->
* py-bcrypt - [https://bitbucket.org/alexandrul/py-bcrypt/downloads/ Download] <!-- https://bitbucket.org/alexandrul/py-bcrypt/downloads/py-bcrypt-0.2.post1.win32-py2.7.exe -->
* Node JS


----


'''You can help:'''
'''You can help:'''


If you have instructions for other GNU/Linux distributions to set
If you have instructions for other GNU/Linux distributions, Windows, or Mac OS X to set
up requirements, let us know!
up requirements, [https://mediagoblin.org/pages/join.html let us know]!


== How to set up and maintain an environment for hacking with buildout ==
== How to set up and maintain an environment for hacking with virtualenv ==


'''Requirements'''
'''Requirements'''
Line 64: Line 162:
After installing the requirements, follow these steps:
After installing the requirements, follow these steps:


# Clone the repository: <tt>git clone git://gitorious.org/mediagoblin/mediagoblin.git</tt>
* Clone the repository: {{Cmd|git clone <nowiki>git://git.savannah.gnu.org/mediagoblin.git</nowiki>}}
* Make the environment: {{Cmd|./bootstrap.sh && ./configure && make}}
# Bootstrap and run buildout: <tt>cd mediagoblin</tt> and then <tt>python bootstrap.py && ./bin/buildout</tt>
* If you want to use audio or video, this is the time to add the appropriate plugins to <tt>mediagoblin.ini</tt>
* Init the database:
{{Cmd|./bin/gmg dbupdate}}


That's it!


If you want to make sure things are working, consider running the test suite:
That's it! Using this method, buildout should create a <tt>user_dev</tt>
{{Cmd|./runtests.sh}}
directory, in which certain things will be stored (media, beaker
session stuff, etc). You can change this, but for development
purposes this default should be fine.


(If you have troubles in the remaining steps, consider try installing
virtualenv with one of the flags --setuptools, --distribute or possibly --no-site-packages. Additionally, if your system has python3.X as the default, you might need to do virtualenv --python=python2.7 or --python=python2.6)

If you have problems, please [https://mediagoblin.org/pages/join.html let us know]!

== Updating an existing environment ==


'''Updating for dependency changes'''
'''Updating for dependency changes'''
Line 78: Line 184:
While hacking on GNU MediaGoblin over time, you'll eventually have to
While hacking on GNU MediaGoblin over time, you'll eventually have to
update your development environment because the dependencies have
update your development environment because the dependencies have
changed. To do that, run:
changed.


To do that, run:
./bin/buildout && ./bin/gmg migrate

{{Cmd|./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate}}




'''Updating for code changes'''
'''Updating for code changes'''


{{Cmd|git pull -u}}
You don't need to do anything---code changes are automatically available.
{{Cmd|git submodule update}}




'''NOTE:''' MediaGoblin used to be hosted on gitorious.org, but that is moving to read-only mode, so we're now on Savannah. If you have an old checkout, please update it by running the following:
'''Deleting your buildout'''

At some point, you may want to delete your buildout. Perhaps it's to
start over. Perhaps it's to test building development environments
with buildout.

To do this, do:

rm -rf bin develop-eggs eggs mediagoblin.egg-info parts user_dev


{{Cmd|git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git}}


== Running the server ==
== Running the server ==
Line 104: Line 206:
run:
run:


./lazyserver.sh
{{Cmd|./lazyserver.sh}}



This will start up a python server where you can begin playing with
This will start up a python server where you can begin playing with
mediagoblin. It will also run celery in "always eager" mode so you
mediagoblin, listening on 127.0.0.1:6543. It will also run celery in "always eager" mode so you
don't have to start a separate process for it.
don't have to start a separate process for it.

By default, the instance is not sending out confirmation mails. Instead they are redirected to the standard output (the console) of lazyserver.sh.

You can change this behavior setting <code>email_debug_mode</code> to <code>false</code> in mediagoblin.ini



This is fine in development, but if you want to actually run celery
This is fine in development, but if you want to actually run celery
Line 114: Line 222:
the server independently:
the server independently:


./bin/paster serve paste.ini --reload
{{Cmd|./bin/paster serve paste.ini --reload}}




Line 127: Line 235:
Run:
Run:


CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd
{{Cmd|<nowiki>CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd</nowiki>}}




== Running the test suite ==
== Running the test suite ==

See also [[UnitTests]] and [[Manual Functional Testing]].


Run:
Run:


./runtests.sh
{{Cmd|./runtests.sh}}



== Running a shell ==
== Running a shell ==


If you want a shell with your database pre-setup and an instantiated
If you want a shell with your database pre-setup and an instantiated
application ready and at your fingertips....
application ready and at your fingertips ......


Run:
Run:


./bin/gmg shell
{{Cmd|./bin/gmg shell}}



== Troubleshooting ==
== Troubleshooting ==


If you see a python error, search for the full line in [https://issues.mediagoblin.org/report trac] to find issues that mention the same . If it has not been fixed, you can update corresponding bug reports or open a new one.
=== pymongo.errors.AutoReconnect: could not find master/primary ===

If you see this:

pymongo.errors.AutoReconnect: could not find master/primary

then make sure mongodb is installed and running.

If it's installed, check the mongodb log. On my machine, that's
<tt>/var/log/mongodb/mongodb.log</tt>. If you see something like:

old lock file: /var/lib/mongodb/mongod.lock. probably means...

in that case you might have had an unclean shutdown. Try:

sudo mongod --repair

If that didn't work, just delete the lock file and relaunch mongodb.

Anyway, then start the mongodb server in whatever way is appropriate
for your distro / OS.


=== pkg_resources.DistributionNotFound: distribute ===

If you get this while running buildout:

pkg_resources.DistributionNotFound: distribute

Try this commmand instead:

python bootstrap.py --distribute && ./bin/buildout


For developers it is a special pleasure to test and apply user submitted patches. Here is a good minimal [https://issues.mediagoblin.org/timeline?from=2016-02-28T17%3A16%3A39-05%3A00&precision=second git workflow] as example.


== Wiping your user data ==
== Wiping your user data ==
Line 187: Line 265:
You can completely wipe all data from the instance by doing:
You can completely wipe all data from the instance by doing:


{{Cmd|rm -rf mediagoblin.db kombu.db celery.db user_dev; ./bin/gmg dbupdate}}
gmg wipealldata



'''Note:'''
'''Note:'''


Unless you're doing development and working on and testing creating
Unless you're doing development and working on and testing creating
a new instance, you will probably never have to do this. Will
a new instance, you will probably never have to do this.
plans to do this work and thus he documented it.



== Quickstart for Django programmers ==
== Quickstart for Django programmers ==
Line 203: Line 278:


* <tt>routing.py</tt> is like <tt>urls.py</tt> in Django
* <tt>routing.py</tt> is like <tt>urls.py</tt> in Django
* <tt>models.py</tt> has mongokit ORM definitions
* <tt>models.py</tt> has SQLAlchemy ORM definitions
* <tt>views.py</tt> is where the views go
* <tt>views.py</tt> is where the views go


We're using MongoDB. Basically, instead of a relational database with
We're using SQLAlchemy, which is semi-similar to the Django ORM, but
not really because you can get a lot more fine-grained. The
tables, you have a big JSON structure which acts a lot like a Python
[http://docs.sqlalchemy.org/en/latest/orm/tutorial.html SQLAlchemy ORM tutorial] is a great place to start.
dict.




Line 216: Line 291:
new to GNU MediaGoblin but coming from Django, let us know!
new to GNU MediaGoblin but coming from Django, let us know!


== Showing off your work with PageKite ==


If you're doing development with MediaGoblin, it's sometimes helpful to show off your work to gather feedback from other contributors. A number of the MediaGoblin developers use something called [http://pagekite.net PageKite], which is a fellow free software web service which makes temporarily showing off work on your machine easy. There's a [http://pagekite.net/wiki/Howto/UsePageKiteWithMediaGoblin/ tutorial on how to use PageKite and MediaGoblin together] available on the PageKite wiki.
== Bite-sized bugs to start with ==

'''May 3rd, 2011''': We don't have a list of bite-sized bugs, yet, but
this is important to us. If you're interested in things to work on,
let us know on [http://mediagoblin.org/join/ the mailing list] or
on the [http://mediagoblin.org/join/ IRC channel].


If you are doing a lot of MediaGoblin development, the PageKite people have graciously offered us a good amount of bandwidth at no cost in an effort to help out fellow free software projects. If you've been making significant contributions, PM Chris Webber on freenode (who is paroneayea there) and ask if you can be added to our group plan.


== Tips for people new to coding ==
== Bite-sized bugs to start with ==

=== Learning Python ===

GNU MediaGoblin is written using a programming language called [http://python.org/ Python].

There are two different incompatible iterations of Python which I'll
refer to as Python 2 and Python 3. GNU MediaGoblin is written in
Python 2 and requires Python 2.6 or 2.7. At some point, we might
switch to Python 3, but that's a future thing.

You can learn how to code in Python 2 from several excellent books
that are freely available on the Internet:

* [http://learnpythonthehardway.org/ Learn Python the Hard Way]
* [http://diveintopython.org/ Dive Into Python]
* [http://www.greenteapress.com/thinkpython/ Python for Software Design]
* [http://www.swaroopch.com/notes/Python A Byte of Python]

These are all excellent texts.

'''You Can Help'''

If you know of other good quality Python tutorials and Python
tutorial videos, let us know!


=== Learning Libraries GNU MediaGoblin uses ===

GNU MediaGoblin uses a variety of libraries in order to do what it
does. These libraries are listed in the :ref:`codebase-chapter`
along with links to the project Web sites and documentation for the
libraries.

There are a variety of Python-related conferences every year that have
sessions covering many aspects of these libraries. You can find them
at [http://python.mirocommunity.org Python Miro Community] -- This is a shameless plug; Will Kahn-Greene runs Python Miro Community.

If you have questions or need help, find us on the mailing list and on
IRC.

=== Learning git ===

git is an interesting and very powerful tool. Like all powerful
tools, it has a learning curve.

If you're new to git, we highly recommend the following resources for
getting the hang of it:

* [http://learn.github.com/p/intro.html Learn Git] --- the GitHub intro to git
* [http://progit.org/book/ Pro Git] --- fantastic book
* [http://gitcasts.com/ Git casts] --- screencast covering git usage
* [http://gitref.org/ Git Reference] --- Git reference that makes it easier to get the hang of git if you're coming from other version control systems

There's also a git mission at [http://openhatch.org/ OpenHatch].

=== Learning other utilities ===

The [http://openhatch.org/ OpenHatch] site has a series of
[http://openhatch.org/missions/ training missions] which are
designed to help you learn how to use these tools.


Now you should visit our latest list of [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bite-sized issues] because squishing bugs is messy fun. If you're interested in other things to work on, or need help getting started on a bug, let us know on [https://mediagoblin.org/pages/join.html the mailing list] or on the [https://mediagoblin.org/pages/join.html IRC channel].
If you're new to tar, diff, patch and git, we highly recommend you sign
up with OpenHatch and do the missions.

Latest revision as of 13:33, 1 November 2018

Hacking HOWTO

So you want to hack on GNU MediaGoblin?

First thing to do is check out the web site where we list all the project infrastructure including:

  • the IRC channel
  • the mailing list
  • the issue tracker

Additionally, we have information on how to get involved, who to talk to, what needs to be worked on, and other things besides!

Second thing to do is take a look at codebase chapter where we've started documenting how GNU MediaGoblin is built and how to add new things. If you're planning on contributing in python, you should be aware of PEP-8, the official Python style guide, which we follow.

Third you'll need to get the requirements.

Fourth, you'll need to build a development environment. We use an in-package checkout of virtualenv. This isn't the convenional way to install virtualenv (normally you don't install virtualenv inside the package itself) but we've found that it's significantly easier for newcomers who aren't already familiar with virtualenv. If you *are* already familiar with virtualenv, feel free to just install mediagoblin in your own virtualenv setup... the necessary adjustments should be obvious.

Python 2 vs Python 3

GNU MediaGoblin was originally written for Python 2, and much of the documentation on the wiki reflects that. However, since release v0.9.0, GNU MediaGoblin has attempted to provide support for Python 3. Unfortunately, not everything is working for Python 3 environments yet. Below is a non-exhaustive list of things that might not work if you set up GNU MediaGoblin a Python 3 environment.

Python 3 Issues

Getting requirements

First, you need to have the following installed before you can build an environment for hacking on GNU MediaGoblin:

GNU/Linux

Debian and derivatives

If you're running Debian GNU/Linux or a Debian-derived distribution such as Debian, Mint, or Ubuntu 10.10+, running the following should install these requirements:

sudo apt-get install git-core python python-dev python-lxml python-imaging python-virtualenv python-gst-1.0 libjpeg8-dev autoconf nodejs npm nodejs-legacy rabbitmq-server

Debian GNU/Linux jessie

sudo apt-get install -y git-core python python-dev python-lxml python-imaging python-virtualenv libjpeg-dev autoconf nodejs npm nodejs-legacy python-gst-1.0 gstreamer1.0-plugins-base gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-libav python-numpy python-scipy libsndfile1-dev libasound2-dev libgstreamer-plugins-base1.0-dev rabbitmq-server

Fedora / RedHat(?) / CentOS

On Fedora:

yum install python-paste-deploy python-paste-script git-core python python-devel python-lxml python-imaging python-virtualenv gstreamer-python nodejs

On CentOS7:

epel-release is the prerequisite for most of the below listed packages

sudo yum install epel-release

sudo yum update

sudo yum install python-paste-deploy python-paste-script \ git-core python python-devel python-lxml python-imaging \ python-virtualenv npm automake nginx gcc

for video to work correctly (as per Enabling Media-Types Video) you may choose to install AWEL 3rd party repo which provides with the latest gstreamer bindings and more:

once installed:

sudo yum install gstreamer1 gstreamer1-libav gstreamer1-plugins-base gstreamer1-plugins-base-tools gstreamer1-plugins-good gstreamer1-plugins-ugly gstreamer1-plugins-bad gstreamer1-python ffmpeg

openSUSE

This instructions might be incomplete. Try on openSUSE 13.2 or 13.1:

zypper install python-devel python-gstreamer-0_10 git-core python python-lxml python-Pillow python-virtualenv npm

ArchLinux / Parabola

The following command should work (not tested on a new ArchLinux / Parabola install. tested, it works):

pacman -S git python2 python2-lxml python2-pillow python2-virtualenv gstreamer0.10-python

Mac OS X

Mac OS X Lion

Download the Newest Python.

Git is already installed.

  • Note for Pillow and lxml, you can: pip install Pillow lxml

Python-lxml: http://muffinresearch.co.uk/archives/2009/03/05/install-lxml-on-osx/ with sudo

Pillow: http://pillow.readthedocs.org/en/3.0.x/installation.html#os-x-installation

Libjpeg & Libpng: http://ethan.tira-thompson.com/Mac_OS_X_Ports.html Combo Installer

Mac OS X Snow Leopard

  1. You will probably want to install MacPorts this will give you access to many free software packages in the same manner to apt-get and yum: https://www.macports.org/install.php
  2. Ensure you install Git and the command line tools: https://help.github.com/articles/set-up-git#platform-mac
  3. Once both of those are installed type this in your terminal and enter your password when prompted for it sudo port install python27 py27-lxml py27-sqlalchemy py27-Pillow py27-virtualenv py27-gst-python py27-pastescript

Microsoft Windows

Thanks wctype!

Getting requirements

You can help:

If you have instructions for other GNU/Linux distributions, Windows, or Mac OS X to set up requirements, let us know!

How to set up and maintain an environment for hacking with virtualenv

Requirements

No additional requirements.


Create a development environment

After installing the requirements, follow these steps:

  • Clone the repository: git clone git://git.savannah.gnu.org/mediagoblin.git
  • Make the environment: ./bootstrap.sh && ./configure && make
  • If you want to use audio or video, this is the time to add the appropriate plugins to mediagoblin.ini
  • Init the database:
 ./bin/gmg dbupdate

That's it!

If you want to make sure things are working, consider running the test suite: 

 ./runtests.sh

(If you have troubles in the remaining steps, consider try installing virtualenv with one of the flags --setuptools, --distribute or possibly --no-site-packages. Additionally, if your system has python3.X as the default, you might need to do virtualenv --python=python2.7 or --python=python2.6)

If you have problems, please let us know!

Updating an existing environment

Updating for dependency changes

While hacking on GNU MediaGoblin over time, you'll eventually have to update your development environment because the dependencies have changed.

To do that, run:

./bin/python setup.py develop --upgrade && ./bin/gmg dbupdate


Updating for code changes

git pull -u git submodule update


NOTE: MediaGoblin used to be hosted on gitorious.org, but that is moving to read-only mode, so we're now on Savannah. If you have an old checkout, please update it by running the following:

git remote set-url origin git://git.savannah.gnu.org/mediagoblin.git

Running the server

If you want to get things running quickly and without hassle, just run:

./lazyserver.sh


This will start up a python server where you can begin playing with mediagoblin, listening on 127.0.0.1:6543. It will also run celery in "always eager" mode so you don't have to start a separate process for it.

By default, the instance is not sending out confirmation mails. Instead they are redirected to the standard output (the console) of lazyserver.sh.

You can change this behavior setting email_debug_mode to false in mediagoblin.ini


This is fine in development, but if you want to actually run celery separately for testing (or deployment purposes), you'll want to run the server independently:

./bin/paster serve paste.ini --reload


Running celeryd

If you aren't using ./lazyserver.sh or otherwise aren't running celery in always eager mode, you'll need to do this if you want your media to process and actually show up. It's probably a good idea in development to have the web server (above) running in one terminal and celeryd in another window.

Run:

CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd


Running the test suite

See also UnitTests and Manual Functional Testing.

Run:

./runtests.sh

Running a shell

If you want a shell with your database pre-setup and an instantiated application ready and at your fingertips ......

Run:

./bin/gmg shell

Troubleshooting

If you see a python error, search for the full line in trac to find issues that mention the same . If it has not been fixed, you can update corresponding bug reports or open a new one.

For developers it is a special pleasure to test and apply user submitted patches. Here is a good minimal git workflow as example.

Wiping your user data

You can completely wipe all data from the instance by doing:

rm -rf mediagoblin.db kombu.db celery.db user_dev; ./bin/gmg dbupdate

Note:

Unless you're doing development and working on and testing creating a new instance, you will probably never have to do this.

Quickstart for Django programmers

We're not using Django, but the codebase is very Django-like in its structure.

  • routing.py is like urls.py in Django
  • models.py has SQLAlchemy ORM definitions
  • views.py is where the views go

We're using SQLAlchemy, which is semi-similar to the Django ORM, but not really because you can get a lot more fine-grained. The SQLAlchemy ORM tutorial is a great place to start.


YouCanHelp

If there are other things that you think would help orient someone new to GNU MediaGoblin but coming from Django, let us know!

Showing off your work with PageKite

If you're doing development with MediaGoblin, it's sometimes helpful to show off your work to gather feedback from other contributors. A number of the MediaGoblin developers use something called PageKite, which is a fellow free software web service which makes temporarily showing off work on your machine easy. There's a tutorial on how to use PageKite and MediaGoblin together available on the PageKite wiki.

If you are doing a lot of MediaGoblin development, the PageKite people have graciously offered us a good amount of bandwidth at no cost in an effort to help out fellow free software projects. If you've been making significant contributions, PM Chris Webber on freenode (who is paroneayea there) and ask if you can be added to our group plan.

Bite-sized bugs to start with

Now you should visit our latest list of bite-sized issues because squishing bugs is messy fun. If you're interested in other things to work on, or need help getting started on a bug, let us know on the mailing list or on the IRC channel.

Retrieved from "https://wiki.mediagoblin.org/index.php?title=HackingHowto&oldid=1916"