GNU MediaGoblin Wiki - User contributions [en]

HackingHowto

2011-08-21T00:28:47Z

Goblin-jim: Moved the 'how-to-program' info to its own page

{{Needswork}}

= Hacking HOWTO =

== So you want to hack on GNU MediaGoblin? ==

First thing to do is check out the [http://mediagoblin.org/join/ web site] where we list all the project
infrastructure including:

* the IRC channel
* the mailing list
* the issue tracker

Additionally, we have information on how to get involved, who to talk
to, what needs to be worked on, and other things besides!

Second thing to do is take a look at :ref:`codebase-chapter` where
we've started documenting how GNU MediaGoblin is built and how to add
new things.

Third you'll need to get the requirements.

Fourth, you'll need to build a development environment. We use buildout,
but if you want to use virtualenv, there's a set of mediocre not-very-supported
steps in the [https://gitorious.org/mediagoblin/pages/Home evil wiki].

== Getting requirements ==

First, you need to have the following installed before you can build
an environment for hacking on GNU MediaGoblin:

* Python 2.6 or 2.7 - http://www.python.org/ (You'll need Python as well as the dev files for building modules.)
* python-lxml - http://lxml.de/
* git - http://git-scm.com/
* MongoDB - http://www.mongodb.org/
* Python Imaging Library (PIL) - http://www.pythonware.com/products/pil/

If you're running Debian GNU/Linux or a Debian-derived distribution
such as Debian, Mint, or [http://bugs.foocorp.net/issues/478 Ubuntu 10.10+], running the following should install these
requirements:

{{Cmd|sudo apt-get install mongodb git-core python python-dev python-lxml python-imaging}}

On Fedora:

{{Cmd|yum install mongodb-server python-paste-deploy python-paste-script git-core python python-devel python-lxml python-imaging}}

----

On Mac OS X Lion:

Download the Newest Python.

Git is already installed.

Install MongoDB from these instructions: http://www.mongodb.org/display/DOCS/Quickstart+OS+X

Python-lxml: http://muffinresearch.co.uk/archives/2009/03/05/install-lxml-on-osx/ with sudo

Python Imaging Library (PIL): http://code.google.com/appengine/docs/python/images/installingPIL.html#mac

Libjpeg & Libpng: http://ethan.tira-thompson.org/Mac_OS_X_Ports.html Combo Installer

----

'''You can help:'''

If you have instructions for other GNU/Linux distributions or Mac OS X to set
up requirements, let us know!

== How to set up and maintain an environment for hacking with buildout ==

'''Requirements'''

No additional requirements.

'''Create a development environment'''

After installing the requirements, follow these steps:

# Clone the repository: {{Cmd|git clone <nowiki>git://gitorious.org/mediagoblin/mediagoblin.git</nowiki>}}
# Bootstrap and run buildout: {{Cmd|cd mediagoblin}} and then one of the following:
#* {{Cmd|python bootstrap.py && ./bin/buildout}}, '''OR'''
#* {{Cmd|python bootstrap.py --distribute && ./bin/buildout}}

Why would you want one over the other? If the first doesn't work, then try the second.

That's it! Using this method, buildout should create a <tt>user_dev</tt>
directory, in which certain things will be stored (media, beaker
session stuff, etc). You can change this, but for development
purposes this default should be fine.

If you have problems, please [http://mediagoblin.org/join/ let us know]!

'''Updating for dependency changes'''

While hacking on GNU MediaGoblin over time, you'll eventually have to
update your development environment because the dependencies have
changed.

To do that, run:

{{Cmd|./bin/buildout && ./bin/gmg migrate}}.

'''Updating for code changes'''

You don't need to do anything---code changes are automatically available.

'''Deleting your buildout'''

At some point, you may want to delete your buildout. Perhaps it's to
start over. Perhaps it's to test building development environments
with buildout.

To do this, run:

{{Cmd|rm -rf bin develop-eggs eggs mediagoblin.egg-info parts user_dev}}.

== Running Mongo Database ==

Startup the Database: {{Cmd|./mongodb-xxxxxxx/bin/mongod}}.

== Running the server ==

If you want to get things running quickly and without hassle, just
run:

{{Cmd|./lazyserver.sh}}.

This will start up a python server where you can begin playing with
mediagoblin. It will also run celery in "always eager" mode so you
don't have to start a separate process for it.

This is fine in development, but if you want to actually run celery
separately for testing (or deployment purposes), you'll want to run
the server independently:

{{Cmd|./bin/paster serve paste.ini --reload}}

== Running celeryd ==

If you aren't using <tt>./lazyserver.sh</tt> or otherwise aren't running celery
in always eager mode, you'll need to do this if you want your media to
process and actually show up. It's probably a good idea in
development to have the web server (above) running in one terminal and
celeryd in another window.

Run:

{{Cmd|<nowiki>CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery ./bin/celeryd</nowiki>}}.

== Running the test suite ==

Run:

{{Cmd|./runtests.sh}}

== Running a shell ==

If you want a shell with your database pre-setup and an instantiated
application ready and at your fingertips....

Run:

{{Cmd|./bin/gmg shell}}

== Troubleshooting ==

=== pymongo.errors.AutoReconnect: could not find master/primary ===

If you see this:

pymongo.errors.AutoReconnect: could not find master/primary

then make sure mongodb is installed and running.

If it's installed, check the mongodb log. On my machine, that's
<tt>/var/log/mongodb/mongodb.log</tt>. If you see something like:

old lock file: /var/lib/mongodb/mongod.lock. probably means...

in that case you might have had an unclean shutdown. Try:

{{Cmd|sudo mongod --repair}}

If that didn't work, just delete the lock file and relaunch mongodb.

Anyway, then start the mongodb server in whatever way is appropriate
for your distro / OS.

== Wiping your user data ==

You can completely wipe all data from the instance by doing:

{{Cmd|gmg wipealldata}}

'''Note:'''

Unless you're doing development and working on and testing creating
a new instance, you will probably never have to do this. Will
plans to do this work and thus he documented it.

== Quickstart for Django programmers ==

We're not using Django, but the codebase is very Django-like in its
structure.

* <tt>routing.py</tt> is like <tt>urls.py</tt> in Django
* <tt>models.py</tt> has mongokit ORM definitions
* <tt>views.py</tt> is where the views go

We're using MongoDB. Basically, instead of a relational database with
tables, you have a big JSON structure which acts a lot like a Python
dict.

'''YouCanHelp'''

If there are other things that you think would help orient someone
new to GNU MediaGoblin but coming from Django, let us know!

== Bite-sized bugs to start with ==

'''May 3rd, 2011''': We don't have a list of bite-sized bugs, yet, but
this is important to us. If you're interested in things to work on,
let us know on [http://mediagoblin.org/join/ the mailing list] or
on the [http://mediagoblin.org/join/ IRC channel].

Main Page

2011-08-21T00:27:42Z

Goblin-jim: Continued restructuring - created a separate 'beginners corner' page.

GNU MediaGoblin wiki of bizarre and unusual things

In this wiki you will find beasts the likes of which you have never laid eyes on before! Expand your imagination, ladies and gentlemen, and examine the errata in this corpus of knowledge. May it transform you and let you see the world in a new light forevermore.

= Want to try out GNU MediaGoblin? =

* If you want to run your own server to test the best information is currently in the [[HackingHowto]]
* There's info on a server you can test on our [[User Experience]] page

= Want to hack on GNU MediaGoblin? =
This section provides the magical incantations to help you start hacking.

== Hacking and Deploying MediaGoblin ==
* [[HackingHowto|Hacking]] - Get your computer set up to hack on GNU MediaGoblin.
** [[BeginnersCorner|Beginner's Corner]] - Resources for those who are new to Python, MongoDB, or Git.
** We'll be adding some distribution/OS specific hacking guides as separate pages soon.
* [[Configure_MediaGoblin|Configuration]] - Learn about MediaGoblin configuration files and file options.
* [[Virtual_Machine_Hosting|Deployment]] - Deploy your own publicly available MediaGoblin server using [http://aws.amazon.com/free/?utm_source=adwords&utm_medium=cpc&utm_campaign=CPC_Google_AWS_ec2&utm_content=TextV01_PP_V01_EC2&trk=CPC_Google_AWS_ec2 Amazon's free EC2 tier].

== Testing and Release Guidelines ==
* [[UnitTests|Unit Tests]] - all about the unit tests
* [[User Experience]] - user experience testing. Includes link to an instance you can try!
* [[ReleaseProcess|Release Process]] - covers the release process

= Documentation and Translations =
* [[ManualStandards]] - covers the standards for writing the user manual
* [[Translations]] - How to translate stuff or update the translations

= Inner workings of the secret sanctum (Contacting the team) =
* [http://mediagoblin.org/irclogs/ IRC meeting logs]
* [[IRCBot]] - covers our irc bot

== Technical Considerations ==
* [[DesignDecisions]] - covers design decisions (FIXME - this needs to be split up)
* [[Storage]] - How MediaGoblin's internal storage system works.
* [[Processing]] - What happens after you submit your image/video/etc? Processing! More about that.
* [[Scaling Down]] - Minimizing MediaGoblin's resource requirements
* [https://gitorious.org/mediagoblin/mediagoblin/blobs/master/extlib/README External Library Policy] - covers use of external libraries

= Project Miscellany =
* [http://python.mirocommunity.org/video/4337/gnu-mediagoblin-for-a-federate A video with Chris Webber talking about MediaGoblin] at [http://chipy.org ChiPy]
* [[User:Cwebber/braindumps]] - Chris Webber's braindumps (you can help refactoring these into real sections of the site!)

BeginnersCorner

2011-08-21T00:25:56Z

Goblin-jim: Broke off the "learning resources" links from the rest of the hackinghowto.

The MediaGoblin project is welcoming of new contributors, even those who are new to programming. The following resources can help you learn enough to start hacking on MediaGoblin.

== Tips for people new to coding ==

=== Learning Python ===

GNU MediaGoblin is written using a programming language called [http://python.org/ Python].

There are two different incompatible iterations of Python which I'll
refer to as Python 2 and Python 3. GNU MediaGoblin is written in
Python 2 and requires Python 2.6 or 2.7. At some point, we might
switch to Python 3, but that's a future thing.

You can learn how to code in Python 2 from several excellent books
that are freely available on the Internet:

* [http://learnpythonthehardway.org/ Learn Python the Hard Way]
* [http://diveintopython.org/ Dive Into Python]
* [http://www.greenteapress.com/thinkpython/ Python for Software Design]
* [http://www.swaroopch.com/notes/Python A Byte of Python]

These are all excellent texts.

'''You Can Help'''

If you know of other good quality Python tutorials and Python
tutorial videos, let us know!

=== Learning Libraries GNU MediaGoblin uses ===

GNU MediaGoblin uses a variety of libraries in order to do what it
does. These libraries are listed in the :ref:`codebase-chapter`
along with links to the project Web sites and documentation for the
libraries.

There are a variety of Python-related conferences every year that have
sessions covering many aspects of these libraries. You can find them
at [http://python.mirocommunity.org Python Miro Community] -- This is a shameless plug; Will Kahn-Greene runs Python Miro Community.

If you have questions or need help, [http://mediagoblin.org/join/ let us know]!

=== Learning git ===

git is an interesting and very powerful tool. Like all powerful
tools, it has a learning curve.

If you're new to git, we highly recommend the following resources for
getting the hang of it:

* [http://learn.github.com/p/intro.html Learn Git] --- the GitHub intro to git
* [http://progit.org/book/ Pro Git] --- fantastic book
* [http://gitcasts.com/ Git casts] --- screencast covering git usage
* [http://gitref.org/ Git Reference] --- Git reference that makes it easier to get the hang of git if you're coming from other version control systems

There's also a git mission at [http://openhatch.org/ OpenHatch].

=== Learning other utilities ===

The [http://openhatch.org/ OpenHatch] site has a series of
[http://openhatch.org/missions/ training missions] which are
designed to help you learn how to use these tools.

If you're new to tar, diff, patch and git, we highly recommend you sign
up with OpenHatch and do the missions.

Main Page

2011-08-16T11:49:01Z

Goblin-jim:

GNU MediaGoblin wiki of bizarre and unusual things

In this wiki you will find beasts the likes of which you have never laid eyes on before! Expand your imagination, ladies and gentlemen, and examine the errata in this corpus of knowledge. May it transform you and let you see the world in a new light forevermore.

= Want to try out GNU MediaGoblin? =

* If you want to run your own server to test the best information is currently in the [[HackingHowto]]
* There's info on a server you can test on our [[User Experience]] page

= Want to hack on GNU MediaGoblin? =
This section provides the magical incantations to help you start hacking.

== Hacking and Deploying MediaGoblin ==
* [[HackingHowto|Hacking]] - Get your computer set up to hack on GNU MediaGoblin.
** We'll be adding some distribution/OS specific hacking guides as separate pages soon.
* [[Configure_MediaGoblin|Configuration]] - Learn how to configure MediaGoblin for all your local needs.
* [[Virtual_Machine_Hosting|Deployment]] - Deploy your own publicly available MediaGoblin server using [http://aws.amazon.com/free/?utm_source=adwords&utm_medium=cpc&utm_campaign=CPC_Google_AWS_ec2&utm_content=TextV01_PP_V01_EC2&trk=CPC_Google_AWS_ec2 Amazon's free EC2 tier].

== Testing and Release Guidelines ==
* [[UnitTests]] - all about the unit tests
* [[User Experience]] - user experience testing. Includes link to an instance you can try!
* [[ReleaseProcess]] - covers the release process

= Documentation and Translations =
* [[ManualStandards]] - covers the standards for writing the user manual
* [[Translations]] - How to translate stuff or update the translations

= Inner workings of the secret sanctum (Contacting the team) =
* [http://mediagoblin.org/irclogs/ IRC meeting logs]
* [[IRCBot]] - covers our irc bot

== Technical Considerations ==
* [[DesignDecisions]] - covers design decisions (FIXME - this needs to be split up)
* [[Storage]] - How MediaGoblin's internal storage system works.
* [[Processing]] - What happens after you submit your image/video/etc? Processing! More about that.
* [[Scaling Down]] - Minimizing MediaGoblin's resource requirements
* [https://gitorious.org/mediagoblin/mediagoblin/blobs/master/extlib/README External Library Policy] - covers use of external libraries

= Project Miscellany =
* [http://python.mirocommunity.org/video/4337/gnu-mediagoblin-for-a-federate A video with Chris Webber talking about MediaGoblin] at [http://chipy.org ChiPy]
* [[User:Cwebber/braindumps]] - Chris Webber's braindumps (you can help refactoring these into real sections of the site!)

Main Page

2011-08-16T02:27:14Z

Goblin-jim:

GNU MediaGoblin wiki of bizarre and unusual things

In this wiki you will find beasts the likes of which you have never laid eyes on before! Expand your imagination, ladies and gentlemen, and examine the errata in this corpus of knowledge. May it transform you and let you see the world in a new light forevermore.

= Want to try out GNU MediaGoblin? =

* If you want to run your own server to test the best information is currently in the [[HackingHowto]]
* There's info on a server you can test on our [[User Experience]] page

= Want to hack on GNU MediaGoblin? =
Sure, we all do. This section provides the magical incantations to help you get started.

== Hacking and Deploying MediaGoblin ==
* [[HackingHowto|Hacking]] - Get your computer set up to hack on GNU MediaGoblin.
** We'll be adding some distribution/OS specific hacking guides as separate pages soon.
* [[Configure_MediaGoblin|Configuration]] - Learn how to configure MediaGoblin for all your local needs.
* [[Virtual_Machine_Hosting|Deployment]] - Deploy your own publicly available MediaGoblin server using [http://aws.amazon.com/free/?utm_source=adwords&utm_medium=cpc&utm_campaign=CPC_Google_AWS_ec2&utm_content=TextV01_PP_V01_EC2&trk=CPC_Google_AWS_ec2 Amazon's free EC2 tier].

== Testing and Release Guidelines ==
* [[UnitTests]] - all about the unit tests
* [[User Experience]] - user experience testing. Includes link to an instance you can try!
* [[ReleaseProcess]] - covers the release process

= Documentation and Translations =
* [[ManualStandards]] - covers the standards for writing the user manual
* [[Translations]] - How to translate stuff or update the translations

= Inner workings of the secret sanctum (Contacting the team) =
* [http://mediagoblin.org/irclogs/ IRC meeting logs]
* [[IRCBot]] - covers our irc bot

== Technical Considerations ==
* [[DesignDecisions]] - covers design decisions (FIXME - this needs to be split up)
* [[Storage]] - How MediaGoblin's internal storage system works.
* [[Processing]] - What happens after you submit your image/video/etc? Processing! More about that.
* [[Scaling Down]] - Minimizing MediaGoblin's resource requirements
* [https://gitorious.org/mediagoblin/mediagoblin/blobs/master/extlib/README External Library Policy] - covers use of external libraries

= Project Miscellany =
* [http://python.mirocommunity.org/video/4337/gnu-mediagoblin-for-a-federate A video with Chris Webber talking about MediaGoblin] at [http://chipy.org ChiPy]
* [[User:Cwebber/braindumps]] - Chris Webber's braindumps (you can help refactoring these into real sections of the site!)

Main Page

2011-08-16T02:25:06Z

Goblin-jim: Revised content for parallel structure like a boss.

GNU MediaGoblin wiki of bizarre and unusual things

In this wiki you will find beasts the likes of which you have never laid eyes on before! Expand your imagination, ladies and gentlemen, and examine the errata in this corpus of knowledge. May it transform you and let you see the world in a new light forevermore.

= Want to try out GNU MediaGoblin? =

* If you want to run your own server to test the best information is currently in the [[HackingHowto]]
* There's info on a server you can test on our [[User Experience]] page

= Want to hack on GNU MediaGoblin? =
Sure, we all do. What magical incantations are needed to get you started, though? That info is here.

== Hacking and Deploying MediaGoblin ==
* [[HackingHowto|Hacking]] - Get your computer set up to hack on GNU MediaGoblin.
** We'll be adding some distribution/OS specific hacking guides as separate pages soon.
* [[Configure_MediaGoblin|Configuration]] - Learn how to configure MediaGoblin for all your local needs.
* [[Virtual_Machine_Hosting|Deployment]] - Deploy your own publicly available MediaGoblin server using [http://aws.amazon.com/free/?utm_source=adwords&utm_medium=cpc&utm_campaign=CPC_Google_AWS_ec2&utm_content=TextV01_PP_V01_EC2&trk=CPC_Google_AWS_ec2 Amazon's free EC2 tier].

== Testing and Release Guidelines ==
* [[UnitTests]] - all about the unit tests
* [[User Experience]] - user experience testing. Includes link to an instance you can try!
* [[ReleaseProcess]] - covers the release process

= Documentation and Translations =
* [[ManualStandards]] - covers the standards for writing the user manual
* [[Translations]] - How to translate stuff or update the translations

= Inner workings of the secret sanctum (Contacting the team) =
* [http://mediagoblin.org/irclogs/ IRC meeting logs]
* [[IRCBot]] - covers our irc bot

== Technical Considerations ==
* [[DesignDecisions]] - covers design decisions (FIXME - this needs to be split up)
* [[Storage]] - How MediaGoblin's internal storage system works.
* [[Processing]] - What happens after you submit your image/video/etc? Processing! More about that.
* [[Scaling Down]] - Minimizing MediaGoblin's resource requirements
* [https://gitorious.org/mediagoblin/mediagoblin/blobs/master/extlib/README External Library Policy] - covers use of external libraries

= Project Miscellany =
* [http://python.mirocommunity.org/video/4337/gnu-mediagoblin-for-a-federate A video with Chris Webber talking about MediaGoblin] at [http://chipy.org ChiPy]
* [[User:Cwebber/braindumps]] - Chris Webber's braindumps (you can help refactoring these into real sections of the site!)

Main Page

2011-08-13T02:55:14Z

Goblin-jim: minor edits to wiki organization

GNU MediaGoblin wiki of bizarre and unusual things

In this wiki you will find beasts the likes of which you have never laid eyes on before! Expand your imagination, ladies and gentlemen, and examine the errata in this corpus of knowledge. May it transform you and let you see the world in a new light forevermore.

= Want to hack on GNU MediaGoblin? =
== Hacking and Deploying MediaGoblin ==
* [[HackingHowto]] - generally covers getting set up for hacking on GNU MediaGoblin
** FIXME - add distribution/OS specific hacking guides as separate pages here
* [[Configure MediaGoblin]] - how to configure MediaGoblin for all your local needs.
* Deployment - How to run your own publicly available MediaGoblin server
** [[Virtual Machine Hosting]] - using [http://aws.amazon.com/free/?utm_source=adwords&utm_medium=cpc&utm_campaign=CPC_Google_AWS_ec2&utm_content=TextV01_PP_V01_EC2&trk=CPC_Google_AWS_ec2 Amazon's free EC2 tier]

== Testing and Release Guidelines ==
* [[UnitTests]] - all about the unit tests
* [[ReleaseProcess]] - covers the release process

= Documentation and Translations =
* [[ManualStandards]] - covers the standards for writing the user manual
* [[Translations]] - How to translate stuff or update the translations

= Inner workings of the secret sanctum (Contacting the team) =
* [http://mediagoblin.org/irclogs/ IRC meeting logs]
* [[IRCBot]] - covers our irc bot

== Technical Considerations ==
* [[DesignDecisions]] - covers design decisions (FIXME - this needs to be split up)
* [[Scaling Down]] - Minimizing MediaGoblin's resource requirements
* [https://gitorious.org/mediagoblin/mediagoblin/blobs/master/extlib/README External Library Policy] - covers use of external libraries

= Project Miscellany =
* [http://python.mirocommunity.org/video/4337/gnu-mediagoblin-for-a-federate A video with Chris Webber talking about MediaGoblin] at [http://chipy.org ChiPy]
* [[User:Cwebber/braindumps]] - Chris Webber's braindumps (you can help refactoring these into real sections of the site!)

Main Page

2011-08-13T02:50:17Z

Goblin-jim: initial draft of the wiki restructuring

GNU MediaGoblin wiki of bizarre and unusual things

In this wiki you will find beasts the likes of which you have never laid eyes on before! Expand your imagination, ladies and gentlemen, and examine the errata in this corpus of knowledge. May it transform you and let you see the world in a new light forevermore.

= Want to hack on GNU MediaGoblin? =

* [[HackingHowto]] - generally covers getting set up for hacking on GNU MediaGoblin
** FIXME - add distribution/OS specific hacking guides as separate pages here
* [[Configure MediaGoblin]] - how to configure MediaGoblin for all your local needs.
* Deployment - How to run your own publicly available MediaGoblin server
** [[Virtual Machine Hosting]] - using [http://aws.amazon.com/free/?utm_source=adwords&utm_medium=cpc&utm_campaign=CPC_Google_AWS_ec2&utm_content=TextV01_PP_V01_EC2&trk=CPC_Google_AWS_ec2 Amazon's free EC2 tier]

== Testing and Release Guidelines ==
* [[UnitTests]] - all about the unit tests
* [[ReleaseProcess]] - covers the release process

== Technical Considerations ==
* [[DesignDecisions]] - covers design decisions (FIXME - this needs to be split up)
* [[Scaling Down]] - Minimizing MediaGoblin's resource requirements
* [https://gitorious.org/mediagoblin/mediagoblin/blobs/master/extlib/README External Library Policy] - covers use of external libraries

= Documentation and Translations =
* [[ManualStandards]] - covers the standards for writing the user manual
* [[Translations]] - How to translate stuff or update the translations

= Inner workings of the secret sanctum (Contacting the team) =
* [http://mediagoblin.org/irclogs/ IRC meeting logs]
* [[IRCBot]] - covers our irc bot

= Project Miscellany =
* [http://python.mirocommunity.org/video/4337/gnu-mediagoblin-for-a-federate A video with Chris Webber talking about MediaGoblin] at [http://chipy.org ChiPy]
* [[User:Cwebber/braindumps]] - Chris Webber's braindumps (you can help refactoring these into real sections of the site!)