https://wiki.mediagoblin.org/api.php?action=feedcontributions&user=Unhammer&feedformat=atomGNU MediaGoblin Wiki - User contributions [en]2024-03-29T04:59:24ZUser contributionsMediaWiki 1.39.5https://wiki.mediagoblin.org/index.php?title=PluginsTips&diff=1474PluginsTips2014-01-26T11:24:42Z<p>Unhammer: /* Translating / Localising plugins */</p>
<hr />
<div>==Quickstart==<br />
In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get started with plugin creation. The simplest way to get up and running is to <br />
# Follow [[HackingHowto#How_to_set_up_and_maintain_an_environment_for_hacking_with_virtualenv]] to set up a local virtualenv mediagoblin instance <br />
#* run it with ./lazyserver.sh so you get debug output (if you run celeryd separately, you won't see logging.info from stuff run in celery tasks)<br />
# <tt>cp -r mediagoblin/plugins/sampleplugin mediagoblin/plugins/myplugin</tt> (where "myplugin" is your plugin name)<br />
# <tt>cp mediagoblin.ini mediagoblin_local.ini</tt><br />
# edit <tt>mediagoblin_local.ini</tt> and add <nowiki>[[mediagoblin.plugins.myplugin]]</nowiki> under the <nowiki>[plugins]</nowiki> section to enable your plugin<br />
<br />
Now you can look at e.g. other core or non-core plugins for inspiration. See [[Available Plugins]] for non-core plugins.<br />
<br />
<br />
The way you change things in plugins is by use of '''hooks'''. At certain points in the mediagoblin code, a function will say "run all hooks with name XYZ", and if you've defined a hook with such a name in a plugin you've enabled, it'll get run there. The [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/api.html documentation on Plugin API] is your friend for defining hooks. The sample plugin shows a use of the 'setup' hook.<br />
<br />
Note: templates are hooked with a call to pluginapi.register_template_hooks, instead of adding to the hooks variable.<br />
<br />
==Making an installable plugin==<br />
If you followed the steps above, you can use your plugin by copying it into the plugins folder; however, to get a plugin that is easily installable by users (e.g. with <tt>pip install myplugin</tt>), it should have a certain folder layout. <br />
<br />
A good example is the [https://github.com/commonsmachinery/mg-rdfa/ RDFa plugin]. This uses a setup.py file to install the files under the mediagoblin_rdfa folder into the lib/ folder of your mediagoblin installation. If you've checked out both mediagoblin and mg-rdfa in the same folder, e.g. ~/src/, you can do <pre>cd ~/src/mg-rdfa/<br />
../mediagoblin/bin/python setup.py build<br />
../mediagoblin/bin/python setup.py install</pre> to install it to your mediagoblin instance.<br />
<br />
The file layout of the repo is:<br />
<pre><br />
setup.py<br />
MANIFEST.in<br />
README.md<br />
mediagoblin_rdfa/<br />
__init.py__<br />
templates/<br />
mediagoblin/<br />
plugins/<br />
rdfa/<br />
metadata.html<br />
</pre><br />
<br />
The setup.py file defines an option include_package_data=True, which makes it read the file MANIFEST.in; MANIFEST.in contains rules for which files to include when installing. In this case, rule <tt>recursive-include mediagoblin_rdfa *.html</tt> makes it include the HTML template file (and any other you put under mediagoblin_rdfa), retaining the folder layout.<br />
<br />
==Referencing the mediagoblin test set in a plugin==<br />
<br />
The tests in mediagoblin define a nice "test_app" and functions for logging in and posting and so on. You can use this in your installable plugin.<br />
<br />
From your plugin (assuming a layout like mg-rdfa described above), doing<br />
<br />
cp -r ../mediagoblin/mediagoblin/tests .<br />
<br />
lets you run<br />
<br />
../mediagoblin/bin/py.test tests --boxed <br />
<br />
In fact, the only files you need are `conftest.py` and `pytest.ini`, and then you can copy over a single `test_foo.py` (only making sure import statements are absolute) and it should run fine.<br />
<br />
You also need to call setup_plugins() in your setup() function for your plugin to be properly loaded before testing.<br />
<br />
An example of this test method is in the [https://gitorious.org/mediagoblin-stock/mediagoblin-hidden_original/ "hidden original" plugin], which subclasses the submission tests.<br />
<br />
<br />
See also [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/tests.html Writing unit tests for plugins] in the docs.<br />
<br />
== Translating / Localising plugins ==<br />
''How do we do [[Translations]] in plugins?''<br />
<br />
(At least, a plugin would have to install a .mo file in e.g. lib/python2.7/site-packages/pluginname-0.1.3-py2.7.egg/pluginname/i18n/nn_NO/LC_MESSAGES/pluginname.mo (with domain "pluginname"?). Then mediagoblin would have to either discover that the file exists when setting up the plugin, or the plugin would have to add that to some list in some hook.<br />
<br />
mg_globals.py seems to create a gettext variable pointing at mediagoblin/i18n:<br />
<pre><br />
thread_scope.translations = gettext.translation(<br />
'mediagoblin',<br />
pkg_resources.resource_filename(<br />
'mediagoblin', 'i18n'), ['en'], fallback=True)<br />
</pre><br />
which is installed into the jinja template in template.py:<br />
<pre><br />
template_env.install_gettext_callables(<br />
mg_globals.thread_scope.translations.ugettext,<br />
mg_globals.thread_scope.translations.ungettext)<br />
</pre><br />
in a function called by app.py:<br />
<pre><br />
request.template_env = template.get_jinja_env(<br />
self.template_loader, request.locale)<br />
</pre><br />
)<br />
<br />
== Making database models ==<br />
See [[SQLAlchemy Tips]]</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=PluginsTips&diff=1473PluginsTips2014-01-26T11:24:15Z<p>Unhammer: /* Translating / Localising plugins */</p>
<hr />
<div>==Quickstart==<br />
In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get started with plugin creation. The simplest way to get up and running is to <br />
# Follow [[HackingHowto#How_to_set_up_and_maintain_an_environment_for_hacking_with_virtualenv]] to set up a local virtualenv mediagoblin instance <br />
#* run it with ./lazyserver.sh so you get debug output (if you run celeryd separately, you won't see logging.info from stuff run in celery tasks)<br />
# <tt>cp -r mediagoblin/plugins/sampleplugin mediagoblin/plugins/myplugin</tt> (where "myplugin" is your plugin name)<br />
# <tt>cp mediagoblin.ini mediagoblin_local.ini</tt><br />
# edit <tt>mediagoblin_local.ini</tt> and add <nowiki>[[mediagoblin.plugins.myplugin]]</nowiki> under the <nowiki>[plugins]</nowiki> section to enable your plugin<br />
<br />
Now you can look at e.g. other core or non-core plugins for inspiration. See [[Available Plugins]] for non-core plugins.<br />
<br />
<br />
The way you change things in plugins is by use of '''hooks'''. At certain points in the mediagoblin code, a function will say "run all hooks with name XYZ", and if you've defined a hook with such a name in a plugin you've enabled, it'll get run there. The [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/api.html documentation on Plugin API] is your friend for defining hooks. The sample plugin shows a use of the 'setup' hook.<br />
<br />
Note: templates are hooked with a call to pluginapi.register_template_hooks, instead of adding to the hooks variable.<br />
<br />
==Making an installable plugin==<br />
If you followed the steps above, you can use your plugin by copying it into the plugins folder; however, to get a plugin that is easily installable by users (e.g. with <tt>pip install myplugin</tt>), it should have a certain folder layout. <br />
<br />
A good example is the [https://github.com/commonsmachinery/mg-rdfa/ RDFa plugin]. This uses a setup.py file to install the files under the mediagoblin_rdfa folder into the lib/ folder of your mediagoblin installation. If you've checked out both mediagoblin and mg-rdfa in the same folder, e.g. ~/src/, you can do <pre>cd ~/src/mg-rdfa/<br />
../mediagoblin/bin/python setup.py build<br />
../mediagoblin/bin/python setup.py install</pre> to install it to your mediagoblin instance.<br />
<br />
The file layout of the repo is:<br />
<pre><br />
setup.py<br />
MANIFEST.in<br />
README.md<br />
mediagoblin_rdfa/<br />
__init.py__<br />
templates/<br />
mediagoblin/<br />
plugins/<br />
rdfa/<br />
metadata.html<br />
</pre><br />
<br />
The setup.py file defines an option include_package_data=True, which makes it read the file MANIFEST.in; MANIFEST.in contains rules for which files to include when installing. In this case, rule <tt>recursive-include mediagoblin_rdfa *.html</tt> makes it include the HTML template file (and any other you put under mediagoblin_rdfa), retaining the folder layout.<br />
<br />
==Referencing the mediagoblin test set in a plugin==<br />
<br />
The tests in mediagoblin define a nice "test_app" and functions for logging in and posting and so on. You can use this in your installable plugin.<br />
<br />
From your plugin (assuming a layout like mg-rdfa described above), doing<br />
<br />
cp -r ../mediagoblin/mediagoblin/tests .<br />
<br />
lets you run<br />
<br />
../mediagoblin/bin/py.test tests --boxed <br />
<br />
In fact, the only files you need are `conftest.py` and `pytest.ini`, and then you can copy over a single `test_foo.py` (only making sure import statements are absolute) and it should run fine.<br />
<br />
You also need to call setup_plugins() in your setup() function for your plugin to be properly loaded before testing.<br />
<br />
An example of this test method is in the [https://gitorious.org/mediagoblin-stock/mediagoblin-hidden_original/ "hidden original" plugin], which subclasses the submission tests.<br />
<br />
<br />
See also [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/tests.html Writing unit tests for plugins] in the docs.<br />
<br />
== Translating / Localising plugins ==<br />
How do we do [[Translations]] in plugins? At least, a plugin would have to install a .mo file in e.g. lib/python2.7/site-packages/pluginname-0.1.3-py2.7.egg/pluginname/i18n/nn_NO/LC_MESSAGES/pluginname.mo (with domain "pluginname"?). Then mediagoblin would have to either discover that the file exists when setting up the plugin, or the plugin would have to add that to some list in some hook.<br />
<br />
mg_globals.py seems to create a gettext variable pointing at mediagoblin/i18n:<br />
<pre><br />
thread_scope.translations = gettext.translation(<br />
'mediagoblin',<br />
pkg_resources.resource_filename(<br />
'mediagoblin', 'i18n'), ['en'], fallback=True)<br />
</pre><br />
which is installed into the jinja template in template.py:<br />
<pre><br />
template_env.install_gettext_callables(<br />
mg_globals.thread_scope.translations.ugettext,<br />
mg_globals.thread_scope.translations.ungettext)<br />
</pre><br />
in a function called by app.py:<br />
<pre><br />
request.template_env = template.get_jinja_env(<br />
self.template_loader, request.locale)<br />
</pre><br />
<br />
== Making database models ==<br />
See [[SQLAlchemy Tips]]</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=Talk:HackingHowto&diff=1472Talk:HackingHowto2014-01-26T11:23:42Z<p>Unhammer: </p>
<hr />
<div>With the SQL transition underway, on my Debian stable system, I had to install the <tt>python-sqlalchemy</tt> and <tt>python-sqlalchemy-ext</tt> packages to make MediaGoblin go. IRC advised me to get them from backports, which I did, and that's worked out, although there are packages in stable too -- not sure if they're new enough. Either way, should instructions about this be added to the documentation now? Or is a temporary transition thing? --[[User:Brett|Brett]] 09:18, 31 March 2012 (EDT)<br />
: When using a virtualenv, these are downloaded for you in ./bin/python setup.py develop --upgrade</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=SQLAlchemy_Tips&diff=1471SQLAlchemy Tips2014-01-24T14:58:26Z<p>Unhammer: sql</p>
<hr />
<div>Some tips for hacking around with the MediaGoblin databases and SQLAlchemy.<br />
<br />
First off, read all of http://docs.sqlalchemy.org/en/latest/orm/tutorial.html – great tutorial. <br />
<br />
Then, to use MediaGoblin database classes/tables from your python interpreter, you can start up <tt>bin/python2 -i</tt> (if you use Emacs, you can hand it <tt>/path/to/mediagoblin/bin/python2 -i</tt> when it asks which python to start) and begin with this boilerplate:<br />
<pre><br />
from sqlalchemy import create_engine<br />
engine = create_engine('sqlite:///:memory:', echo=True)<br />
# Use an in-memory table, and have all SQL statements echoed back in<br />
# the interpreter for debuggability<br />
<br />
from mediagoblin.db.base import Session<br />
Session.configure(bind=engine)<br />
session=Session()<br />
<br />
# Some of the tables need to have some entries in order for the rest<br />
# to be usable:<br />
from mediagoblin.db.models import FOUNDATIONS as MAIN_FOUNDATIONS<br />
for Model,rows in MAIN_FOUNDATIONS.items():<br />
for parameters in rows:<br />
new_row = Model(**parameters)<br />
session.add(new_row)<br />
<br />
from mediagoblin.db.base import Base<br />
from mediagoblin.db.models import * # User, MediaEntry, etc<br />
Base.metadata.create_all(engine) # creates all tables<br />
</pre><br />
<br />
Now try it out:<br />
<pre><br />
session.query(User).all() # An empty list<br />
<br />
goblinda=User(username='goblinda',email='gob@lin.da')<br />
session.add(goblinda)<br />
session.query(User).all() # Now has an entry =D<br />
</pre></div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=SQL_Database_Backend&diff=1470SQL Database Backend2014-01-24T13:56:22Z<p>Unhammer: </p>
<hr />
<div>'''This is all outdated; the switch to SQL has happened.'''<br />
<br />
<br />
== Introduction ==<br />
MediaGoblin currently uses MongoDB as its database backend. There have been various reasons for this decision (see: [[Why MongoDB]], yes really, look at it. It has a lot of insight what are the problems with sql). Still, the idea of an SQL backend is coming up from time to time.<br />
<br />
...<br />
<br />
The existence of this wiki page does not give any evidence of the SQL backend getting developed soon!<br />
<br />
== People ==<br />
Even an experimental implementation of an SQL backend, that will only support basic features will need a good amount of time/energy from some (core) developers. That time/energy will not be available to other projects.<br />
* Some knowing sqlalchemy is needed.<br />
* Someone having a good idea on the thin db layer is needed (Elrond will most likely be available for helping)<br />
* Someone having an idea on the current queries used in the code is needed too.<br />
<br />
== Development models ==<br />
* The biggest question is: Do we want to try an experimental sql backend? This is a tricky question. If it fails, it will be a lot of wasted time/energy. And contributors wouldn't be happy to have wasted their time/energy.<br />
* The wasted time/energy could be limited by limiting the first target. For example one could dedide, that the sql backend is not intended to support the full feature set.<br />
* Then there is the question on how to develop this:<br />
*# Long term branch: Develop an SQL replacement in a long term branch. The good: Anything can be changed as needed. Not so good: Some people dislike long term branches for various reasons.<br />
*# Trying to develop this alongside the current backend, in tree.<br />
<br />
== Major Issues ==<br />
The current db model uses dicts to allow flexible storage for extensions and different media types. Representing these data in SQL is not simple. The dicts are normally keyed by a string and have arbitrary values. There are basically two ways:<br />
<br />
# Have a table with a key column (string) and a value column (string, json encoded). This will allow easy reconstruction of the dict for an object fetch and will allow queries on the key of the dict.<br />
#: ''However'', it will not allow queries on the values. For example the imaginary geolocation extension will store the lat/lon there and might want to search for other media "nearby".<br />
#: If the value column is split into three columns (i.e. <code>value_int value_string value_json</code>), some queries, such as those on numbers, would be possible. .<br />
#: '''Note:''' The table would actually look something like this: <code>create table super_data_table(media_id references media(id), plugin_id int, key, value_int, value_string);</code> ... note that this design was predicted, determined by Chris Webber to be gross, and was aimed to be avoided, hence the choice of MongoDB. See [[Why MongoDB]] for details.<br />
# Actually structure the things into distinct tables. The above mentioned imaginary geolocation extension would have its own table with lat/lon fields and be able to perform efficient queries on it.<br />
<br />
== Pros and Cons ==<br />
<br />
=== MongoDB ===<br />
<br />
==== Pros ====<br />
* Allows you to lazily flexibly store stuff like multiple media type information, plugin data, etc<br />
* Supposedly scales up really well!<br />
* Programming interface feels clean and pythonic.<br />
* Migrations + flexible stuff might be a bit easier<br />
* Shows a lot of promise<br />
* Seems to be pretty fast if you have the right resources and carefully tuned your database<br />
* We've already built our tooling around it!<br />
<br />
==== Cons ====<br />
<br />
* Much, much more resource dependent on low and middle scale deployments than SQL would be<br />
* Seems like you have to really tune your server around the database<br />
* Have to think super carefully about indexes and (mostly) construct one index per common query. (That's a bit misleading, but pretty close to the truth.) Furthermore, one feels wary of creating more indexes because memory mapping means each index (and so each query) sucks up even more RAM. (At scale you also have to think about indexes for relational databases.) <br />
* "On the fly" queries not so easy, could be much more expensive than <br />
* No joins (normalized data on insertion to obviate the need for joins)<br />
* How much do we really care about scaling up on the database layer anyway? (What's the expected write-load of media-goblin.) <br />
** DeviantArt at Libre Graphics Meeting 2011 expressed that SQL + memcached works just fine for them and didn't think more complex things were necessary. That's a huge install base, and it seems hard to believe that MediaGoblin sites will hit that scale level. (How big is/are their SQL box/en? With automatic-sharding, MognoDB might be able to give better write performance per-dollar than SQL solutions of the same size.)<br />
** We're much more likely to run into media scaling issues before database scaling issues<br />
** In an ideal federated world, scaling down might be more important than scaling up because everyone will be running their own servers<br />
* In the end, flexibility doesn't seem to be worth much because you can't really do arbitrary queries on stuff you dump in if you want it to have any sort of speed whatsoever because of indexing considerations. (Good SQL schema design, and performance limitations of joins presents similar limitations.)<br />
* Though promising looking and solutions to issues keep coming up fast and hitting the core software it's also very new and not as well established.<br />
* It's not like migrations entirely disappeared, but they're probably easier with certain flexible things<br />
* Every few weeks someone brings up an SQL backend ;)<br />
<br />
=== SQL(alchemy) ===<br />
<br />
==== Pros ====<br />
<br />
* Scales down and up pretty well (possibly not as high up as MongoDB but again if it's high enough for DeviantArt...) and scaling down really does matter in our case<br />
* Developers are generally pretty used to it<br />
* SQLAlchemy has a really strong and established codebase<br />
* Arbitrary, unplanned queries!<br />
* Flexible schemas still possible but in some different ways (but not as nicely directly and certainly in no way as loosely flexible)<br />
* Extremely well established.<br />
<br />
==== Cons ====<br />
<br />
* Would mean a *lot* of rewriting!<br />
* Would mean having to write a careful migration path from mongodb->SQL!<br />
* Not as directly flexible as MongoDB (though we can design cleverly for a certain type of flexibility)<br />
* Migrations with certain types of database flexibility could break in really irritating for users (and people helping the users) ways<br />
* Working on this move could slow down other work or be hard to coordinate in parallell with other development<br />
* Probably does not scale up quite as high (again, it'd have to be "much higher than deviantart", which sounds like an unlikely problem we'd like to have)<br />
* Rewriting could possibly hurt our momentum in a serious way, especially if there's a feature freeze during the transition (and if there isn't, merging in master could become a real irritation)... even if we agree to do a rewrite, would we want to wait a while to do so? (Alternately, waiting to rewrite means the rewrite just gets harder.)<br />
<br />
== Statements by some developers ==<br />
=== Chris Webber's weigh-in ===<br />
<br />
So I think several things about the whole possible move to SQL.<br />
<br />
First of all, after having written out the Pros & Cons of each, it<br />
seems like maybe MongoDB is a lot of extra complexity and not gains in<br />
the areas I expected it to be. The supposed win wasn't scaling (and<br />
as predicted, scaling down has been something we've had to work around<br />
carefully), it was flexibility. Does MongoDB allow for extra<br />
flexibility? (And we *do* need flexibility for MediaGoblin's design.)<br />
Yes in the sense that you can dump in whatever, but if you intend to<br />
query on any of those attributes, it's mostly no. Indexes are<br />
expensive, and we have to spend a lot of time carefully pussyfooting<br />
around them.<br />
<br />
While planning MediaGoblin, I knew that there were two patterns for<br />
making things flexible in SQL... one of them is the table with "key,<br />
value, type". I thought that was unacceptably gross, and still do<br />
think so.<br />
<br />
The other option is that you have a "main" table (like MediaEntry), it<br />
references what "type" it is (such as "video", "image", whatever), and<br />
external tables for the extra information for that type point to the<br />
MediaEntry via a foreignkey and provide whatever media type specific<br />
data. Similarly, use external tables for plugins. The main reason I<br />
didn't want to deal with this is because I imagined migrations<br />
becoming a convoluted mess. It wasn't that we *wouldn't need*<br />
migrations in MongoDB, it's that maybe migrations would be less<br />
nightmarish with extensible stuff involved. In retrospect this was<br />
pretty reactive to a number of frustrating times I've had to try and<br />
walk people through broken migrations. But I'm getting the sense that<br />
I'll have to walk people through database complexity or breakage as<br />
much or more in MongoDB, and the complexity of managing indexes for<br />
any sort of extensibility is as bad or worse than dealing with<br />
migrations with an extensible SQL setup.<br />
<br />
So, okay. I think I just made a pretty compelling case for moving<br />
back to SQL. So what then?<br />
<br />
Two options have been proposed: try and support both SQL and MongoDB<br />
at the same time, or create a branch that switches from MongoDB to<br />
SQL. I'm afraid that the former just doesn't sound like a good idea<br />
to me at all... it seems like it'll result in a system with a<br />
massively bloated codebase, hard for new contributors to work with,<br />
hard to maintain, and "worst of both worlds" types compromises. Think<br />
about this for a second: how do you map things like migrations,<br />
indexing, etc over? Do we really want to completely rewrite the<br />
MongoDB query tools over to SQL? Some people have said that "it looks<br />
like we have a pretty simple use of MongoDB so this layer won't be so<br />
complex." To me that sounds like classic hacker "well that can't be<br />
so hard" underestimation of the complexity of the problem. Anyway, I<br />
already see a ton of complexities, and I'm sure there are more I<br />
haven't even been able to see.<br />
<br />
So the remaining option is to do a branch to switch from MongoDB -><br />
SQL. There's some risk of this also... it's hard to maintain a big<br />
overhaul branch while the mainline is constantly changing. There's<br />
also a risk of fracturing, and if we change our mind and stay with<br />
MongoDB, there's even a risk of forking! Not to mention that working<br />
on a branch that's so huge that doesn't get pulled in is incredibly<br />
demoralizing.<br />
<br />
But we can reduce all those risks if we can come to a *consensus* that<br />
this is what we want to do. So I propose at the next meeting we<br />
discuss this and try to make sure we're at community consensus before<br />
agreeing to move to SQL (if that's indeed what we intend to do) and if<br />
we do so, move to SQL *entirely*.<br />
<br />
Here's what I envision the path to that future will look like:<br />
<br />
* Create a branch that prototypes all the models being switched over to SQLAlchemy, included "multiple media types" implemented with a friendly API.<br />
* Assuming that works nice, continue work in that branch to switch all code over to using SQL.<br />
* Figure out how to do migrations nicely in SQL, including with multiple media types (I have some thoughts on how to do this "nicely")<br />
* Create a MongoDB->SQL migration tool<br />
<br />
Anyway, let's discuss this at next meeting!<br />
<br />
=== Elrond's comments ===<br />
I'm quite with Chris Webber here.<br />
<br />
One thing, I have to add/where I'm thinking a bit different: We should try to identify tasks in the sql-migration that can be done just "now". What does that mean? Moving to sql(alchemy) means a lot of changes. And some of these changes can be done with the mongodb backend as well. For example we're currently changing the document attribute access over from <code>doc["field"]</code> to <code>doc.field</code>. This does not hurt the current code (mostly) and makes the remaining "move to sql(alchemy)" easier. This reduces the amount of work happening in a long term branch. And that's the critical work (can be dropped, is demotivating, ...). So moving as much work as possible before actually starting the sql branch will increase motivation and decrease chances of failure. This sounds like more work, because the same code possibly needs to be touched twice. This might be true. It might also mean more work, as we might need to create some extra support code to assist in moving to a new idea. I still think, this is better, because it's outside of this long lived branch.<br />
<br />
== Planing the Migration to SQL ==<br />
This chapter contains some ideas, sketches, schedules, whatever developers feel they need to create a big plan.<br />
<br />
=== Some sketchup in SQL ===<br />
<pre><br />
create table users(<br />
id integer primary key not null,<br />
username varchar(30) unique not null,<br />
email text unique,<br />
created timestamp default CURRENT_TIMESTAMP,<br />
pw_hash text,<br />
email_verified boolean default FALSE,<br />
status varchar(30) default 'needs_email_verification',<br />
verification_key uuid,<br />
is_admin boolean default FALSE,<br />
url text,<br />
bio text,<br />
fp_verification_key uuid,<br />
fp_token_expire timestamp<br />
);<br />
<br />
create table file_records(<br />
id integer primary key not null,<br />
filename text array -- or json?<br />
);<br />
<br />
create table media_entries(<br />
id integer primary key not null,<br />
uploader integer references users(id) not null,<br />
title text,<br />
slug text,<br />
created timestamp default CURRENT_TIMESTAMP,<br />
description text,<br />
state text default 'unprocessed',<br />
queued_media_file integer references file_records(id),<br />
queued_task_id text,<br />
fail_error text,<br />
fail_metadata text -- json<br />
);<br />
<br />
create table media_files(<br />
id integer primary key not null,<br />
media_id integer references media_entries(id) not null,<br />
name varchar(30) not null,<br />
-- "original", "thumbnail", ...<br />
-- maybe also "attachment"-something?<br />
file integer references file_records(id) not null<br />
);<br />
<br />
create table media_comments(<br />
id integer primary key not null,<br />
media_entry integer references media_entries(id) not null,<br />
author integer references users(id) not null,<br />
created timestamp default CURRENT_TIMESTAMP,<br />
content text<br />
);<br />
</pre><br />
<br />
=== Migration needs ===<br />
<br />
If we switch to SQL(Alchemy), we'll need to handle migrations. This seems to have three requirements:<br />
<br />
# It has to be easy for users (as easy or easier than ./bin/gmg migrate)<br />
# It has to be easy for developers<br />
# It has to be able to handle our "extensible" infrastructure. There will have to be separately handled migrations for MediaGoblin core, as well as for each media type and plugin. All while remaining pretty simple!<br />
<br />
As far as I can tell, this should be pretty possible. We have three options:<br />
<br />
# Completely roll our own system based off of SQLAlchemy<br />
# Use [http://readthedocs.org/docs/sqlalchemy-migrate/en/latest/ SQLAlchemy-migrate]<br />
# Roll our own system that uses components of SQLAlchemy-migrate.<br />
<br />
One way or another I suspect we are going with option 2 or 3. SQLAlchemy-migrate provides some useful extensions to SQLAlchemy that are database related but can be used outside of SQLAlchemy-migrate's "complete tooling", namely the [http://readthedocs.org/docs/sqlalchemy-migrate/en/v0.7.2/changeset.html#changeset-system changeset system]. Using the whole tooling system provides some cool stuff like some experimental auto-migration-detection, but might fall short in handling our extensible infrastructure... it's not clear yet.<br />
<br />
Whatever we do, it looks like we can probably fit our 3 needs using option #2 or option #3. That's good!<br />
<br />
=== SQL switch tentative action plan ===<br />
<br />
Here's a tentative plan of what an action plan for switching to SQL(alchemy).<br />
<br />
Items marked with (*) might or should be done before the actual sql branch is created. Meaning, they can likely be done "in master".<br />
<br />
# Port all models to SQLAlchemy models. (*)<br />
# Make sure that port includes a nice way to handle media types in an extensible way! (*)<br />
# Add filepath conversion tools (basically write 2 simple functions to switch filepaths -> slash-joined-strings and back.), use them everywhere (*)<br />
# Add easy-database-init ./bin/gmg function (*)<br />
# Port all code over to using sqlalchemy<br />
# Add db-migrations tools (*)<br />
# Add a mongodb->SQL converter script (*)<br />
# ???<br />
# Profit!!! (Sorry, obligatory)<br />
<br />
== Testing SQL ==<br />
As the SQL stuff in master starts to get somewhat useable, we would like to invite adventurous/experienced users/developers to try it out and give us feedback.<br />
<br />
'''WARNING''': ''The SQL code is still in flux, so don't put any valuable data in your SQL based setup. For example the schema might still change in an incompatible way. When the switch to SQL has happened, there will of course be migrations to keep all data across schema changes.''<br />
<br />
=== Communication and Issues ===<br />
If you're going to test SQL support, please join IRC and talk to the developers first (or at least after testing). The whole thing is alpha quality and there are things not working. We know about most (we think!), but we surely want to learn about new problems!<br />
<br />
Some issues are documented in the [http://issues.mediagoblin.org/query?status=!closed&keywords=~sql issue tracker]<br />
<br />
=== Setting the SQL database ===<br />
The default is an sqlite database <code>mediagoblin.db</code> next to your config file. If you want to change that, put a variable <code>sql_engine</code> into the <code>[mediagoblin]</code> section. It is an sqlalchemy engine URI.<br />
<br />
=== Switching mediagoblin to SQL ===<br />
First: Set the database URI as above, if you need to.<BR>Then after switching mediagoblin to SQL, read on the next sections and find the one relevant to you.<br />
<br />
So switching the complete mediagoblin code to use sql is not super simple, but it's quite doable.<BR><br />
To enable:<br />
* You need to create a file named <code>mediagoblin/db/sql_switch.py</code> and put one this in it: <code>use_sql = True</code><br />
To disable:<br />
* Either delete <code>mediagoblin/db/sql_switch.py</code> AND <code>mediagoblin/db/sql_switch.pyc</code><br />
* Or: Change the contents to: <code>use_sql = False</code><br />
<br />
=== Testing the conversion tool ===<br />
So you want to convert your existing mongo data into the sql database. Maybe just to look at the resulting database layout and the data in there? It's easy:<br />
# Configure your database uri (see above)<br />
# Clean your database, drop all previous tables.<br />
# Run this command:<br />
bin/gmg [-cf your_mediagoblin.ini] convert_mongo_to_sql<br />
<br />
=== Creating a new empty database (and later: keeping it current) ===<br />
Very simple:<br />
# Configure your database uri (see above)<br />
# Run this:<br />
bin/gmg [-cf your_mediagoblin.ini] dbupdate</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=SQL_Database_Backend&diff=1469SQL Database Backend2014-01-24T13:56:10Z<p>Unhammer: </p>
<hr />
<div>'''This is all outdated; the switch to SQL has happened.'''<br />
<br />
{{TOCD}}<br />
<br />
== Introduction ==<br />
MediaGoblin currently uses MongoDB as its database backend. There have been various reasons for this decision (see: [[Why MongoDB]], yes really, look at it. It has a lot of insight what are the problems with sql). Still, the idea of an SQL backend is coming up from time to time.<br />
<br />
...<br />
<br />
The existence of this wiki page does not give any evidence of the SQL backend getting developed soon!<br />
<br />
== People ==<br />
Even an experimental implementation of an SQL backend, that will only support basic features will need a good amount of time/energy from some (core) developers. That time/energy will not be available to other projects.<br />
* Some knowing sqlalchemy is needed.<br />
* Someone having a good idea on the thin db layer is needed (Elrond will most likely be available for helping)<br />
* Someone having an idea on the current queries used in the code is needed too.<br />
<br />
== Development models ==<br />
* The biggest question is: Do we want to try an experimental sql backend? This is a tricky question. If it fails, it will be a lot of wasted time/energy. And contributors wouldn't be happy to have wasted their time/energy.<br />
* The wasted time/energy could be limited by limiting the first target. For example one could dedide, that the sql backend is not intended to support the full feature set.<br />
* Then there is the question on how to develop this:<br />
*# Long term branch: Develop an SQL replacement in a long term branch. The good: Anything can be changed as needed. Not so good: Some people dislike long term branches for various reasons.<br />
*# Trying to develop this alongside the current backend, in tree.<br />
<br />
== Major Issues ==<br />
The current db model uses dicts to allow flexible storage for extensions and different media types. Representing these data in SQL is not simple. The dicts are normally keyed by a string and have arbitrary values. There are basically two ways:<br />
<br />
# Have a table with a key column (string) and a value column (string, json encoded). This will allow easy reconstruction of the dict for an object fetch and will allow queries on the key of the dict.<br />
#: ''However'', it will not allow queries on the values. For example the imaginary geolocation extension will store the lat/lon there and might want to search for other media "nearby".<br />
#: If the value column is split into three columns (i.e. <code>value_int value_string value_json</code>), some queries, such as those on numbers, would be possible. .<br />
#: '''Note:''' The table would actually look something like this: <code>create table super_data_table(media_id references media(id), plugin_id int, key, value_int, value_string);</code> ... note that this design was predicted, determined by Chris Webber to be gross, and was aimed to be avoided, hence the choice of MongoDB. See [[Why MongoDB]] for details.<br />
# Actually structure the things into distinct tables. The above mentioned imaginary geolocation extension would have its own table with lat/lon fields and be able to perform efficient queries on it.<br />
<br />
== Pros and Cons ==<br />
<br />
=== MongoDB ===<br />
<br />
==== Pros ====<br />
* Allows you to lazily flexibly store stuff like multiple media type information, plugin data, etc<br />
* Supposedly scales up really well!<br />
* Programming interface feels clean and pythonic.<br />
* Migrations + flexible stuff might be a bit easier<br />
* Shows a lot of promise<br />
* Seems to be pretty fast if you have the right resources and carefully tuned your database<br />
* We've already built our tooling around it!<br />
<br />
==== Cons ====<br />
<br />
* Much, much more resource dependent on low and middle scale deployments than SQL would be<br />
* Seems like you have to really tune your server around the database<br />
* Have to think super carefully about indexes and (mostly) construct one index per common query. (That's a bit misleading, but pretty close to the truth.) Furthermore, one feels wary of creating more indexes because memory mapping means each index (and so each query) sucks up even more RAM. (At scale you also have to think about indexes for relational databases.) <br />
* "On the fly" queries not so easy, could be much more expensive than <br />
* No joins (normalized data on insertion to obviate the need for joins)<br />
* How much do we really care about scaling up on the database layer anyway? (What's the expected write-load of media-goblin.) <br />
** DeviantArt at Libre Graphics Meeting 2011 expressed that SQL + memcached works just fine for them and didn't think more complex things were necessary. That's a huge install base, and it seems hard to believe that MediaGoblin sites will hit that scale level. (How big is/are their SQL box/en? With automatic-sharding, MognoDB might be able to give better write performance per-dollar than SQL solutions of the same size.)<br />
** We're much more likely to run into media scaling issues before database scaling issues<br />
** In an ideal federated world, scaling down might be more important than scaling up because everyone will be running their own servers<br />
* In the end, flexibility doesn't seem to be worth much because you can't really do arbitrary queries on stuff you dump in if you want it to have any sort of speed whatsoever because of indexing considerations. (Good SQL schema design, and performance limitations of joins presents similar limitations.)<br />
* Though promising looking and solutions to issues keep coming up fast and hitting the core software it's also very new and not as well established.<br />
* It's not like migrations entirely disappeared, but they're probably easier with certain flexible things<br />
* Every few weeks someone brings up an SQL backend ;)<br />
<br />
=== SQL(alchemy) ===<br />
<br />
==== Pros ====<br />
<br />
* Scales down and up pretty well (possibly not as high up as MongoDB but again if it's high enough for DeviantArt...) and scaling down really does matter in our case<br />
* Developers are generally pretty used to it<br />
* SQLAlchemy has a really strong and established codebase<br />
* Arbitrary, unplanned queries!<br />
* Flexible schemas still possible but in some different ways (but not as nicely directly and certainly in no way as loosely flexible)<br />
* Extremely well established.<br />
<br />
==== Cons ====<br />
<br />
* Would mean a *lot* of rewriting!<br />
* Would mean having to write a careful migration path from mongodb->SQL!<br />
* Not as directly flexible as MongoDB (though we can design cleverly for a certain type of flexibility)<br />
* Migrations with certain types of database flexibility could break in really irritating for users (and people helping the users) ways<br />
* Working on this move could slow down other work or be hard to coordinate in parallell with other development<br />
* Probably does not scale up quite as high (again, it'd have to be "much higher than deviantart", which sounds like an unlikely problem we'd like to have)<br />
* Rewriting could possibly hurt our momentum in a serious way, especially if there's a feature freeze during the transition (and if there isn't, merging in master could become a real irritation)... even if we agree to do a rewrite, would we want to wait a while to do so? (Alternately, waiting to rewrite means the rewrite just gets harder.)<br />
<br />
== Statements by some developers ==<br />
=== Chris Webber's weigh-in ===<br />
<br />
So I think several things about the whole possible move to SQL.<br />
<br />
First of all, after having written out the Pros & Cons of each, it<br />
seems like maybe MongoDB is a lot of extra complexity and not gains in<br />
the areas I expected it to be. The supposed win wasn't scaling (and<br />
as predicted, scaling down has been something we've had to work around<br />
carefully), it was flexibility. Does MongoDB allow for extra<br />
flexibility? (And we *do* need flexibility for MediaGoblin's design.)<br />
Yes in the sense that you can dump in whatever, but if you intend to<br />
query on any of those attributes, it's mostly no. Indexes are<br />
expensive, and we have to spend a lot of time carefully pussyfooting<br />
around them.<br />
<br />
While planning MediaGoblin, I knew that there were two patterns for<br />
making things flexible in SQL... one of them is the table with "key,<br />
value, type". I thought that was unacceptably gross, and still do<br />
think so.<br />
<br />
The other option is that you have a "main" table (like MediaEntry), it<br />
references what "type" it is (such as "video", "image", whatever), and<br />
external tables for the extra information for that type point to the<br />
MediaEntry via a foreignkey and provide whatever media type specific<br />
data. Similarly, use external tables for plugins. The main reason I<br />
didn't want to deal with this is because I imagined migrations<br />
becoming a convoluted mess. It wasn't that we *wouldn't need*<br />
migrations in MongoDB, it's that maybe migrations would be less<br />
nightmarish with extensible stuff involved. In retrospect this was<br />
pretty reactive to a number of frustrating times I've had to try and<br />
walk people through broken migrations. But I'm getting the sense that<br />
I'll have to walk people through database complexity or breakage as<br />
much or more in MongoDB, and the complexity of managing indexes for<br />
any sort of extensibility is as bad or worse than dealing with<br />
migrations with an extensible SQL setup.<br />
<br />
So, okay. I think I just made a pretty compelling case for moving<br />
back to SQL. So what then?<br />
<br />
Two options have been proposed: try and support both SQL and MongoDB<br />
at the same time, or create a branch that switches from MongoDB to<br />
SQL. I'm afraid that the former just doesn't sound like a good idea<br />
to me at all... it seems like it'll result in a system with a<br />
massively bloated codebase, hard for new contributors to work with,<br />
hard to maintain, and "worst of both worlds" types compromises. Think<br />
about this for a second: how do you map things like migrations,<br />
indexing, etc over? Do we really want to completely rewrite the<br />
MongoDB query tools over to SQL? Some people have said that "it looks<br />
like we have a pretty simple use of MongoDB so this layer won't be so<br />
complex." To me that sounds like classic hacker "well that can't be<br />
so hard" underestimation of the complexity of the problem. Anyway, I<br />
already see a ton of complexities, and I'm sure there are more I<br />
haven't even been able to see.<br />
<br />
So the remaining option is to do a branch to switch from MongoDB -><br />
SQL. There's some risk of this also... it's hard to maintain a big<br />
overhaul branch while the mainline is constantly changing. There's<br />
also a risk of fracturing, and if we change our mind and stay with<br />
MongoDB, there's even a risk of forking! Not to mention that working<br />
on a branch that's so huge that doesn't get pulled in is incredibly<br />
demoralizing.<br />
<br />
But we can reduce all those risks if we can come to a *consensus* that<br />
this is what we want to do. So I propose at the next meeting we<br />
discuss this and try to make sure we're at community consensus before<br />
agreeing to move to SQL (if that's indeed what we intend to do) and if<br />
we do so, move to SQL *entirely*.<br />
<br />
Here's what I envision the path to that future will look like:<br />
<br />
* Create a branch that prototypes all the models being switched over to SQLAlchemy, included "multiple media types" implemented with a friendly API.<br />
* Assuming that works nice, continue work in that branch to switch all code over to using SQL.<br />
* Figure out how to do migrations nicely in SQL, including with multiple media types (I have some thoughts on how to do this "nicely")<br />
* Create a MongoDB->SQL migration tool<br />
<br />
Anyway, let's discuss this at next meeting!<br />
<br />
=== Elrond's comments ===<br />
I'm quite with Chris Webber here.<br />
<br />
One thing, I have to add/where I'm thinking a bit different: We should try to identify tasks in the sql-migration that can be done just "now". What does that mean? Moving to sql(alchemy) means a lot of changes. And some of these changes can be done with the mongodb backend as well. For example we're currently changing the document attribute access over from <code>doc["field"]</code> to <code>doc.field</code>. This does not hurt the current code (mostly) and makes the remaining "move to sql(alchemy)" easier. This reduces the amount of work happening in a long term branch. And that's the critical work (can be dropped, is demotivating, ...). So moving as much work as possible before actually starting the sql branch will increase motivation and decrease chances of failure. This sounds like more work, because the same code possibly needs to be touched twice. This might be true. It might also mean more work, as we might need to create some extra support code to assist in moving to a new idea. I still think, this is better, because it's outside of this long lived branch.<br />
<br />
== Planing the Migration to SQL ==<br />
This chapter contains some ideas, sketches, schedules, whatever developers feel they need to create a big plan.<br />
<br />
=== Some sketchup in SQL ===<br />
<pre><br />
create table users(<br />
id integer primary key not null,<br />
username varchar(30) unique not null,<br />
email text unique,<br />
created timestamp default CURRENT_TIMESTAMP,<br />
pw_hash text,<br />
email_verified boolean default FALSE,<br />
status varchar(30) default 'needs_email_verification',<br />
verification_key uuid,<br />
is_admin boolean default FALSE,<br />
url text,<br />
bio text,<br />
fp_verification_key uuid,<br />
fp_token_expire timestamp<br />
);<br />
<br />
create table file_records(<br />
id integer primary key not null,<br />
filename text array -- or json?<br />
);<br />
<br />
create table media_entries(<br />
id integer primary key not null,<br />
uploader integer references users(id) not null,<br />
title text,<br />
slug text,<br />
created timestamp default CURRENT_TIMESTAMP,<br />
description text,<br />
state text default 'unprocessed',<br />
queued_media_file integer references file_records(id),<br />
queued_task_id text,<br />
fail_error text,<br />
fail_metadata text -- json<br />
);<br />
<br />
create table media_files(<br />
id integer primary key not null,<br />
media_id integer references media_entries(id) not null,<br />
name varchar(30) not null,<br />
-- "original", "thumbnail", ...<br />
-- maybe also "attachment"-something?<br />
file integer references file_records(id) not null<br />
);<br />
<br />
create table media_comments(<br />
id integer primary key not null,<br />
media_entry integer references media_entries(id) not null,<br />
author integer references users(id) not null,<br />
created timestamp default CURRENT_TIMESTAMP,<br />
content text<br />
);<br />
</pre><br />
<br />
=== Migration needs ===<br />
<br />
If we switch to SQL(Alchemy), we'll need to handle migrations. This seems to have three requirements:<br />
<br />
# It has to be easy for users (as easy or easier than ./bin/gmg migrate)<br />
# It has to be easy for developers<br />
# It has to be able to handle our "extensible" infrastructure. There will have to be separately handled migrations for MediaGoblin core, as well as for each media type and plugin. All while remaining pretty simple!<br />
<br />
As far as I can tell, this should be pretty possible. We have three options:<br />
<br />
# Completely roll our own system based off of SQLAlchemy<br />
# Use [http://readthedocs.org/docs/sqlalchemy-migrate/en/latest/ SQLAlchemy-migrate]<br />
# Roll our own system that uses components of SQLAlchemy-migrate.<br />
<br />
One way or another I suspect we are going with option 2 or 3. SQLAlchemy-migrate provides some useful extensions to SQLAlchemy that are database related but can be used outside of SQLAlchemy-migrate's "complete tooling", namely the [http://readthedocs.org/docs/sqlalchemy-migrate/en/v0.7.2/changeset.html#changeset-system changeset system]. Using the whole tooling system provides some cool stuff like some experimental auto-migration-detection, but might fall short in handling our extensible infrastructure... it's not clear yet.<br />
<br />
Whatever we do, it looks like we can probably fit our 3 needs using option #2 or option #3. That's good!<br />
<br />
=== SQL switch tentative action plan ===<br />
<br />
Here's a tentative plan of what an action plan for switching to SQL(alchemy).<br />
<br />
Items marked with (*) might or should be done before the actual sql branch is created. Meaning, they can likely be done "in master".<br />
<br />
# Port all models to SQLAlchemy models. (*)<br />
# Make sure that port includes a nice way to handle media types in an extensible way! (*)<br />
# Add filepath conversion tools (basically write 2 simple functions to switch filepaths -> slash-joined-strings and back.), use them everywhere (*)<br />
# Add easy-database-init ./bin/gmg function (*)<br />
# Port all code over to using sqlalchemy<br />
# Add db-migrations tools (*)<br />
# Add a mongodb->SQL converter script (*)<br />
# ???<br />
# Profit!!! (Sorry, obligatory)<br />
<br />
== Testing SQL ==<br />
As the SQL stuff in master starts to get somewhat useable, we would like to invite adventurous/experienced users/developers to try it out and give us feedback.<br />
<br />
'''WARNING''': ''The SQL code is still in flux, so don't put any valuable data in your SQL based setup. For example the schema might still change in an incompatible way. When the switch to SQL has happened, there will of course be migrations to keep all data across schema changes.''<br />
<br />
=== Communication and Issues ===<br />
If you're going to test SQL support, please join IRC and talk to the developers first (or at least after testing). The whole thing is alpha quality and there are things not working. We know about most (we think!), but we surely want to learn about new problems!<br />
<br />
Some issues are documented in the [http://issues.mediagoblin.org/query?status=!closed&keywords=~sql issue tracker]<br />
<br />
=== Setting the SQL database ===<br />
The default is an sqlite database <code>mediagoblin.db</code> next to your config file. If you want to change that, put a variable <code>sql_engine</code> into the <code>[mediagoblin]</code> section. It is an sqlalchemy engine URI.<br />
<br />
=== Switching mediagoblin to SQL ===<br />
First: Set the database URI as above, if you need to.<BR>Then after switching mediagoblin to SQL, read on the next sections and find the one relevant to you.<br />
<br />
So switching the complete mediagoblin code to use sql is not super simple, but it's quite doable.<BR><br />
To enable:<br />
* You need to create a file named <code>mediagoblin/db/sql_switch.py</code> and put one this in it: <code>use_sql = True</code><br />
To disable:<br />
* Either delete <code>mediagoblin/db/sql_switch.py</code> AND <code>mediagoblin/db/sql_switch.pyc</code><br />
* Or: Change the contents to: <code>use_sql = False</code><br />
<br />
=== Testing the conversion tool ===<br />
So you want to convert your existing mongo data into the sql database. Maybe just to look at the resulting database layout and the data in there? It's easy:<br />
# Configure your database uri (see above)<br />
# Clean your database, drop all previous tables.<br />
# Run this command:<br />
bin/gmg [-cf your_mediagoblin.ini] convert_mongo_to_sql<br />
<br />
=== Creating a new empty database (and later: keeping it current) ===<br />
Very simple:<br />
# Configure your database uri (see above)<br />
# Run this:<br />
bin/gmg [-cf your_mediagoblin.ini] dbupdate</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=User_privileges&diff=1468User privileges2014-01-22T13:58:08Z<p>Unhammer: /* To change the default set of privileges */</p>
<hr />
<div>Since 0.6.0, there's a user privileges/permissions system. You can make the user "USERNAME" an admin by doing<br />
<pre><br />
./bin/gmg makeadmin USERNAME<br />
</pre><br />
<br />
An admin gets access to an Admin Panel, and when looking at other user pages, is able to revoke/give privileges. Default ones are:<br />
<pre><br />
active Yes <br />
admin No <br />
commenter Yes <br />
moderator No <br />
reporter Yes <br />
uploader Yes <br />
</pre><br />
<br />
==To change the default set of privileges==<br />
This is currently possible by checking out a branch that tilly-q is working on.<br />
<pre><br />
git remote add tilly-q git://gitorious.org/mediagoblin/npigeons-mediagoblin.git<br />
git fetch --all<br />
git cherry-pick 0a5981fd5416c60858c3e460f943692d1c62629d<br />
</pre><br />
Then set something like <pre>user_privilege_scheme= "reporter,commenter"</pre> in your mediagoblin_local.ini</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=User_privileges&diff=1467User privileges2014-01-22T13:30:37Z<p>Unhammer: </p>
<hr />
<div>Since 0.6.0, there's a user privileges/permissions system. You can make the user "USERNAME" an admin by doing<br />
<pre><br />
./bin/gmg makeadmin USERNAME<br />
</pre><br />
<br />
An admin gets access to an Admin Panel, and when looking at other user pages, is able to revoke/give privileges. Default ones are:<br />
<pre><br />
active Yes <br />
admin No <br />
commenter Yes <br />
moderator No <br />
reporter Yes <br />
uploader Yes <br />
</pre><br />
<br />
==To change the default set of privileges==<br />
This is currently possible by checking out a branch that tilly-q is working on.<br />
<pre><br />
git remote add tilly-q git://gitorious.org/mediagoblin/npigeons-mediagoblin.git<br />
git fetch --all<br />
git cherry-pick 0a5981fd5416c60858c3e460f943692d1c62629d<br />
</pre></div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=User_privileges&diff=1466User privileges2014-01-22T12:24:46Z<p>Unhammer: </p>
<hr />
<div>Since 0.6.0, there's a user privileges/permissions system. You can make the user "USERNAME" an admin by doing<br />
<pre><br />
./bin/gmg makeadmin USERNAME<br />
</pre><br />
<br />
An admin gets access to an Admin Panel, and when looking at other user pages, is able to revoke/give privileges. Default ones are:<br />
<pre><br />
active Yes <br />
admin No <br />
commenter Yes <br />
moderator No <br />
reporter Yes <br />
uploader Yes <br />
</pre><br />
<br />
==To change the default set of privileges==<br />
This is currently possible by checking out a branch that tilly-q is working on.</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=User_privileges&diff=1465User privileges2014-01-21T20:41:34Z<p>Unhammer: </p>
<hr />
<div>Since 0.6.0, there's a user privileges/permissions system. You can make the user "USERNAME" an admin by doing<br />
<pre><br />
./bin/gmg makeadmin USERNAME<br />
</pre><br />
<br />
An admin gets access to an Admin Panel, and when looking at other user pages, is able to revoke/give privileges. Default ones are:<br />
<pre><br />
active Yes <br />
admin No <br />
commenter Yes <br />
moderator No <br />
reporter Yes <br />
uploader Yes <br />
</pre><br />
<br />
: ''How do we change the default set of privileges?''</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=User_privileges&diff=1464User privileges2014-01-21T20:30:15Z<p>Unhammer: Created page with "Since 0.6.0, there's a user privileges/permissions system. You can make the user "USERNAME" an admin by doing <pre> ./bin/gmg makeadmin USERNAME </pre>"</p>
<hr />
<div>Since 0.6.0, there's a user privileges/permissions system. You can make the user "USERNAME" an admin by doing<br />
<pre><br />
./bin/gmg makeadmin USERNAME<br />
</pre></div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=Available_Plugins&diff=1463Available Plugins2014-01-21T17:02:11Z<p>Unhammer: /* Third party plugins */</p>
<hr />
<div>= A list of Available Plugins for MediaGoblin =<br />
<br />
==Core plugins ==<br />
<br />
The core plugins are available right away with your mediaboblin instance, they are not necessarily activated by default, tho. Please check the [http://docs.mediagoblin.org/siteadmin/plugins.html plugin documentation] to learn how to activate them.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name !! Options !! Usage !! Doc !! Author<br />
|-<br />
| api || || || || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| flatpagesfile || || It allows you to add pages to your MediaGoblin instance which are not generated from user content || [http://docs.mediagoblin.org/plugindocs/flatpagesfile.html doc] ||<br />
|-<br />
|httpapiauth|| || Enables [http://en.wikipedia.org/wiki/Basic_access_authentication HTTP basic authentication] for the API || || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| oauth || || The [http://en.wikipedia.org/wiki/Oauth oauth] plugin enables third party web applications to authenticate as one or more GNU MediaGoblin users ||[http://docs.mediagoblin.org/plugindocs/oauth.html doc] || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| sampleplugin || || Dummy plugin for example purpose || ||<br />
|-<br />
|trim_whitespace|| || This plugin inserts a Middleware that filters out whitespace from the returned HTML in the Response() objects.||[http://docs.mediagoblin.org/plugindocs/trim_whitespace.html doc] ||<br />
|}<br />
<br />
== Third party plugins==<br />
<br />
Those plugins have been made by various contributors to extend mediagoblin's possibilities<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name !! What !! Options !! Doc !! How to get it ? !! Author<br />
|-<br />
| mediagoblin-licenses || Extend or replace the default license list ||<br />
* Append to existing list (true/false)<br />
* Licence list<br />
|| || pip install mediagoblin-licenses ||<br />
|-<br />
|[https://github.com/Velmont/gmg_localfiles gmg_localfiles] || Import files from your local filesystem without duplicating them. || || [https://github.com/Velmont/gmg_localfiles/blob/master/README.md doc] || git clone https://github.com/Velmont/gmg_localfiles.git || odinho/Velmont<br />
|-<br />
|[https://github.com/commonsmachinery/mediagoblin_svg mediagoblin_svg] || SVG image support || Display previews and thumbnails as SVG || || Check README ||<br />
|-<br />
|[https://gitorious.org/mediagoblin-stock/mediagoblin-hidden_original mediagoblin-hidden_original] || Hidden original || Give original upload a secret url, only publish downsized version || || pip install mediagoblin-hidden_original || [[User:Unhammer|Unhammer]] <br />
|}<br />
<br />
== Work in progress ==<br />
<br />
You're working on a plugin but it's not ready ? Please let us know, cause if someone wants to make the same feature, this person will help you rather than doing the same thing from scratch ! <br />
<br />
{| class="wikitable"<br />
|-<br />
! Temporary name !! Planned features !! Repo or a place to see the WIP !! Author !! Status<br />
|-<br />
| GoblinCommons ||<br />
* Area for storing a user's existing Wikimedia Commons account details<br />
* Support for uploading copy of media to Wikimedia Commons along with description, tags, licence details(editable + defaulting to the ones the media has in MediaWiki if any)<br />
* Support for editing uploaded copy details (description + new image version along with comments)<br />
* Available only for free media formats as per https://commons.wikimedia.org/wiki/Commons:File_types<br />
|| T.B.A., about mid Jan 2014<br />
|| gts <br />
|| Just began on a few hours per week basis. E.T.A., about end Feb/early March 2014.<br />
|-<br />
| Raw file support for GMG ||<br />
* Support NEF, '''done'''<br />
* Support CR2<br />
* Investigate support for other formats (if they include full-quality JPEG inside the file)<br />
|| https://github.com/Velmont/mediagoblin/tree/raw_image_mediatype || odinho/Velmont ||<br />
|-<br />
| [https://gitorious.org/~spaetz/mediagoblin/spaetz-mediagoblin/trees/WIP/tag_cloud_plugin Tag Cloud] || Add a tag cloud feature for your templates || git clone git://gitorious.org/~spaetz/mediagoblin/spaetz-mediagoblin.git HEAD:WIP/tag_cloud_plugin || Spaetz || [http://www.media.sspaeth.de/u/spaetz/ It works] but still needs core modifications.<br />
|-<br />
| Image property export for use externally ||<br />
* Basically, this would allow other sites (WordPress, etc.) to use their own plugins to easily grab image properties (description, name, url, alt text, license) for automatic insertion.<br />
* Include additional image properties (eg. alt text, short description) for scraping.<br />
|| [none] || exhipigeonist || Absolutely no work and very little planning. Currently no more than an idea I'd like to work on later if I get time, in conjunction with a WordPress plugin that would utilise it, of course.<br />
|- <br />
| HTML5-Multiupload || Adds a the html5 multifile input to the submit form || https://github.com/VenKamikaze/mediagoblin-html5-multi-upload<br />
|| kamikaze-m || Working on 0.4.1, but needs proper review. Test and report bugs, it might not work, but it's absolutly harmless to your MG instance <br />
|-<br />
| [http://chiselapp.com/user/saulgoode/repository/mg-gimp MG-GIMP] || Allows uploading of XCF images and GIMP resources (palettes, patterns, curves, gradients, brushes, dynamics, and tool presets). Thumbnails and previews are generated by an instance of GIMP running headlessly as a daemon. || [http://chiselapp.com/user/saulgoode/repository/mg-gimp MG-GIMP docs and code repository] [http://barn.kerosenecow.net/ Instance featuring GIMP media]|| saulgoode || Nearing completion. Still needs to be packaged for easier deployment.<br />
|-<br />
|[https://github.com/commonsmachinery/mg-rdfa mg-rdfa] || Embedding image metadata as RDFa || https://github.com/commonsmachinery/mg-rdfa || artfwo || Experimental, only SVG images are supported at the moment. <br />
|-<br />
|[https://gitorious.org/mediagoblin-stock/mediagoblin-purchase mediagoblin-purchase] || Purchase button || https://gitorious.org/mediagoblin-stock/mediagoblin-purchase || [[User:Unhammer|Unhammer]] || Just started <br />
|}</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=Available_Plugins&diff=1462Available Plugins2014-01-21T16:50:01Z<p>Unhammer: /* Third party plugins */</p>
<hr />
<div>= A list of Available Plugins for MediaGoblin =<br />
<br />
==Core plugins ==<br />
<br />
The core plugins are available right away with your mediaboblin instance, they are not necessarily activated by default, tho. Please check the [http://docs.mediagoblin.org/siteadmin/plugins.html plugin documentation] to learn how to activate them.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name !! Options !! Usage !! Doc !! Author<br />
|-<br />
| api || || || || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| flatpagesfile || || It allows you to add pages to your MediaGoblin instance which are not generated from user content || [http://docs.mediagoblin.org/plugindocs/flatpagesfile.html doc] ||<br />
|-<br />
|httpapiauth|| || Enables [http://en.wikipedia.org/wiki/Basic_access_authentication HTTP basic authentication] for the API || || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| oauth || || The [http://en.wikipedia.org/wiki/Oauth oauth] plugin enables third party web applications to authenticate as one or more GNU MediaGoblin users ||[http://docs.mediagoblin.org/plugindocs/oauth.html doc] || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| sampleplugin || || Dummy plugin for example purpose || ||<br />
|-<br />
|trim_whitespace|| || This plugin inserts a Middleware that filters out whitespace from the returned HTML in the Response() objects.||[http://docs.mediagoblin.org/plugindocs/trim_whitespace.html doc] ||<br />
|}<br />
<br />
== Third party plugins==<br />
<br />
Those plugins have been made by various contributors to extend mediagoblin's possibilities<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name !! What !! Options !! Doc !! How to get it ? !! Author<br />
|-<br />
| mediagoblin-licenses || Extend or replace the default license list ||<br />
* Append to existing list (true/false)<br />
* Licence list<br />
|| || pip install mediagoblin-licenses ||<br />
|-<br />
|[https://github.com/Velmont/gmg_localfiles gmg_localfiles] || Import files from your local filesystem without duplicating them. || || [https://github.com/Velmont/gmg_localfiles/blob/master/README.md doc] || git clone https://github.com/Velmont/gmg_localfiles.git || odinho/Velmont<br />
|-<br />
|[https://github.com/commonsmachinery/mediagoblin_svg mediagoblin_svg] || SVG image support || Display previews and thumbnails as SVG || || Check README ||<br />
|-<br />
|[https://gitorious.org/mediagoblin-stock/mediagoblin-hidden_original mediagoblin-hidden_original] || Hidden original || Give original upload a secret url, only publish downsized version || || git clone it for now || [[User:Unhammer|Unhammer]] <br />
|}<br />
<br />
== Work in progress ==<br />
<br />
You're working on a plugin but it's not ready ? Please let us know, cause if someone wants to make the same feature, this person will help you rather than doing the same thing from scratch ! <br />
<br />
{| class="wikitable"<br />
|-<br />
! Temporary name !! Planned features !! Repo or a place to see the WIP !! Author !! Status<br />
|-<br />
| GoblinCommons ||<br />
* Area for storing a user's existing Wikimedia Commons account details<br />
* Support for uploading copy of media to Wikimedia Commons along with description, tags, licence details(editable + defaulting to the ones the media has in MediaWiki if any)<br />
* Support for editing uploaded copy details (description + new image version along with comments)<br />
* Available only for free media formats as per https://commons.wikimedia.org/wiki/Commons:File_types<br />
|| T.B.A., about mid Jan 2014<br />
|| gts <br />
|| Just began on a few hours per week basis. E.T.A., about end Feb/early March 2014.<br />
|-<br />
| Raw file support for GMG ||<br />
* Support NEF, '''done'''<br />
* Support CR2<br />
* Investigate support for other formats (if they include full-quality JPEG inside the file)<br />
|| https://github.com/Velmont/mediagoblin/tree/raw_image_mediatype || odinho/Velmont ||<br />
|-<br />
| [https://gitorious.org/~spaetz/mediagoblin/spaetz-mediagoblin/trees/WIP/tag_cloud_plugin Tag Cloud] || Add a tag cloud feature for your templates || git clone git://gitorious.org/~spaetz/mediagoblin/spaetz-mediagoblin.git HEAD:WIP/tag_cloud_plugin || Spaetz || [http://www.media.sspaeth.de/u/spaetz/ It works] but still needs core modifications.<br />
|-<br />
| Image property export for use externally ||<br />
* Basically, this would allow other sites (WordPress, etc.) to use their own plugins to easily grab image properties (description, name, url, alt text, license) for automatic insertion.<br />
* Include additional image properties (eg. alt text, short description) for scraping.<br />
|| [none] || exhipigeonist || Absolutely no work and very little planning. Currently no more than an idea I'd like to work on later if I get time, in conjunction with a WordPress plugin that would utilise it, of course.<br />
|- <br />
| HTML5-Multiupload || Adds a the html5 multifile input to the submit form || https://github.com/VenKamikaze/mediagoblin-html5-multi-upload<br />
|| kamikaze-m || Working on 0.4.1, but needs proper review. Test and report bugs, it might not work, but it's absolutly harmless to your MG instance <br />
|-<br />
| [http://chiselapp.com/user/saulgoode/repository/mg-gimp MG-GIMP] || Allows uploading of XCF images and GIMP resources (palettes, patterns, curves, gradients, brushes, dynamics, and tool presets). Thumbnails and previews are generated by an instance of GIMP running headlessly as a daemon. || [http://chiselapp.com/user/saulgoode/repository/mg-gimp MG-GIMP docs and code repository] [http://barn.kerosenecow.net/ Instance featuring GIMP media]|| saulgoode || Nearing completion. Still needs to be packaged for easier deployment.<br />
|-<br />
|[https://github.com/commonsmachinery/mg-rdfa mg-rdfa] || Embedding image metadata as RDFa || https://github.com/commonsmachinery/mg-rdfa || artfwo || Experimental, only SVG images are supported at the moment. <br />
|-<br />
|[https://gitorious.org/mediagoblin-stock/mediagoblin-purchase mediagoblin-purchase] || Purchase button || https://gitorious.org/mediagoblin-stock/mediagoblin-purchase || [[User:Unhammer|Unhammer]] || Just started <br />
|}</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=Available_Plugins&diff=1461Available Plugins2014-01-21T16:49:04Z<p>Unhammer: /* Work in progress */</p>
<hr />
<div>= A list of Available Plugins for MediaGoblin =<br />
<br />
==Core plugins ==<br />
<br />
The core plugins are available right away with your mediaboblin instance, they are not necessarily activated by default, tho. Please check the [http://docs.mediagoblin.org/siteadmin/plugins.html plugin documentation] to learn how to activate them.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name !! Options !! Usage !! Doc !! Author<br />
|-<br />
| api || || || || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| flatpagesfile || || It allows you to add pages to your MediaGoblin instance which are not generated from user content || [http://docs.mediagoblin.org/plugindocs/flatpagesfile.html doc] ||<br />
|-<br />
|httpapiauth|| || Enables [http://en.wikipedia.org/wiki/Basic_access_authentication HTTP basic authentication] for the API || || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| oauth || || The [http://en.wikipedia.org/wiki/Oauth oauth] plugin enables third party web applications to authenticate as one or more GNU MediaGoblin users ||[http://docs.mediagoblin.org/plugindocs/oauth.html doc] || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| sampleplugin || || Dummy plugin for example purpose || ||<br />
|-<br />
|trim_whitespace|| || This plugin inserts a Middleware that filters out whitespace from the returned HTML in the Response() objects.||[http://docs.mediagoblin.org/plugindocs/trim_whitespace.html doc] ||<br />
|}<br />
<br />
== Third party plugins==<br />
<br />
Those plugins have been made by various contributors to extend mediagoblin's possibilities<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name !! What !! Options !! Doc !! How to get it ? !! Author<br />
|-<br />
| mediagoblin-licenses || Extend or replace the default license list ||<br />
* Append to existing list (true/false)<br />
* Licence list<br />
|| || pip install mediagoblin-licenses ||<br />
|-<br />
|[https://github.com/Velmont/gmg_localfiles gmg_localfiles] || Import files from your local filesystem without duplicating them. || || [https://github.com/Velmont/gmg_localfiles/blob/master/README.md doc] || git clone https://github.com/Velmont/gmg_localfiles.git || odinho/Velmont<br />
|-<br />
|[https://github.com/commonsmachinery/mediagoblin_svg mediagoblin_svg] || SVG image support || Display previews and thumbnails as SVG || || Check README ||<br />
|-<br />
|[https://gitorious.org/mediagoblin-stock/mediagoblin-hidden_original mediagoblin-hidden_original] || Hidden original || Give original upload a secret url, only publish downsized version || [[User:Unhammer|Unhammer]] <br />
|}<br />
<br />
== Work in progress ==<br />
<br />
You're working on a plugin but it's not ready ? Please let us know, cause if someone wants to make the same feature, this person will help you rather than doing the same thing from scratch ! <br />
<br />
{| class="wikitable"<br />
|-<br />
! Temporary name !! Planned features !! Repo or a place to see the WIP !! Author !! Status<br />
|-<br />
| GoblinCommons ||<br />
* Area for storing a user's existing Wikimedia Commons account details<br />
* Support for uploading copy of media to Wikimedia Commons along with description, tags, licence details(editable + defaulting to the ones the media has in MediaWiki if any)<br />
* Support for editing uploaded copy details (description + new image version along with comments)<br />
* Available only for free media formats as per https://commons.wikimedia.org/wiki/Commons:File_types<br />
|| T.B.A., about mid Jan 2014<br />
|| gts <br />
|| Just began on a few hours per week basis. E.T.A., about end Feb/early March 2014.<br />
|-<br />
| Raw file support for GMG ||<br />
* Support NEF, '''done'''<br />
* Support CR2<br />
* Investigate support for other formats (if they include full-quality JPEG inside the file)<br />
|| https://github.com/Velmont/mediagoblin/tree/raw_image_mediatype || odinho/Velmont ||<br />
|-<br />
| [https://gitorious.org/~spaetz/mediagoblin/spaetz-mediagoblin/trees/WIP/tag_cloud_plugin Tag Cloud] || Add a tag cloud feature for your templates || git clone git://gitorious.org/~spaetz/mediagoblin/spaetz-mediagoblin.git HEAD:WIP/tag_cloud_plugin || Spaetz || [http://www.media.sspaeth.de/u/spaetz/ It works] but still needs core modifications.<br />
|-<br />
| Image property export for use externally ||<br />
* Basically, this would allow other sites (WordPress, etc.) to use their own plugins to easily grab image properties (description, name, url, alt text, license) for automatic insertion.<br />
* Include additional image properties (eg. alt text, short description) for scraping.<br />
|| [none] || exhipigeonist || Absolutely no work and very little planning. Currently no more than an idea I'd like to work on later if I get time, in conjunction with a WordPress plugin that would utilise it, of course.<br />
|- <br />
| HTML5-Multiupload || Adds a the html5 multifile input to the submit form || https://github.com/VenKamikaze/mediagoblin-html5-multi-upload<br />
|| kamikaze-m || Working on 0.4.1, but needs proper review. Test and report bugs, it might not work, but it's absolutly harmless to your MG instance <br />
|-<br />
| [http://chiselapp.com/user/saulgoode/repository/mg-gimp MG-GIMP] || Allows uploading of XCF images and GIMP resources (palettes, patterns, curves, gradients, brushes, dynamics, and tool presets). Thumbnails and previews are generated by an instance of GIMP running headlessly as a daemon. || [http://chiselapp.com/user/saulgoode/repository/mg-gimp MG-GIMP docs and code repository] [http://barn.kerosenecow.net/ Instance featuring GIMP media]|| saulgoode || Nearing completion. Still needs to be packaged for easier deployment.<br />
|-<br />
|[https://github.com/commonsmachinery/mg-rdfa mg-rdfa] || Embedding image metadata as RDFa || https://github.com/commonsmachinery/mg-rdfa || artfwo || Experimental, only SVG images are supported at the moment. <br />
|-<br />
|[https://gitorious.org/mediagoblin-stock/mediagoblin-purchase mediagoblin-purchase] || Purchase button || https://gitorious.org/mediagoblin-stock/mediagoblin-purchase || [[User:Unhammer|Unhammer]] || Just started <br />
|}</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=User:Unhammer&diff=1460User:Unhammer2014-01-21T16:48:58Z<p>Unhammer: Created page with "[http://wiki.apertium.org/wiki/User:Unhammer me]"</p>
<hr />
<div>[http://wiki.apertium.org/wiki/User:Unhammer me]</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=Available_Plugins&diff=1459Available Plugins2014-01-21T16:48:36Z<p>Unhammer: /* Third party plugins */</p>
<hr />
<div>= A list of Available Plugins for MediaGoblin =<br />
<br />
==Core plugins ==<br />
<br />
The core plugins are available right away with your mediaboblin instance, they are not necessarily activated by default, tho. Please check the [http://docs.mediagoblin.org/siteadmin/plugins.html plugin documentation] to learn how to activate them.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name !! Options !! Usage !! Doc !! Author<br />
|-<br />
| api || || || || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| flatpagesfile || || It allows you to add pages to your MediaGoblin instance which are not generated from user content || [http://docs.mediagoblin.org/plugindocs/flatpagesfile.html doc] ||<br />
|-<br />
|httpapiauth|| || Enables [http://en.wikipedia.org/wiki/Basic_access_authentication HTTP basic authentication] for the API || || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| oauth || || The [http://en.wikipedia.org/wiki/Oauth oauth] plugin enables third party web applications to authenticate as one or more GNU MediaGoblin users ||[http://docs.mediagoblin.org/plugindocs/oauth.html doc] || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| sampleplugin || || Dummy plugin for example purpose || ||<br />
|-<br />
|trim_whitespace|| || This plugin inserts a Middleware that filters out whitespace from the returned HTML in the Response() objects.||[http://docs.mediagoblin.org/plugindocs/trim_whitespace.html doc] ||<br />
|}<br />
<br />
== Third party plugins==<br />
<br />
Those plugins have been made by various contributors to extend mediagoblin's possibilities<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name !! What !! Options !! Doc !! How to get it ? !! Author<br />
|-<br />
| mediagoblin-licenses || Extend or replace the default license list ||<br />
* Append to existing list (true/false)<br />
* Licence list<br />
|| || pip install mediagoblin-licenses ||<br />
|-<br />
|[https://github.com/Velmont/gmg_localfiles gmg_localfiles] || Import files from your local filesystem without duplicating them. || || [https://github.com/Velmont/gmg_localfiles/blob/master/README.md doc] || git clone https://github.com/Velmont/gmg_localfiles.git || odinho/Velmont<br />
|-<br />
|[https://github.com/commonsmachinery/mediagoblin_svg mediagoblin_svg] || SVG image support || Display previews and thumbnails as SVG || || Check README ||<br />
|-<br />
|[https://gitorious.org/mediagoblin-stock/mediagoblin-hidden_original mediagoblin-hidden_original] || Hidden original || Give original upload a secret url, only publish downsized version || [[User:Unhammer|Unhammer]] <br />
|}<br />
<br />
== Work in progress ==<br />
<br />
You're working on a plugin but it's not ready ? Please let us know, cause if someone wants to make the same feature, this person will help you rather than doing the same thing from scratch ! <br />
<br />
{| class="wikitable"<br />
|-<br />
! Temporary name !! Planned features !! Repo or a place to see the WIP !! Author !! Status<br />
|-<br />
| GoblinCommons ||<br />
* Area for storing a user's existing Wikimedia Commons account details<br />
* Support for uploading copy of media to Wikimedia Commons along with description, tags, licence details(editable + defaulting to the ones the media has in MediaWiki if any)<br />
* Support for editing uploaded copy details (description + new image version along with comments)<br />
* Available only for free media formats as per https://commons.wikimedia.org/wiki/Commons:File_types<br />
|| T.B.A., about mid Jan 2014<br />
|| gts <br />
|| Just began on a few hours per week basis. E.T.A., about end Feb/early March 2014.<br />
|-<br />
| Raw file support for GMG ||<br />
* Support NEF, '''done'''<br />
* Support CR2<br />
* Investigate support for other formats (if they include full-quality JPEG inside the file)<br />
|| https://github.com/Velmont/mediagoblin/tree/raw_image_mediatype || odinho/Velmont ||<br />
|-<br />
| [https://gitorious.org/~spaetz/mediagoblin/spaetz-mediagoblin/trees/WIP/tag_cloud_plugin Tag Cloud] || Add a tag cloud feature for your templates || git clone git://gitorious.org/~spaetz/mediagoblin/spaetz-mediagoblin.git HEAD:WIP/tag_cloud_plugin || Spaetz || [http://www.media.sspaeth.de/u/spaetz/ It works] but still needs core modifications.<br />
|-<br />
| Image property export for use externally ||<br />
* Basically, this would allow other sites (WordPress, etc.) to use their own plugins to easily grab image properties (description, name, url, alt text, license) for automatic insertion.<br />
* Include additional image properties (eg. alt text, short description) for scraping.<br />
|| [none] || exhipigeonist || Absolutely no work and very little planning. Currently no more than an idea I'd like to work on later if I get time, in conjunction with a WordPress plugin that would utilise it, of course.<br />
|- <br />
| HTML5-Multiupload || Adds a the html5 multifile input to the submit form || https://github.com/VenKamikaze/mediagoblin-html5-multi-upload<br />
|| kamikaze-m || Working on 0.4.1, but needs proper review. Test and report bugs, it might not work, but it's absolutly harmless to your MG instance <br />
|-<br />
| [http://chiselapp.com/user/saulgoode/repository/mg-gimp MG-GIMP] || Allows uploading of XCF images and GIMP resources (palettes, patterns, curves, gradients, brushes, dynamics, and tool presets). Thumbnails and previews are generated by an instance of GIMP running headlessly as a daemon. || [http://chiselapp.com/user/saulgoode/repository/mg-gimp MG-GIMP docs and code repository] [http://barn.kerosenecow.net/ Instance featuring GIMP media]|| saulgoode || Nearing completion. Still needs to be packaged for easier deployment.<br />
|-<br />
|[https://github.com/commonsmachinery/mg-rdfa mg-rdfa] || Embedding image metadata as RDFa || https://github.com/commonsmachinery/mg-rdfa || artfwo || Experimental, only SVG images are supported at the moment. <br />
|-<br />
|[https://gitorious.org/mediagoblin-stock/mediagoblin-purchase mediagoblin-purchase] || Purchase button || https://gitorious.org/mediagoblin-stock/mediagoblin-purchase || [[User:Unhammer]] || Just started <br />
|}</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=Available_Plugins&diff=1458Available Plugins2014-01-21T16:47:18Z<p>Unhammer: /* Work in progress */</p>
<hr />
<div>= A list of Available Plugins for MediaGoblin =<br />
<br />
==Core plugins ==<br />
<br />
The core plugins are available right away with your mediaboblin instance, they are not necessarily activated by default, tho. Please check the [http://docs.mediagoblin.org/siteadmin/plugins.html plugin documentation] to learn how to activate them.<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name !! Options !! Usage !! Doc !! Author<br />
|-<br />
| api || || || || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| flatpagesfile || || It allows you to add pages to your MediaGoblin instance which are not generated from user content || [http://docs.mediagoblin.org/plugindocs/flatpagesfile.html doc] ||<br />
|-<br />
|httpapiauth|| || Enables [http://en.wikipedia.org/wiki/Basic_access_authentication HTTP basic authentication] for the API || || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| oauth || || The [http://en.wikipedia.org/wiki/Oauth oauth] plugin enables third party web applications to authenticate as one or more GNU MediaGoblin users ||[http://docs.mediagoblin.org/plugindocs/oauth.html doc] || [[User:Joar]], MediaGoblin contributors<br />
|-<br />
| sampleplugin || || Dummy plugin for example purpose || ||<br />
|-<br />
|trim_whitespace|| || This plugin inserts a Middleware that filters out whitespace from the returned HTML in the Response() objects.||[http://docs.mediagoblin.org/plugindocs/trim_whitespace.html doc] ||<br />
|}<br />
<br />
== Third party plugins==<br />
<br />
Those plugins have been made by various contributors to extend mediagoblin's possibilities<br />
<br />
{| class="wikitable"<br />
|-<br />
! Name !! What !! Options !! Doc !! How to get it ? !! Author<br />
|-<br />
| mediagoblin-licenses || Extend or replace the default license list ||<br />
* Append to existing list (true/false)<br />
* Licence list<br />
|| || pip install mediagoblin-licenses ||<br />
|-<br />
|[https://github.com/Velmont/gmg_localfiles gmg_localfiles] || Import files from your local filesystem without duplicating them. || || [https://github.com/Velmont/gmg_localfiles/blob/master/README.md doc] || git clone https://github.com/Velmont/gmg_localfiles.git || odinho/Velmont<br />
|-<br />
|[https://github.com/commonsmachinery/mediagoblin_svg mediagoblin_svg] || SVG image support || Display previews and thumbnails as SVG || || Check README ||<br />
|}<br />
<br />
== Work in progress ==<br />
<br />
You're working on a plugin but it's not ready ? Please let us know, cause if someone wants to make the same feature, this person will help you rather than doing the same thing from scratch ! <br />
<br />
{| class="wikitable"<br />
|-<br />
! Temporary name !! Planned features !! Repo or a place to see the WIP !! Author !! Status<br />
|-<br />
| GoblinCommons ||<br />
* Area for storing a user's existing Wikimedia Commons account details<br />
* Support for uploading copy of media to Wikimedia Commons along with description, tags, licence details(editable + defaulting to the ones the media has in MediaWiki if any)<br />
* Support for editing uploaded copy details (description + new image version along with comments)<br />
* Available only for free media formats as per https://commons.wikimedia.org/wiki/Commons:File_types<br />
|| T.B.A., about mid Jan 2014<br />
|| gts <br />
|| Just began on a few hours per week basis. E.T.A., about end Feb/early March 2014.<br />
|-<br />
| Raw file support for GMG ||<br />
* Support NEF, '''done'''<br />
* Support CR2<br />
* Investigate support for other formats (if they include full-quality JPEG inside the file)<br />
|| https://github.com/Velmont/mediagoblin/tree/raw_image_mediatype || odinho/Velmont ||<br />
|-<br />
| [https://gitorious.org/~spaetz/mediagoblin/spaetz-mediagoblin/trees/WIP/tag_cloud_plugin Tag Cloud] || Add a tag cloud feature for your templates || git clone git://gitorious.org/~spaetz/mediagoblin/spaetz-mediagoblin.git HEAD:WIP/tag_cloud_plugin || Spaetz || [http://www.media.sspaeth.de/u/spaetz/ It works] but still needs core modifications.<br />
|-<br />
| Image property export for use externally ||<br />
* Basically, this would allow other sites (WordPress, etc.) to use their own plugins to easily grab image properties (description, name, url, alt text, license) for automatic insertion.<br />
* Include additional image properties (eg. alt text, short description) for scraping.<br />
|| [none] || exhipigeonist || Absolutely no work and very little planning. Currently no more than an idea I'd like to work on later if I get time, in conjunction with a WordPress plugin that would utilise it, of course.<br />
|- <br />
| HTML5-Multiupload || Adds a the html5 multifile input to the submit form || https://github.com/VenKamikaze/mediagoblin-html5-multi-upload<br />
|| kamikaze-m || Working on 0.4.1, but needs proper review. Test and report bugs, it might not work, but it's absolutly harmless to your MG instance <br />
|-<br />
| [http://chiselapp.com/user/saulgoode/repository/mg-gimp MG-GIMP] || Allows uploading of XCF images and GIMP resources (palettes, patterns, curves, gradients, brushes, dynamics, and tool presets). Thumbnails and previews are generated by an instance of GIMP running headlessly as a daemon. || [http://chiselapp.com/user/saulgoode/repository/mg-gimp MG-GIMP docs and code repository] [http://barn.kerosenecow.net/ Instance featuring GIMP media]|| saulgoode || Nearing completion. Still needs to be packaged for easier deployment.<br />
|-<br />
|[https://github.com/commonsmachinery/mg-rdfa mg-rdfa] || Embedding image metadata as RDFa || https://github.com/commonsmachinery/mg-rdfa || artfwo || Experimental, only SVG images are supported at the moment. <br />
|-<br />
|[https://gitorious.org/mediagoblin-stock/mediagoblin-purchase mediagoblin-purchase] || Purchase button || https://gitorious.org/mediagoblin-stock/mediagoblin-purchase || [[User:Unhammer]] || Just started <br />
|}</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=Search&diff=1457Search2014-01-21T15:52:57Z<p>Unhammer: Created page with "Currently MG does not have any built-in search engine. This might be an interesting GSOC project."</p>
<hr />
<div>Currently MG does not have any built-in search engine. This might be an interesting [[GSOC_2013#Search_interface|GSOC]] project.</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=GSOC_2013&diff=1456GSOC 20132014-01-21T15:52:30Z<p>Unhammer: /* Search interface */</p>
<hr />
<div>We are participating in [http://www.google-melange.com/gsoc/homepage/google/gsoc2013 GSOC 2013] via the [http://www.gnu.org/software/soc-projects/ideas-2013.html#mediagoblin GNU umbrella]!<br />
<br />
We are also participating in [[Outreach Program for Women 2013]]... see that page for specific information about that program, but the projects suggested on this page are the same as those for OPW 2013 :)<br />
<br />
To both students and mentors, please read up on the [http://www.google-melange.com/document/show/gsoc_program/google/gsoc2013/help_page GSOC 2013 help page]!<br />
<br />
= How do I apply as a student? =<br />
<br />
Well, we need to be accepted as a mentoring org first :)<br />
<br />
But then:<br />
* Submit your application. Please see the application submission template on the [http://www.gnu.org/software/soc-projects/guidelines.html GNU GSoC guidelines page]. (You can also use this template for OPW, though they provide another template; it's fine to use that if you like.)<br />
* [http://mediagoblin.org/pages/join.html Join us] on IRC and on our mailing lists.<br />
* Set up a development environment via our [[HackingHowto]]<br />
* If you have never done web development in python before, MediaGoblin is a pretty good place to start! However, we highly recommend going through the [https://docs.djangoproject.com/en/1.5/intro/tutorial01/ Django tutorial]... this isn't a requirement, but it will help you be better prepared.<br />
* Work on a small task, and mention it in your application. The [http://issues.mediagoblin.org/query?status=!closed&keywords=~bitesized bitesized list] is often helpful.<br />
* Also, be aware when submitting: both Outreach Program for Women and Summer of Code applicants are expected to work '''40 hours per week'''... so, a full time internship! If you propose, please be sure you are ready to make that level of commitment.<br />
<br />
It's important that you communicate... most MediaGoblin communication happens on IRC, so you should [http://webchat.freenode.net/?channels=mediagoblin join us there] and discuss (#mediagoblin on irc.freenode.net)! Please, please join our channel and introduce yourself. We'd love to hear from you!<br />
<br />
= Possible projects =<br />
<br />
Here are a list of projects that students may wish to apply for for GSOC 2013:<br />
<br />
== Blogging system ==<br />
<br />
We've had an increasing number of people ask if MediaGoblin would be a useful blogging system. The answer is that at present, it isn't: even though we support an ascii art media type, media types aren't really the right way to handle blogging since they don't really fit with the "gallery" style of editing things in MediaGoblin.<br />
<br />
But people are interested in something that's more along the lines of Tumblr: a blogging platform with good media embedding integration. In many ways, MediaGoblin is perfect for this!<br />
<br />
Some thoughts:<br />
<br />
* Blogs might go at /u/foo-user/b/ and individual blogposts at /u/foo-user/b/blogpost-slug/<br />
* Also take a look at Chris' post about this, it has some important design notes: http://lists.mediagoblin.org/pipermail/devel/2013-April/000491.html<br />
* Blogging should be a plugin<br />
* To make blogging more useful (either to the blogging plugin or to external blogging engines) it would be good to have much better embedding support in MediaGoblin. So improving embedding support should be mentioned in the proposal.<br />
* HTML with something like TinyMCE integration would be good. HTML would need to be cleaned on the backend.<br />
* Maybe other types would be allowed (markdown, restructured text) though I'm not sure how much more complex that makes cleaning HTML output.<br />
* What about commenting? Would we need a seaparate comments table?<br />
<br />
Possible Mentor: spaetz<br />
<br />
== Pluggable user authentication & implementations ==<br />
<br />
We'd like a pluggable user authentication module. [http://openid.net/ OpenID] is very common authentication method, used e.g. by Google. [http://oauth.net/ OAuth] is also commonly (mis?)used (e.g. by Facebook and Twitter), although it was never designed for this purpose. Some people would also like [https://github.com/mozilla/browserid Persona/BrowserID] integration. Some people want to use things like [http://en.wikipedia.org/wiki/LDAP LDAP].<br />
<br />
With our new work toward pluginification it would be good to see the structure for user authentication to be interface'ified. It would also be good to have implementation for several types of logins such as OpenID, Persona/BrowserID, LDAP, and central authentication system stuff.<br />
<br />
'''Possible mentors:''' Nathan Yergler, [[User:Joar|Joar Wandborg]], [[User:Cwebber|Chris Webber]]<br />
<br />
== Administrative interface / moderation tools ==<br />
<br />
At present there isn't much as in terms of tooling to deal with things as an administrator. Several things would be greatly of help to admins at present:<br />
<br />
* Views to search for users and take actions upon them<br />
* Tools to deal with handling problematic content/users<br />
* More generalized panel for looking at things being processed<br />
* ???<br />
<br />
'''Possible mentors:''' --[[User:Copiesofcopies|Copiesofcopies]] ([[User talk:Copiesofcopies|talk]]) 15:02, 1 April 2013 (EDT), [[User:Joar|Joar Wandborg]], [[User:Cwebber|Chris Webber]]<br />
<br />
== Processing panel improvements ==<br />
<br />
While media is being uploaded, there's not very good and clear indications of this, though we have some basics.<br />
<br />
* Show the number of entries that are currently in processing in the dropdown user panel at the top of mediagoblin instances<br />
* In media entries that are currently in progress, give clearer information about amount of work left? (Note, showing percentages might be hard)<br />
* Nicer looking demonstrations of what failed.<br />
<br />
Note: this one might require a lot of graphic design and discussion; anyone interested in it would have to work closely with people on IRC before submitting a real proposal.<br />
<br />
'''Possible mentors:''' --[[User:Copiesofcopies|Copiesofcopies]] ([[User talk:Copiesofcopies|talk]]) 15:03, 1 April 2013 (EDT), [[User:Joar|Joar Wandborg]], [[User:Cwebber|Chris Webber]]<br />
<br />
== User upload account limits ==<br />
<br />
It would be great to have the ability to set limits on the amount of stuff people can upload. This would involve both hooks to track the space of media as it's saved (maybe this should be fairly core?) as well as the ability to refuse an upload because a limit has been already hit. Tools to change limits would also be good, especially via an administrative interface and ./bin/gmg<br />
<br />
--[[User:Joar|Joar]] ([[User talk:Joar|talk]]) 07:41, 30 March 2013 (EDT) <code>mediagoblin.storage.StorageInterface</code> should probably implement a <code>get_size</code> method or similar.<br />
<br />
'''Possible mentors:''' [[User:Joar|Joar Wandborg]], [[User:Cwebber|Chris Webber]]<br />
<br />
== Search interface ==<br />
<br />
We don't have any sort of [[Search]] for media whatsoever at present. Obviously people want this!<br />
<br />
There's been some discussion of a search interface that would be federated, but for the scope of this, we don't need to worry about that. It would be enough to investigate a search engine that would allow for searching across the subjects/titles/descriptions of media.<br />
<br />
'''Possible mentors:''' Nathan Yergler, Aeva NTSC, [[User:Cwebber|Chris Webber]]<br />
<br />
== Pluginifying media types ==<br />
<br />
Media types currently have their own way of being configured separately. It would be good to be able to make them into bona-fide plugins (if anything with some special case metadata). This will also allow media types to define things like special-case views, etc.<br />
<br />
This might actually be very simple, so consider making a proposal that's directly combined with the media type reprocessing framework below.<br />
<br />
Possible mentors: --[[User:Copiesofcopies|Copiesofcopies]] ([[User talk:Copiesofcopies|talk]]) 15:02, 1 April 2013 (EDT), [[User:Joar|Joar Wandborg]]<br />
<br />
== Media type reprocessing framework ==<br />
<br />
This one is tricky! Basically, allowing media to go back into processing... ie, resizing an image to a new size, re-transcoding video, etc. There are several cases where you may want to reprocess things: if your processing step failed, if you want to resize images to new sizes, if you want to extract new and relevant data from old videos, etc.<br />
<br />
To understand this, please read up on:<br />
<br />
* the [[processing]] system<br />
* The [[MediaTypeRefactor|media type refactor]] page<br />
* [[Feature_Ideas/Reprocessing]]<br />
<br />
'''Possible mentors:''' [[User:Joar|Joar Wandborg]], [[User:Cwebber|Chris Webber]]</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=Feature_Ideas&diff=1455Feature Ideas2014-01-21T15:52:08Z<p>Unhammer: /* Unsorted Ideas */</p>
<hr />
<div>== Introduction ==<br />
<br />
There are many features that one can think of for MediaGoblin. Some should be implemented really soon, because they are needed right now. Other features would be nice to have, but are currently really hard to implement. And finally there are the Feature Ideas that can be classified as "brain storming".<br />
<br />
This wiki page is mostly for long term feature ideas. This specifically means there are no promises that anything listed here will ever happen. It means nobody is currently working on this feature.<br />
<br />
If you have an idea for a new feature, that is not listed here or in the Bug Tracker, please talk to some developers, or add it below in the "Yet Unsorted Ideas" section. If you really think, that your idea is extremely important and needs to be acted upon soon, you could file a bug.<br />
<br />
== The List ==<br />
If there is a bug (closed or open), please link to it.<br />
<br />
=== Access Control ===<br />
* Renaming of an account<br />
** by its user<br />
** by an admin<br />
* Account creation / activation, considering e-mail address<br />
** Expire inactive accounts<br />
** A method of account activation: ask the user to send e-mail message with a specified text from the address they entered, instead of GMG sending a message to that address (which could be somebody else’s).<br />
*** What about faked addresses? If that’s a serious problem for this, could it still be a requirement? Or is it so serious that it would bother malicious misusers least of all people? --[[User:Aleksejrs|Aleksejrs]] 15:17, 7 December 2011 (EST)<br />
** The above allows accepting multiple inactive accounts for the same e-mail address without bothering its owner and without making it difficult for him to register.<br />
*** But accounts could have the same name… --[[User:Aleksejrs|Aleksejrs]] 15:14, 7 December 2011 (EST)<br />
* An account may constitute a part of user's identity, one may want to prohibit creation of a new account with the same name/id by a different person. --[[User:Aleksejrs|Aleksejrs]] ([[User talk:Aleksejrs|talk]]) 09:11, 18 August 2013 (EDT)<br />
<br />
=== Design (UI / HTML) ===<br />
* Point and click theme options: Colors, layout, typefaces, size, thumbnail options, etc.<br />
** Simple settable options to "personalize" the instance without fiddling with templates.<br />
* Responsive base theme<br />
* User Avatar ([http://issues.mediagoblin.org/ticket/499 #499])<br />
<br />
=== Extensions ===<br />
* A link to the page about installed (and not installed?) plugins.<br />
<br />
=== Federation ===<br />
* [[User:Aleksejrs/ideas/federation|Two federation ideas]]<br />
<br />
=== Import/Export ===<br />
* [[Feature_Ideas/Flickr_Import|flickr Import]] ( ≈ The user has the possibility to make backup of your content that can be migrated to another instance of mediagoblin)<br />
<br />
* MediaWiki export<br />
* digiKam plugin (export to mediagoblin)<br />
* Mount GMG as a file system (e.g. FUSE)<br />
* [http://issues.mediagoblin.org/ticket/600 #600: "Liberate my data" (data export button)]<br />
* [http://issues.mediagoblin.org/ticket/398 #398] wontfixed at some early point<br />
<br />
=== Licensing ===<br />
* ([http://issues.mediagoblin.org/ticket/407 FIXED]) Help the site admin comply with the AGPL.<br />
* ([http://issues.mediagoblin.org/ticket/521 FIXED]) Creative Commons choice on user profile. E.g. default license choice for media uploaded by that user.<br />
* “Other (please specify: [ ])” license as an alternative to “All rights reserved”.<br />
<br />
=== Media Processing ===<br />
* [[Feature_Ideas/Reprocessing|Retry media processing]]<br />
** Related: rotating images that have no proper Orientation tags.<br />
<br />
=== Media Types ===<br />
* For non-still-images:<br />
** Duration<br />
** Bitrate<br />
** Format structure (e.g. “Ogg/Theora+Vorbis”)<br />
** Permit pre-encoded uploads to eliminate server-side processing (available as of 0.3.3)<br />
** Support Ogg/Theora player (already implemented?)<br />
** Subtitles for video and audio<br />
* A way to find out what formats can be uploaded.<br />
** E.g.: A simple type list near the upload field.<br />
* For images with transparency, let the uploader specify background (e.g. color or checkerboard). [http://issues.mediagoblin.org/ticket/482 #482 (background color)] wontfix'ed in the past.<br />
<br />
=== Metadata ===<br />
* Ability to write metadata (comments, tags, etc.) to files.<br />
* Copy (some) metadata from the full‐size image into the smaller versions. If possible (according to metadata formats), add a note to them that they are not exactly the original.<br />
** [http://issues.mediagoblin.org/ticket/94 #94]: exif data handling for users (about privacy)<br />
** <del>[http://issues.mediagoblin.org/ticket/284 #284]: Support "Orientation" EXIF tag</del> (fixed)<br />
** EXIF info?<br />
* Display more info about the file on its page<br />
** Resolution of the original<br />
** File size of the original<br />
** Resolution of the scaled-down version<br />
* "trans-tagging": Adding tags to other peoples media [http://issues.mediagoblin.org/ticket/251 #251]<br />
** [[Many images usecase#Crowd tagging/captioning/commenting]]<br />
* Geotagging support. <br />
** Both read and write. <br />
** Display on OpenStreetMap.<br />
* Consider [https://en.wikipedia.org/wiki/Semantic_Web Semantic Web]<br />
** Microformats, RDFa, Microdata<br />
** Consider the privacy implications when choosing what to mark up.<br />
* Add hashes etc, something like magnet links or Metalink, of the files. The information can then be used for downloading, searching, linking to.<br />
** Maybe also optional for privacy.<br />
** Say, a server is loaded a lot, or is short on traffic, so it asks that the visitor use something like Miro to watch the video while/by downloading it via BitTorrent (there is also web seeding, so it may be not a problem if the BitTorrent download doesn't work while the GMG does).<br />
<br />
=== Security ===<br />
* DONE: CSRF ([http://issues.mediagoblin.org/ticket/76 #76])<br />
* <code>X-Content-Type-Options: nosniff</code><br />
*: Served pages have the content-type set. And the browser should not be allowed to guess a different type. See: [https://bugzilla.mozilla.org/show_bug.cgi?id=471020 Firefox bug #471020]<br />
* "Content Security Policy" (CSP) might really be a good add on to have. Noone should rely solely on this, but it might make things a lot safer if other security guards fail.<br />
*: A simple allow 'self' might already get a lot of things better.<br />
*: [https://developer.mozilla.org/en/Security/CSP/Introducing_Content_Security_Policy Link1] [https://developer.mozilla.org/en/Security/CSP/CSP_policy_directives#options Link2]<br />
* Possibly disallowing pages to be shown in frames.<br />
<br />
=== Unsorted Ideas ===<br />
Put your new ideas here:<br />
* [[Search]].<br />
* Related Content<br />
* Generate pages dynamically using files on the file system.<br />
* Like the UI of an ordinary image viewer (e.g. geeqie) or a file manager generates thumbnails (optionally saving them), shows metadata…<br />
* MediaWiki can generate thumbnails of arbitrary size on demand. (hopefully not arbitrarily large?)<br />
* Categorize media as videos, images and audio, put media of each type in a row of the grid<br />
* Offer random image/video/audio option<br />
* Integration with Diaspora<br />
* Facilitate viewing at different resolutions ( quality )<br />
* Icons of software used in the footer with description of what it was used for appearing as popup on hover</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=PluginsTips&diff=1454PluginsTips2014-01-20T14:50:02Z<p>Unhammer: /* Translating / Localising plugins */</p>
<hr />
<div>==Quickstart==<br />
In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get started with plugin creation. The simplest way to get up and running is to <br />
# Follow [[HackingHowto#How_to_set_up_and_maintain_an_environment_for_hacking_with_virtualenv]] to set up a local virtualenv mediagoblin instance <br />
#* run it with ./lazyserver.sh so you get debug output (if you run celeryd separately, you won't see logging.info from stuff run in celery tasks)<br />
# <tt>cp -r mediagoblin/plugins/sampleplugin mediagoblin/plugins/myplugin</tt> (where "myplugin" is your plugin name)<br />
# <tt>cp mediagoblin.ini mediagoblin_local.ini</tt><br />
# edit <tt>mediagoblin_local.ini</tt> and add <nowiki>[[mediagoblin.plugins.myplugin]]</nowiki> under the <nowiki>[plugins]</nowiki> section to enable your plugin<br />
<br />
Now you can look at e.g. other core or non-core plugins for inspiration. See [[Available Plugins]] for non-core plugins.<br />
<br />
<br />
The way you change things in plugins is by use of '''hooks'''. At certain points in the mediagoblin code, a function will say "run all hooks with name XYZ", and if you've defined a hook with such a name in a plugin you've enabled, it'll get run there. The [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/api.html documentation on Plugin API] is your friend for defining hooks. The sample plugin shows a use of the 'setup' hook.<br />
<br />
Note: templates are hooked with a call to pluginapi.register_template_hooks, instead of adding to the hooks variable.<br />
<br />
==Making an installable plugin==<br />
If you followed the steps above, you can use your plugin by copying it into the plugins folder; however, to get a plugin that is easily installable by users (e.g. with <tt>pip install myplugin</tt>), it should have a certain folder layout. <br />
<br />
A good example is the [https://github.com/commonsmachinery/mg-rdfa/ RDFa plugin]. This uses a setup.py file to install the files under the mediagoblin_rdfa folder into the lib/ folder of your mediagoblin installation. If you've checked out both mediagoblin and mg-rdfa in the same folder, e.g. ~/src/, you can do <pre>cd ~/src/mg-rdfa/<br />
../mediagoblin/bin/python setup.py build<br />
../mediagoblin/bin/python setup.py install</pre> to install it to your mediagoblin instance.<br />
<br />
The file layout of the repo is:<br />
<pre><br />
setup.py<br />
MANIFEST.in<br />
README.md<br />
mediagoblin_rdfa/<br />
__init.py__<br />
templates/<br />
mediagoblin/<br />
plugins/<br />
rdfa/<br />
metadata.html<br />
</pre><br />
<br />
The setup.py file defines an option include_package_data=True, which makes it read the file MANIFEST.in; MANIFEST.in contains rules for which files to include when installing. In this case, rule <tt>recursive-include mediagoblin_rdfa *.html</tt> makes it include the HTML template file (and any other you put under mediagoblin_rdfa), retaining the folder layout.<br />
<br />
==Referencing the mediagoblin test set in a plugin==<br />
<br />
The tests in mediagoblin define a nice "test_app" and functions for logging in and posting and so on. You can use this in your installable plugin.<br />
<br />
From your plugin (assuming a layout like mg-rdfa described above), doing<br />
<br />
cp -r ../mediagoblin/mediagoblin/tests .<br />
<br />
lets you run<br />
<br />
../mediagoblin/bin/py.test tests --boxed <br />
<br />
In fact, the only files you need are `conftest.py` and `pytest.ini`, and then you can copy over a single `test_foo.py` (only making sure import statements are absolute) and it should run fine.<br />
<br />
You also need to call setup_plugins() in your setup() function for your plugin to be properly loaded before testing.<br />
<br />
An example of this test method is in the [https://gitorious.org/mediagoblin-stock/mediagoblin-hidden_original/ "hidden original" plugin], which subclasses the submission tests.<br />
<br />
<br />
See also [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/tests.html Writing unit tests for plugins] in the docs.<br />
<br />
== Translating / Localising plugins ==<br />
How do we do [[Translations]] in plugins? At least, a plugin would have to install a .mo file in e.g. lib/python2.7/site-packages/pluginname-0.1.3-py2.7.egg/pluginname/i18n/nn_NO/LC_MESSAGES/pluginname.mo (with domain "pluginname"?). Then mediagoblin would have to either discover that the file exists when setting up the plugin, or the plugin would have to add that to some list in some hook.<br />
<br />
mg_globals.py seems to create a gettext variable pointing at mediagoblin/i18n:<br />
<pre><br />
thread_scope.translations = gettext.translation(<br />
'mediagoblin',<br />
pkg_resources.resource_filename(<br />
'mediagoblin', 'i18n'), ['en'], fallback=True)<br />
</pre><br />
which is installed into the jinja template in template.py:<br />
<pre><br />
template_env.install_gettext_callables(<br />
mg_globals.thread_scope.translations.ugettext,<br />
mg_globals.thread_scope.translations.ungettext)<br />
</pre><br />
in a function called by app.py:<br />
<pre><br />
request.template_env = template.get_jinja_env(<br />
self.template_loader, request.locale)<br />
</pre></div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=Feature_Ideas&diff=1453Feature Ideas2014-01-20T14:08:38Z<p>Unhammer: /* Unsorted Ideas */</p>
<hr />
<div>== Introduction ==<br />
<br />
There are many features that one can think of for MediaGoblin. Some should be implemented really soon, because they are needed right now. Other features would be nice to have, but are currently really hard to implement. And finally there are the Feature Ideas that can be classified as "brain storming".<br />
<br />
This wiki page is mostly for long term feature ideas. This specifically means there are no promises that anything listed here will ever happen. It means nobody is currently working on this feature.<br />
<br />
If you have an idea for a new feature, that is not listed here or in the Bug Tracker, please talk to some developers, or add it below in the "Yet Unsorted Ideas" section. If you really think, that your idea is extremely important and needs to be acted upon soon, you could file a bug.<br />
<br />
== The List ==<br />
If there is a bug (closed or open), please link to it.<br />
<br />
=== Access Control ===<br />
* Renaming of an account<br />
** by its user<br />
** by an admin<br />
* Account creation / activation, considering e-mail address<br />
** Expire inactive accounts<br />
** A method of account activation: ask the user to send e-mail message with a specified text from the address they entered, instead of GMG sending a message to that address (which could be somebody else’s).<br />
*** What about faked addresses? If that’s a serious problem for this, could it still be a requirement? Or is it so serious that it would bother malicious misusers least of all people? --[[User:Aleksejrs|Aleksejrs]] 15:17, 7 December 2011 (EST)<br />
** The above allows accepting multiple inactive accounts for the same e-mail address without bothering its owner and without making it difficult for him to register.<br />
*** But accounts could have the same name… --[[User:Aleksejrs|Aleksejrs]] 15:14, 7 December 2011 (EST)<br />
* An account may constitute a part of user's identity, one may want to prohibit creation of a new account with the same name/id by a different person. --[[User:Aleksejrs|Aleksejrs]] ([[User talk:Aleksejrs|talk]]) 09:11, 18 August 2013 (EDT)<br />
<br />
=== Design (UI / HTML) ===<br />
* Point and click theme options: Colors, layout, typefaces, size, thumbnail options, etc.<br />
** Simple settable options to "personalize" the instance without fiddling with templates.<br />
* Responsive base theme<br />
* User Avatar ([http://issues.mediagoblin.org/ticket/499 #499])<br />
<br />
=== Extensions ===<br />
* A link to the page about installed (and not installed?) plugins.<br />
<br />
=== Federation ===<br />
* [[User:Aleksejrs/ideas/federation|Two federation ideas]]<br />
<br />
=== Import/Export ===<br />
* [[Feature_Ideas/Flickr_Import|flickr Import]] ( ≈ The user has the possibility to make backup of your content that can be migrated to another instance of mediagoblin)<br />
<br />
* MediaWiki export<br />
* digiKam plugin (export to mediagoblin)<br />
* Mount GMG as a file system (e.g. FUSE)<br />
* [http://issues.mediagoblin.org/ticket/600 #600: "Liberate my data" (data export button)]<br />
* [http://issues.mediagoblin.org/ticket/398 #398] wontfixed at some early point<br />
<br />
=== Licensing ===<br />
* ([http://issues.mediagoblin.org/ticket/407 FIXED]) Help the site admin comply with the AGPL.<br />
* ([http://issues.mediagoblin.org/ticket/521 FIXED]) Creative Commons choice on user profile. E.g. default license choice for media uploaded by that user.<br />
* “Other (please specify: [ ])” license as an alternative to “All rights reserved”.<br />
<br />
=== Media Processing ===<br />
* [[Feature_Ideas/Reprocessing|Retry media processing]]<br />
** Related: rotating images that have no proper Orientation tags.<br />
<br />
=== Media Types ===<br />
* For non-still-images:<br />
** Duration<br />
** Bitrate<br />
** Format structure (e.g. “Ogg/Theora+Vorbis”)<br />
** Permit pre-encoded uploads to eliminate server-side processing (available as of 0.3.3)<br />
** Support Ogg/Theora player (already implemented?)<br />
** Subtitles for video and audio<br />
* A way to find out what formats can be uploaded.<br />
** E.g.: A simple type list near the upload field.<br />
* For images with transparency, let the uploader specify background (e.g. color or checkerboard). [http://issues.mediagoblin.org/ticket/482 #482 (background color)] wontfix'ed in the past.<br />
<br />
=== Metadata ===<br />
* Ability to write metadata (comments, tags, etc.) to files.<br />
* Copy (some) metadata from the full‐size image into the smaller versions. If possible (according to metadata formats), add a note to them that they are not exactly the original.<br />
** [http://issues.mediagoblin.org/ticket/94 #94]: exif data handling for users (about privacy)<br />
** <del>[http://issues.mediagoblin.org/ticket/284 #284]: Support "Orientation" EXIF tag</del> (fixed)<br />
** EXIF info?<br />
* Display more info about the file on its page<br />
** Resolution of the original<br />
** File size of the original<br />
** Resolution of the scaled-down version<br />
* "trans-tagging": Adding tags to other peoples media [http://issues.mediagoblin.org/ticket/251 #251]<br />
** [[Many images usecase#Crowd tagging/captioning/commenting]]<br />
* Geotagging support. <br />
** Both read and write. <br />
** Display on OpenStreetMap.<br />
* Consider [https://en.wikipedia.org/wiki/Semantic_Web Semantic Web]<br />
** Microformats, RDFa, Microdata<br />
** Consider the privacy implications when choosing what to mark up.<br />
* Add hashes etc, something like magnet links or Metalink, of the files. The information can then be used for downloading, searching, linking to.<br />
** Maybe also optional for privacy.<br />
** Say, a server is loaded a lot, or is short on traffic, so it asks that the visitor use something like Miro to watch the video while/by downloading it via BitTorrent (there is also web seeding, so it may be not a problem if the BitTorrent download doesn't work while the GMG does).<br />
<br />
=== Security ===<br />
* DONE: CSRF ([http://issues.mediagoblin.org/ticket/76 #76])<br />
* <code>X-Content-Type-Options: nosniff</code><br />
*: Served pages have the content-type set. And the browser should not be allowed to guess a different type. See: [https://bugzilla.mozilla.org/show_bug.cgi?id=471020 Firefox bug #471020]<br />
* "Content Security Policy" (CSP) might really be a good add on to have. Noone should rely solely on this, but it might make things a lot safer if other security guards fail.<br />
*: A simple allow 'self' might already get a lot of things better.<br />
*: [https://developer.mozilla.org/en/Security/CSP/Introducing_Content_Security_Policy Link1] [https://developer.mozilla.org/en/Security/CSP/CSP_policy_directives#options Link2]<br />
* Possibly disallowing pages to be shown in frames.<br />
<br />
=== Unsorted Ideas ===<br />
Put your new ideas here:<br />
* Search.<br />
* Related Content<br />
* Generate pages dynamically using files on the file system.<br />
* Like the UI of an ordinary image viewer (e.g. geeqie) or a file manager generates thumbnails (optionally saving them), shows metadata…<br />
* MediaWiki can generate thumbnails of arbitrary size on demand. (hopefully not arbitrarily large?)<br />
* Categorize media as videos, images and audio, put media of each type in a row of the grid<br />
* Offer random image/video/audio option<br />
* Integration with Diaspora<br />
* Facilitate viewing at different resolutions ( quality )<br />
* Icons of software used in the footer with description of what it was used for appearing as popup on hover</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=PluginsTips&diff=1452PluginsTips2014-01-20T13:44:58Z<p>Unhammer: /* Translating / Localising plugins */</p>
<hr />
<div>==Quickstart==<br />
In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get started with plugin creation. The simplest way to get up and running is to <br />
# Follow [[HackingHowto#How_to_set_up_and_maintain_an_environment_for_hacking_with_virtualenv]] to set up a local virtualenv mediagoblin instance <br />
#* run it with ./lazyserver.sh so you get debug output (if you run celeryd separately, you won't see logging.info from stuff run in celery tasks)<br />
# <tt>cp -r mediagoblin/plugins/sampleplugin mediagoblin/plugins/myplugin</tt> (where "myplugin" is your plugin name)<br />
# <tt>cp mediagoblin.ini mediagoblin_local.ini</tt><br />
# edit <tt>mediagoblin_local.ini</tt> and add <nowiki>[[mediagoblin.plugins.myplugin]]</nowiki> under the <nowiki>[plugins]</nowiki> section to enable your plugin<br />
<br />
Now you can look at e.g. other core or non-core plugins for inspiration. See [[Available Plugins]] for non-core plugins.<br />
<br />
<br />
The way you change things in plugins is by use of '''hooks'''. At certain points in the mediagoblin code, a function will say "run all hooks with name XYZ", and if you've defined a hook with such a name in a plugin you've enabled, it'll get run there. The [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/api.html documentation on Plugin API] is your friend for defining hooks. The sample plugin shows a use of the 'setup' hook.<br />
<br />
Note: templates are hooked with a call to pluginapi.register_template_hooks, instead of adding to the hooks variable.<br />
<br />
==Making an installable plugin==<br />
If you followed the steps above, you can use your plugin by copying it into the plugins folder; however, to get a plugin that is easily installable by users (e.g. with <tt>pip install myplugin</tt>), it should have a certain folder layout. <br />
<br />
A good example is the [https://github.com/commonsmachinery/mg-rdfa/ RDFa plugin]. This uses a setup.py file to install the files under the mediagoblin_rdfa folder into the lib/ folder of your mediagoblin installation. If you've checked out both mediagoblin and mg-rdfa in the same folder, e.g. ~/src/, you can do <pre>cd ~/src/mg-rdfa/<br />
../mediagoblin/bin/python setup.py build<br />
../mediagoblin/bin/python setup.py install</pre> to install it to your mediagoblin instance.<br />
<br />
The file layout of the repo is:<br />
<pre><br />
setup.py<br />
MANIFEST.in<br />
README.md<br />
mediagoblin_rdfa/<br />
__init.py__<br />
templates/<br />
mediagoblin/<br />
plugins/<br />
rdfa/<br />
metadata.html<br />
</pre><br />
<br />
The setup.py file defines an option include_package_data=True, which makes it read the file MANIFEST.in; MANIFEST.in contains rules for which files to include when installing. In this case, rule <tt>recursive-include mediagoblin_rdfa *.html</tt> makes it include the HTML template file (and any other you put under mediagoblin_rdfa), retaining the folder layout.<br />
<br />
==Referencing the mediagoblin test set in a plugin==<br />
<br />
The tests in mediagoblin define a nice "test_app" and functions for logging in and posting and so on. You can use this in your installable plugin.<br />
<br />
From your plugin (assuming a layout like mg-rdfa described above), doing<br />
<br />
cp -r ../mediagoblin/mediagoblin/tests .<br />
<br />
lets you run<br />
<br />
../mediagoblin/bin/py.test tests --boxed <br />
<br />
In fact, the only files you need are `conftest.py` and `pytest.ini`, and then you can copy over a single `test_foo.py` (only making sure import statements are absolute) and it should run fine.<br />
<br />
You also need to call setup_plugins() in your setup() function for your plugin to be properly loaded before testing.<br />
<br />
An example of this test method is in the [https://gitorious.org/mediagoblin-stock/mediagoblin-hidden_original/ "hidden original" plugin], which subclasses the submission tests.<br />
<br />
<br />
See also [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/tests.html Writing unit tests for plugins] in the docs.<br />
<br />
== Translating / Localising plugins ==<br />
How do we do this? At least, a plugin would have to install a .mo file in e.g. lib/python2.7/site-packages/pluginname-0.1.3-py2.7.egg/pluginname/i18n/nn_NO/LC_MESSAGES/pluginname.mo (with domain "pluginname"?). Then mediagoblin would have to either discover that the file exists when setting up the plugin, or the plugin would have to add that to some list in some hook.<br />
<br />
mg_globals.py seems to create a gettext variable pointing at mediagoblin/i18n:<br />
<pre><br />
thread_scope.translations = gettext.translation(<br />
'mediagoblin',<br />
pkg_resources.resource_filename(<br />
'mediagoblin', 'i18n'), ['en'], fallback=True)<br />
</pre><br />
which is installed into the jinja template in template.py:<br />
<pre><br />
template_env.install_gettext_callables(<br />
mg_globals.thread_scope.translations.ugettext,<br />
mg_globals.thread_scope.translations.ungettext)<br />
</pre><br />
in a function called by app.py:<br />
<pre><br />
request.template_env = template.get_jinja_env(<br />
self.template_loader, request.locale)<br />
</pre></div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=PluginsTips&diff=1451PluginsTips2014-01-20T13:16:19Z<p>Unhammer: /* Referencing the mediagoblin test set in a plugin */</p>
<hr />
<div>==Quickstart==<br />
In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get started with plugin creation. The simplest way to get up and running is to <br />
# Follow [[HackingHowto#How_to_set_up_and_maintain_an_environment_for_hacking_with_virtualenv]] to set up a local virtualenv mediagoblin instance <br />
#* run it with ./lazyserver.sh so you get debug output (if you run celeryd separately, you won't see logging.info from stuff run in celery tasks)<br />
# <tt>cp -r mediagoblin/plugins/sampleplugin mediagoblin/plugins/myplugin</tt> (where "myplugin" is your plugin name)<br />
# <tt>cp mediagoblin.ini mediagoblin_local.ini</tt><br />
# edit <tt>mediagoblin_local.ini</tt> and add <nowiki>[[mediagoblin.plugins.myplugin]]</nowiki> under the <nowiki>[plugins]</nowiki> section to enable your plugin<br />
<br />
Now you can look at e.g. other core or non-core plugins for inspiration. See [[Available Plugins]] for non-core plugins.<br />
<br />
<br />
The way you change things in plugins is by use of '''hooks'''. At certain points in the mediagoblin code, a function will say "run all hooks with name XYZ", and if you've defined a hook with such a name in a plugin you've enabled, it'll get run there. The [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/api.html documentation on Plugin API] is your friend for defining hooks. The sample plugin shows a use of the 'setup' hook.<br />
<br />
Note: templates are hooked with a call to pluginapi.register_template_hooks, instead of adding to the hooks variable.<br />
<br />
==Making an installable plugin==<br />
If you followed the steps above, you can use your plugin by copying it into the plugins folder; however, to get a plugin that is easily installable by users (e.g. with <tt>pip install myplugin</tt>), it should have a certain folder layout. <br />
<br />
A good example is the [https://github.com/commonsmachinery/mg-rdfa/ RDFa plugin]. This uses a setup.py file to install the files under the mediagoblin_rdfa folder into the lib/ folder of your mediagoblin installation. If you've checked out both mediagoblin and mg-rdfa in the same folder, e.g. ~/src/, you can do <pre>cd ~/src/mg-rdfa/<br />
../mediagoblin/bin/python setup.py build<br />
../mediagoblin/bin/python setup.py install</pre> to install it to your mediagoblin instance.<br />
<br />
The file layout of the repo is:<br />
<pre><br />
setup.py<br />
MANIFEST.in<br />
README.md<br />
mediagoblin_rdfa/<br />
__init.py__<br />
templates/<br />
mediagoblin/<br />
plugins/<br />
rdfa/<br />
metadata.html<br />
</pre><br />
<br />
The setup.py file defines an option include_package_data=True, which makes it read the file MANIFEST.in; MANIFEST.in contains rules for which files to include when installing. In this case, rule <tt>recursive-include mediagoblin_rdfa *.html</tt> makes it include the HTML template file (and any other you put under mediagoblin_rdfa), retaining the folder layout.<br />
<br />
==Referencing the mediagoblin test set in a plugin==<br />
<br />
The tests in mediagoblin define a nice "test_app" and functions for logging in and posting and so on. You can use this in your installable plugin.<br />
<br />
From your plugin (assuming a layout like mg-rdfa described above), doing<br />
<br />
cp -r ../mediagoblin/mediagoblin/tests .<br />
<br />
lets you run<br />
<br />
../mediagoblin/bin/py.test tests --boxed <br />
<br />
In fact, the only files you need are `conftest.py` and `pytest.ini`, and then you can copy over a single `test_foo.py` (only making sure import statements are absolute) and it should run fine.<br />
<br />
You also need to call setup_plugins() in your setup() function for your plugin to be properly loaded before testing.<br />
<br />
An example of this test method is in the [https://gitorious.org/mediagoblin-stock/mediagoblin-hidden_original/ "hidden original" plugin], which subclasses the submission tests.<br />
<br />
<br />
See also [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/tests.html Writing unit tests for plugins] in the docs.<br />
<br />
== Translating / Localising plugins ==<br />
: How do we do this?</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=PluginsTips&diff=1450PluginsTips2014-01-19T12:09:22Z<p>Unhammer: /* Referencing the mediagoblin test set in a plugin */</p>
<hr />
<div>==Quickstart==<br />
In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get started with plugin creation. The simplest way to get up and running is to <br />
# Follow [[HackingHowto#How_to_set_up_and_maintain_an_environment_for_hacking_with_virtualenv]] to set up a local virtualenv mediagoblin instance <br />
#* run it with ./lazyserver.sh so you get debug output (if you run celeryd separately, you won't see logging.info from stuff run in celery tasks)<br />
# <tt>cp -r mediagoblin/plugins/sampleplugin mediagoblin/plugins/myplugin</tt> (where "myplugin" is your plugin name)<br />
# <tt>cp mediagoblin.ini mediagoblin_local.ini</tt><br />
# edit <tt>mediagoblin_local.ini</tt> and add <nowiki>[[mediagoblin.plugins.myplugin]]</nowiki> under the <nowiki>[plugins]</nowiki> section to enable your plugin<br />
<br />
Now you can look at e.g. other core or non-core plugins for inspiration. See [[Available Plugins]] for non-core plugins.<br />
<br />
<br />
The way you change things in plugins is by use of '''hooks'''. At certain points in the mediagoblin code, a function will say "run all hooks with name XYZ", and if you've defined a hook with such a name in a plugin you've enabled, it'll get run there. The [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/api.html documentation on Plugin API] is your friend for defining hooks. The sample plugin shows a use of the 'setup' hook.<br />
<br />
Note: templates are hooked with a call to pluginapi.register_template_hooks, instead of adding to the hooks variable.<br />
<br />
==Making an installable plugin==<br />
If you followed the steps above, you can use your plugin by copying it into the plugins folder; however, to get a plugin that is easily installable by users (e.g. with <tt>pip install myplugin</tt>), it should have a certain folder layout. <br />
<br />
A good example is the [https://github.com/commonsmachinery/mg-rdfa/ RDFa plugin]. This uses a setup.py file to install the files under the mediagoblin_rdfa folder into the lib/ folder of your mediagoblin installation. If you've checked out both mediagoblin and mg-rdfa in the same folder, e.g. ~/src/, you can do <pre>cd ~/src/mg-rdfa/<br />
../mediagoblin/bin/python setup.py build<br />
../mediagoblin/bin/python setup.py install</pre> to install it to your mediagoblin instance.<br />
<br />
The file layout of the repo is:<br />
<pre><br />
setup.py<br />
MANIFEST.in<br />
README.md<br />
mediagoblin_rdfa/<br />
__init.py__<br />
templates/<br />
mediagoblin/<br />
plugins/<br />
rdfa/<br />
metadata.html<br />
</pre><br />
<br />
The setup.py file defines an option include_package_data=True, which makes it read the file MANIFEST.in; MANIFEST.in contains rules for which files to include when installing. In this case, rule <tt>recursive-include mediagoblin_rdfa *.html</tt> makes it include the HTML template file (and any other you put under mediagoblin_rdfa), retaining the folder layout.<br />
<br />
==Referencing the mediagoblin test set in a plugin==<br />
<br />
The tests in mediagoblin define a nice "test_app" and functions for logging in and posting and so on. You can use this in your installable plugin.<br />
<br />
From your plugin (assuming a layout like mg-rdfa described above), doing<br />
<br />
cp -r ../mediagoblin/mediagoblin/tests .<br />
<br />
lets you run<br />
<br />
../mediagoblin/bin/py.test tests --boxed <br />
<br />
In fact, the only files you need are `conftest.py` and `pytest.ini`, and then you can copy over a single `test_foo.py` (only making sure import statements are absolute) and it should run fine.<br />
<br />
You also need to call setup_plugins() in your setup() function for your plugin to be properly loaded before testing.<br />
<br />
An example of this test method is in the [https://gitorious.org/mediagoblin-stock/mediagoblin-hidden_original/ "hidden original" plugin], which subclasses the submission tests.<br />
<br />
<br />
See also [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/tests.html Writing unit tests for plugins] in the docs.</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=Git_workflow&diff=1449Git workflow2014-01-17T20:52:10Z<p>Unhammer: /* What happens next */</p>
<hr />
<div>GNU MediaGoblin uses git for all our version control and we have the<br />
repositories hosted on [http://gitorious.org/ Gitorious]. We have<br />
two repositories:<br />
<br />
* MediaGoblin software: http://gitorious.org/mediagoblin/mediagoblin<br />
* MediaGoblin website: http://gitorious.org/mediagoblin/mediagoblin-website<br />
<br />
It's most likely you want to look at the software repository -- not the<br />
website one.<br />
<br />
The rest of this chapter talks about using the software repository.<br />
<br />
The short of it is: we do not use merge requests. Instead, create a "feature branch" in git that you push somewhere, and link to it on a ticket. Details below.<br />
<br />
= How to clone the project =<br />
<br />
Do::<br />
<br />
git clone git://gitorious.org/mediagoblin/mediagoblin.git<br />
<br />
<br />
= How to contribute changes =<br />
<br />
== Tie your changes to issues in the issue tracker ==<br />
<br />
All patches should be tied to issues in the<br />
[http://issues.mediagoblin.org/ issue tracker].<br />
That makes it a lot easier for everyone to track proposed changes and<br />
make sure your hard work doesn't get dropped on the floor! If there<br />
isn't an issue for what you're working on, please create one. The<br />
better the description of what it is you're trying to fix/implement,<br />
the better everyone else is able to understand why you're doing what<br />
you're doing.<br />
<br />
== Use bugfix branches to make changes ==<br />
<br />
The best way to isolate your changes is to create a branch based off<br />
of the MediaGoblin repository master branch, do the changes related to<br />
that one issue there, and then let us know how to get it.<br />
<br />
It's much easier on us if you isolate your changes to a branch focused<br />
on the issue. Then we don't have to sift through things.<br />
<br />
It's much easier on you if you isolate your changes to a branch<br />
focused on the issue. Then when we merge your changes in, you just<br />
have to do a {{Cmd|git fetch}} and that's it. This is especially true if<br />
we reject some of your changes, but accept others or otherwise tweak<br />
your changes.<br />
<br />
Further, if you isolate your changes to a branch, then you can work on<br />
multiple issues at the same time and they don't conflict with one<br />
another.<br />
<br />
Name your branches using the isue number and something that makes it clear<br />
what it's about. For example, if you were working on tagging, you<br />
might name your branch "360_tagging".<br />
<br />
== Properly document your changes ==<br />
<br />
Include comments in the code.<br />
<br />
Write comprehensive commit messages. The better your commit message<br />
is at describing what you did and why, the easier it is for us to<br />
quickly accept your patch.<br />
<br />
Write comprehensive comments in the issue tracker about what you're<br />
doing and why.<br />
<br />
== How to send us your changes ==<br />
<br />
There are two ways to let us know how to get it:<br />
<br />
=== push changes to publicly available git clone and let us know where to find it ===<br />
<br />
''This is the preferred method of sending changes.''<br />
<br />
Push your feature/bugfix/issue branch to your publicly available<br />
git clone and add a comment to the issue with the url for your<br />
clone and the branch to look at.<br />
<br />
=== attaching the patch files to the issue ===<br />
<br />
Run<br />
<br />
{{Cmd|git format-patch --stdout <remote>/master > issue_<number>.patch}}<br />
<br />
<tt>format-patch</tt> creates a patch of all the commits that are in<br />
your branch that aren't in <tt><remote>/master</tt>. The <tt>--stdout</tt><br />
flag causes all this output to go to stdout where it's redirected<br />
to a file named <tt>issue_<number>.patch</tt>. That file should be<br />
based on the issue you're working with. For example,<br />
<tt>issue_42.patch</tt> is a good filename and <tt>issue_42_rev2.patch</tt><br />
is good if you did a revision of it.<br />
<br />
Having said all that, the filename isn't wildly important.<br />
<br />
<br />
= Example workflow =<br />
<br />
Here's an example workflow.<br />
<br />
== Contributing changes ==<br />
<br />
Slartibartfast from the planet Magrathea far off in the universe has<br />
decided that he is bored with fjords and wants to fix issue 42 (the<br />
meaning of life bug) and send us the changes.<br />
<br />
Slartibartfast has cloned the MediaGoblin repository and his clone<br />
lives on gitorious.<br />
<br />
Slartibartfast works locally. The remote named ``origin`` points to<br />
his clone on gitorious. The remote named ``gmg`` points to the<br />
MediaGoblin repository.<br />
<br />
Slartibartfast does the following:<br />
<br />
1. Fetches the latest from the MediaGoblin repository::<br />
<br />
git fetch --all -p<br />
<br />
This tells <tt>git fetch</tt> to fetch all the recent data from all of<br />
the remotes (<tt>--all</tt>) and prune any branches that have been<br />
deleted in the remotes (<tt>-p</tt>).<br />
<br />
2. Creates a branch from the tip of the MediaGoblin repository (the remote is named <tt>gmg</tt>) master branch called <tt>bug42_meaning_of_life</tt>:<br />
<br />
git checkout -b bug42_meaning_of_life gmg/master<br />
<br />
This creates a new branch (<tt>-b</tt>) named <tt>bug42_meaning_of_life</tt> based<br />
on the tip of the <tt>master</tt> branch of the remote named <tt>gmg</tt> and checks<br />
it out.<br />
<br />
3. Slartibartfast works hard on his changes in the <tt>bug42_meaning_of_life</tt> branch. When done, he wants to notify us that he has made changes he wants us to see.<br />
<br />
4. Slartibartfast pushes his changes to his clone::<br />
<br />
git push origin bug42_meaning_of_life --set-upstream<br />
<br />
This pushes the changes in the <tt>bug42_meaning_of_life</tt> branch to the remote named <tt>origin</tt>.<br />
<br />
5. Slartibartfast adds a comment to issue 42 with the url for his repository and the name of the branch he put the code in. He also explains what he did and why it addresses the issue.<br />
<br />
== Updating a contribution ==<br />
<br />
Slartibartfast brushes his hands off with the sense of accomplishment<br />
that comes with the knowledge of a job well done. He stands, wanders<br />
over to get a cup of water, then realizes that he forgot to run the<br />
unit tests!<br />
<br />
He runs the unit tests and discovers there's a bug in the code!<br />
<br />
Then he does this:<br />
<br />
1. He checks out the <tt>bug42_meaning_of_life</tt> branch::<br />
<br />
git checkout bug42_meaning_of_life<br />
<br />
2. He fixes the bug and checks it into the <tt>bug42_meaning_of_life</tt> branch.<br />
<br />
3. He pushes his changes to his clone (the remote is named <tt>origin</tt>):<br />
<br />
git push origin bug42_meaning_of_life<br />
<br />
4. He adds another comment to issue 42 explaining about the mistake and how he fixed it and that he's pushed the new change to the <tt>bug42_meaning_of_life</tt> branch of his publicly available clone.<br />
<br />
== What happens next ==<br />
<br />
Slartibartfast is once again happy with his work. He finds issue 42<br />
in the issue tracker and adds a comment saying he submitted a merge<br />
request with his changes and explains what they are.<br />
: "merge request"? https://gitorious.org/mediagoblin says "We don't use no stinking merge requests"<br />
<br />
Later, someone checks out his code and finds a problem with it. He<br />
adds a comment to the issue tracker specifying the problem and asks<br />
Slartibartfast to fix it. Slartibartfst goes through the above steps<br />
again, fixes the issue, pushes it to his<br />
<tt>bug42_meaning_of_life</tt> branch and adds another comment to the<br />
issue tracker about how he fixed it.<br />
<br />
Later, someone checks out his code and is happy with it. Someone<br />
pulls it into the master branch of the MediaGoblin repository and adds<br />
another comment to the issue and probably closes the issue out.<br />
<br />
Slartibartfast is notified of this. Slartibartfast does a:<br />
<br />
git fetch --all<br />
<br />
The changes show up in the <tt>master</tt> branch of the <tt>gmg</tt> remote.<br />
Slartibartfast now deletes his <tt>bug42_meaning_of_life</tt> branch<br />
because he doesn't need it anymore.<br />
<br />
= How to learn git =<br />
<br />
[[BeginnersCorner#Learning_git]]</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=PluginsTips&diff=1448PluginsTips2014-01-17T19:57:45Z<p>Unhammer: /* Referencing the mediagoblin test set in a plugin */</p>
<hr />
<div>==Quickstart==<br />
In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get started with plugin creation. The simplest way to get up and running is to <br />
# Follow [[HackingHowto#How_to_set_up_and_maintain_an_environment_for_hacking_with_virtualenv]] to set up a local virtualenv mediagoblin instance <br />
#* run it with ./lazyserver.sh so you get debug output (if you run celeryd separately, you won't see logging.info from stuff run in celery tasks)<br />
# <tt>cp -r mediagoblin/plugins/sampleplugin mediagoblin/plugins/myplugin</tt> (where "myplugin" is your plugin name)<br />
# <tt>cp mediagoblin.ini mediagoblin_local.ini</tt><br />
# edit <tt>mediagoblin_local.ini</tt> and add <nowiki>[[mediagoblin.plugins.myplugin]]</nowiki> under the <nowiki>[plugins]</nowiki> section to enable your plugin<br />
<br />
Now you can look at e.g. other core or non-core plugins for inspiration. See [[Available Plugins]] for non-core plugins.<br />
<br />
<br />
The way you change things in plugins is by use of '''hooks'''. At certain points in the mediagoblin code, a function will say "run all hooks with name XYZ", and if you've defined a hook with such a name in a plugin you've enabled, it'll get run there. The [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/api.html documentation on Plugin API] is your friend for defining hooks. The sample plugin shows a use of the 'setup' hook.<br />
<br />
Note: templates are hooked with a call to pluginapi.register_template_hooks, instead of adding to the hooks variable.<br />
<br />
==Making an installable plugin==<br />
If you followed the steps above, you can use your plugin by copying it into the plugins folder; however, to get a plugin that is easily installable by users (e.g. with <tt>pip install myplugin</tt>), it should have a certain folder layout. <br />
<br />
A good example is the [https://github.com/commonsmachinery/mg-rdfa/ RDFa plugin]. This uses a setup.py file to install the files under the mediagoblin_rdfa folder into the lib/ folder of your mediagoblin installation. If you've checked out both mediagoblin and mg-rdfa in the same folder, e.g. ~/src/, you can do <pre>cd ~/src/mg-rdfa/<br />
../mediagoblin/bin/python setup.py build<br />
../mediagoblin/bin/python setup.py install</pre> to install it to your mediagoblin instance.<br />
<br />
The file layout of the repo is:<br />
<pre><br />
setup.py<br />
MANIFEST.in<br />
README.md<br />
mediagoblin_rdfa/<br />
__init.py__<br />
templates/<br />
mediagoblin/<br />
plugins/<br />
rdfa/<br />
metadata.html<br />
</pre><br />
<br />
The setup.py file defines an option include_package_data=True, which makes it read the file MANIFEST.in; MANIFEST.in contains rules for which files to include when installing. In this case, rule <tt>recursive-include mediagoblin_rdfa *.html</tt> makes it include the HTML template file (and any other you put under mediagoblin_rdfa), retaining the folder layout.<br />
<br />
==Referencing the mediagoblin test set in a plugin==<br />
<br />
The tests in mediagoblin define a nice "test_app" and functions for logging in and posting and so on. You can use this in your installable plugin.<br />
<br />
From your plugin (assuming a layout like mg-rdfa described above), doing<br />
<br />
cp -r ../mediagoblin/mediagoblin/tests .<br />
<br />
lets you run<br />
<br />
../mediagoblin/bin/py.test tests --boxed <br />
<br />
In fact, the only files you need are `conftest.py` and `pytest.ini`, and then you can copy over a single `test_foo.py` (only making sure import statements are absolute) and it should run fine.<br />
<br />
You also need to call setup_plugins() in your setup() function for your plugin to be properly loaded before testing.<br />
<br />
An example of this test method is in the [https://gitorious.org/mediagoblin-stock/mediagoblin-hidden_original/ "hidden original" plugin], which subclasses the submission tests.</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=PluginsTips&diff=1447PluginsTips2014-01-17T19:56:10Z<p>Unhammer: /* Making an installable plugin */</p>
<hr />
<div>==Quickstart==<br />
In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get started with plugin creation. The simplest way to get up and running is to <br />
# Follow [[HackingHowto#How_to_set_up_and_maintain_an_environment_for_hacking_with_virtualenv]] to set up a local virtualenv mediagoblin instance <br />
#* run it with ./lazyserver.sh so you get debug output (if you run celeryd separately, you won't see logging.info from stuff run in celery tasks)<br />
# <tt>cp -r mediagoblin/plugins/sampleplugin mediagoblin/plugins/myplugin</tt> (where "myplugin" is your plugin name)<br />
# <tt>cp mediagoblin.ini mediagoblin_local.ini</tt><br />
# edit <tt>mediagoblin_local.ini</tt> and add <nowiki>[[mediagoblin.plugins.myplugin]]</nowiki> under the <nowiki>[plugins]</nowiki> section to enable your plugin<br />
<br />
Now you can look at e.g. other core or non-core plugins for inspiration. See [[Available Plugins]] for non-core plugins.<br />
<br />
<br />
The way you change things in plugins is by use of '''hooks'''. At certain points in the mediagoblin code, a function will say "run all hooks with name XYZ", and if you've defined a hook with such a name in a plugin you've enabled, it'll get run there. The [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/api.html documentation on Plugin API] is your friend for defining hooks. The sample plugin shows a use of the 'setup' hook.<br />
<br />
Note: templates are hooked with a call to pluginapi.register_template_hooks, instead of adding to the hooks variable.<br />
<br />
==Making an installable plugin==<br />
If you followed the steps above, you can use your plugin by copying it into the plugins folder; however, to get a plugin that is easily installable by users (e.g. with <tt>pip install myplugin</tt>), it should have a certain folder layout. <br />
<br />
A good example is the [https://github.com/commonsmachinery/mg-rdfa/ RDFa plugin]. This uses a setup.py file to install the files under the mediagoblin_rdfa folder into the lib/ folder of your mediagoblin installation. If you've checked out both mediagoblin and mg-rdfa in the same folder, e.g. ~/src/, you can do <pre>cd ~/src/mg-rdfa/<br />
../mediagoblin/bin/python setup.py build<br />
../mediagoblin/bin/python setup.py install</pre> to install it to your mediagoblin instance.<br />
<br />
The file layout of the repo is:<br />
<pre><br />
setup.py<br />
MANIFEST.in<br />
README.md<br />
mediagoblin_rdfa/<br />
__init.py__<br />
templates/<br />
mediagoblin/<br />
plugins/<br />
rdfa/<br />
metadata.html<br />
</pre><br />
<br />
The setup.py file defines an option include_package_data=True, which makes it read the file MANIFEST.in; MANIFEST.in contains rules for which files to include when installing. In this case, rule <tt>recursive-include mediagoblin_rdfa *.html</tt> makes it include the HTML template file (and any other you put under mediagoblin_rdfa), retaining the folder layout.<br />
<br />
==Referencing the mediagoblin test set in a plugin==<br />
<br />
The tests in mediagoblin define a nice "test_app" and functions for logging in and posting and so on. You can use this in your installable plugin.<br />
<br />
From your plugin (assuming a layout like mg-rdfa described above), doing<br />
<br />
cp -r ../mediagoblin/mediagoblin/tests .<br />
<br />
lets you run<br />
<br />
../mediagoblin/bin/py.test tests --boxed <br />
<br />
In fact, the only files you need are `conftest.py` and `pytest.ini`, and then you can copy over a single `test_foo.py` (only making sure import statements are absolute) and it should run fine.<br />
<br />
You also need to call setup_plugins() in your setup function for your plugin to be properly loaded before testing.</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=PluginsTips&diff=1446PluginsTips2014-01-17T19:53:22Z<p>Unhammer: /* Quickstart */</p>
<hr />
<div>==Quickstart==<br />
In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get started with plugin creation. The simplest way to get up and running is to <br />
# Follow [[HackingHowto#How_to_set_up_and_maintain_an_environment_for_hacking_with_virtualenv]] to set up a local virtualenv mediagoblin instance <br />
#* run it with ./lazyserver.sh so you get debug output (if you run celeryd separately, you won't see logging.info from stuff run in celery tasks)<br />
# <tt>cp -r mediagoblin/plugins/sampleplugin mediagoblin/plugins/myplugin</tt> (where "myplugin" is your plugin name)<br />
# <tt>cp mediagoblin.ini mediagoblin_local.ini</tt><br />
# edit <tt>mediagoblin_local.ini</tt> and add <nowiki>[[mediagoblin.plugins.myplugin]]</nowiki> under the <nowiki>[plugins]</nowiki> section to enable your plugin<br />
<br />
Now you can look at e.g. other core or non-core plugins for inspiration. See [[Available Plugins]] for non-core plugins.<br />
<br />
<br />
The way you change things in plugins is by use of '''hooks'''. At certain points in the mediagoblin code, a function will say "run all hooks with name XYZ", and if you've defined a hook with such a name in a plugin you've enabled, it'll get run there. The [http://mediagoblin.readthedocs.org/en/latest/pluginwriter/api.html documentation on Plugin API] is your friend for defining hooks. The sample plugin shows a use of the 'setup' hook.<br />
<br />
Note: templates are hooked with a call to pluginapi.register_template_hooks, instead of adding to the hooks variable.<br />
<br />
==Making an installable plugin==<br />
If you followed the steps above, you can use your plugin by copying it into the plugins folder; however, to get a plugin that is easily installable by users (e.g. with <tt>pip install myplugin</tt>), it should have a certain folder layout. <br />
<br />
A good example is the [https://github.com/commonsmachinery/mg-rdfa/ RDFa plugin]. This uses a setup.py file to install the files under the mediagoblin_rdfa folder into the lib/ folder of your mediagoblin installation. If you've checked out both mediagoblin and mg-rdfa in the same folder, e.g. ~/src/, you can do <pre>cd ~/src/mg-rdfa/<br />
../mediagoblin/bin/python setup.py build<br />
../mediagoblin/bin/python setup.py install</pre> to install it to your mediagoblin instance.<br />
<br />
The file layout of the repo is:<br />
<pre><br />
setup.py<br />
MANIFEST.in<br />
README.md<br />
mediagoblin_rdfa/<br />
__init.py__<br />
templates/<br />
mediagoblin/<br />
plugins/<br />
rdfa/<br />
metadata.html<br />
</pre><br />
<br />
The setup.py file defines an option include_package_data=True, which makes it read the file MANIFEST.in; MANIFEST.in contains rules for which files to include when installing. In this case, rule <tt>recursive-include mediagoblin_rdfa *.html</tt> makes it include the HTML template file (and any other you put under mediagoblin_rdfa), retaining the folder layout.</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=PluginsTips&diff=1445PluginsTips2014-01-17T19:47:52Z<p>Unhammer: /* Making an installable plugin */</p>
<hr />
<div>==Quickstart==<br />
In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get started with plugin creation. The simplest way to get up and running is to <br />
# Follow [[HackingHowto#How_to_set_up_and_maintain_an_environment_for_hacking_with_virtualenv]] to set up a local virtualenv mediagoblin instance <br />
#* run it with ./lazyserver.sh so you get debug output (if you run celeryd separately, you won't see logging.info from stuff run in celery tasks)<br />
# <tt>cp -r mediagoblin/plugins/sampleplugin mediagoblin/plugins/myplugin</tt> (where "myplugin" is your plugin name)<br />
# <tt>cp mediagoblin.ini mediagoblin_local.ini</tt><br />
# edit <tt>mediagoblin_local.ini</tt> and add <nowiki>[[mediagoblin.plugins.myplugin]]</nowiki> under the <nowiki>[plugins]</nowiki> section to enable your plugin<br />
<br />
Now you can look at e.g. other core or non-core plugins for inspiration. See [[Available Plugins]] for non-core plugins.<br />
<br />
==Making an installable plugin==<br />
If you followed the steps above, you can use your plugin by copying it into the plugins folder; however, to get a plugin that is easily installable by users (e.g. with <tt>pip install myplugin</tt>), it should have a certain folder layout. <br />
<br />
A good example is the [https://github.com/commonsmachinery/mg-rdfa/ RDFa plugin]. This uses a setup.py file to install the files under the mediagoblin_rdfa folder into the lib/ folder of your mediagoblin installation. If you've checked out both mediagoblin and mg-rdfa in the same folder, e.g. ~/src/, you can do <pre>cd ~/src/mg-rdfa/<br />
../mediagoblin/bin/python setup.py build<br />
../mediagoblin/bin/python setup.py install</pre> to install it to your mediagoblin instance.<br />
<br />
The file layout of the repo is:<br />
<pre><br />
setup.py<br />
MANIFEST.in<br />
README.md<br />
mediagoblin_rdfa/<br />
__init.py__<br />
templates/<br />
mediagoblin/<br />
plugins/<br />
rdfa/<br />
metadata.html<br />
</pre><br />
<br />
The setup.py file defines an option include_package_data=True, which makes it read the file MANIFEST.in; MANIFEST.in contains rules for which files to include when installing. In this case, rule <tt>recursive-include mediagoblin_rdfa *.html</tt> makes it include the HTML template file (and any other you put under mediagoblin_rdfa), retaining the folder layout.</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=PluginsTips&diff=1444PluginsTips2014-01-17T19:39:36Z<p>Unhammer: </p>
<hr />
<div>==Quickstart==<br />
In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get started with plugin creation. The simplest way to get up and running is to <br />
# Follow [[HackingHowto#How_to_set_up_and_maintain_an_environment_for_hacking_with_virtualenv]] to set up a local virtualenv mediagoblin instance <br />
#* run it with ./lazyserver.sh so you get debug output (if you run celeryd separately, you won't see logging.info from stuff run in celery tasks)<br />
# <tt>cp -r mediagoblin/plugins/sampleplugin mediagoblin/plugins/myplugin</tt> (where "myplugin" is your plugin name)<br />
# <tt>cp mediagoblin.ini mediagoblin_local.ini</tt><br />
# edit <tt>mediagoblin_local.ini</tt> and add <nowiki>[[mediagoblin.plugins.myplugin]]</nowiki> under the <nowiki>[plugins]</nowiki> section to enable your plugin<br />
<br />
Now you can look at e.g. other core or non-core plugins for inspiration. See [[Available Plugins]] for non-core plugins.<br />
<br />
==Making an installable plugin==<br />
If you followed the steps above, you can use your plugin by copying it into the plugins folder; however, to get a plugin that is easily installable by users (e.g. with <tt>pip install myplugin</tt>), it should have a certain folder layout. A good example is the [https://github.com/commonsmachinery/mg-rdfa/ RDFa plugin]. This uses a setup.py file to install the files under the mediagoblin_rdfa folder into the lib/ folder of your mediagoblin installation. If you've checked out both mediagoblin and mg-rdfa in the same folder, e.g. ~/src/, you can do <pre>cd ~/src/mg-rdfa/<br />
../mediagoblin/bin/python setup.py build<br />
../mediagoblin/bin/python setup.py install</pre> to install it to your mediagoblin instance.n</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=PluginsTips&diff=1443PluginsTips2014-01-17T19:35:20Z<p>Unhammer: </p>
<hr />
<div>==Quickstart==<br />
In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get started with plugin creation. The simplest way to get up and running is to <br />
# Follow [[HackingHowto#How_to_set_up_and_maintain_an_environment_for_hacking_with_virtualenv]] to set up a local mediagoblin instance<br />
#* run it with ./lazyserver.sh so you get debug output (if you run celeryd separately, you won't see logging.info from stuff run in celery tasks)<br />
# <tt>cp -r mediagoblin/plugins/sampleplugin mediagoblin/plugins/myplugin</tt> (where "myplugin" is your plugin name)<br />
# <tt>cp mediagoblin.ini mediagoblin_local.ini</tt><br />
# edit <tt>mediagoblin_local.ini</tt> and add <nowiki>[[mediagoblin.plugins.myplugin]]</nowiki> under the <nowiki>[plugins]</nowiki> section to enable your plugin<br />
<br />
Now you can look at e.g. other core or non-core plugins for inspiration. See [[Available Plugins]] for non-core plugins.<br />
<br />
==Making an installable plugin==<br />
If you followed the steps above, you can use your plugin by copying it into the plugins folder; however, to get a plugin that is easily installable by users (e.g. with <tt>pip install myplugin</tt>), it should have a certain folder layout. A good example is the [https://github.com/commonsmachinery/mg-rdfa/ RDFa plugin].</div>Unhammerhttps://wiki.mediagoblin.org/index.php?title=PluginsTips&diff=1442PluginsTips2014-01-17T19:34:06Z<p>Unhammer: Created page with "==Quickstart== In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get starte..."</p>
<hr />
<div>==Quickstart==<br />
In the [https://gitorious.org/mediagoblin mediagoblin repository], there's a sample plugin under mediagoblin/plugins/sampleplugin that you can use to get started with plugin creation. The simplest way to get up and running is to <br />
# Follow [[HackingHowto#How_to_set_up_and_maintain_an_environment_for_hacking_with_virtualenv]] to set up a local mediagoblin instance<br />
#* run it with ./lazyserver.sh so you get debug output (if you run celeryd separately, you won't see logging.info from stuff run in celery tasks)<br />
# <tt>cp -r mediagoblin/plugins/sampleplugin mediagoblin/plugins/myplugin</tt> (where "myplugin" is your plugin name)<br />
# <tt>cp mediagoblin.ini mediagoblin_local.ini</tt><br />
# edit <tt>mediagoblin_local.ini</tt> and add [[mediagoblin.plugins.myplugin]] to enable your plugin<br />
<br />
Now you can look at e.g. other core or non-core plugins for inspiration. See [[Available Plugins]] for non-core plugins.<br />
<br />
==Making an installable plugin==<br />
If you followed the steps above, you can use your plugin by copying it into the plugins folder; however, to get a plugin that is easily installable by users (e.g. with <tt>pip install myplugin</tt>), it should have a certain folder layout. A good example is the [https://github.com/commonsmachinery/mg-rdfa/ RDFa plugin].</div>Unhammer