View source for Multiple media support

= The general idea =

So, MediaGoblin supporting multiple media types!  This has been
thought about in the designs since the beginning.  But what does that
mean?

As far as I can tell, we need to support multiple media types in a few
core ways:

* '''Submitting:''' Is there an extra step for a user to select what media type they have?  ''Should'' there be an extra step?  Should we figure it out for users?  If so what does that mean when two media handlers can handle the same type of media, but differently?
* '''Processing:''' One of the most important aspects, we should be able to define special functions to process the media.  Each media might need to be resized/transcoded... generally processed!... in different ways.  A certain amount of error handling will happen here too.
* '''Displaying:''' How to display the media?  Videos and images clearly don't share the same display code.
* '''Feeds:''' An exciting, perhaps surprising aspect of multiple media types is that we can specify different file types to be available to media views.
* '''Filtering:''' If a user wants to look at just-video, shouldn't we let them?
* '''Per-media metadata:''' We'll need to store different extra data about each type of media.  Where to put that?

= The structure of a Media Manager =

Unlike the StorageInterface, each media type won't be a subclass of
some abstracted class... instead, it will be a dictionary instance.
That's because we don't really need individual instnaces of these
things... they just point to stuff.

The base MEDIA_MANAGER dict looks like:

 from mediagoblin.media_types.image.processing import start_process
 
 MEDIA_MANAGER = {
     "human_readable": "Image",
     "processor": start_process, # alternately a string,
                                 # 'mediagoblin.media_types.image.processing'?
     "display_template": "mediagoblin/media_displays/image.html",
     "default_thumb": "images/media_thumbs/image.jpg",
     "accepted_extensions": ["jpg", "jpeg", "png", "gif", "tiff"],
     "accepted_mimetypes": [
         "image/jpeg", "image/png", "image/gif", "image/tiff"]}

Possible extra parameters can be figured out as needed... for now
we'll also assume "extra_templates_dir" is a possibility of sorts.
That's explained later.

The MEDIA_MANAGER variable is expected to be accessable at the
toplevel of some submodule.  For example, our MediaEntry might have
type:

  'media_type': 'mediagoblin.media_types.image'

Seeing this, our handlers expect to find the MEDIA_MANAGER at:

  mediagoblin.media_types.image.MEDIA_MANAGER

This is looked at for all future directives here.

An examble subdirectory layout for this might looks something like:

#mediagoblin/media_types/image/__init__.py contains MEDIA_MANAGER, possibly a few other things
#mediagoblin/media_types/image/processing.py contains most processing-relevant code
#mediagoblin/media_types/image/errors.py Do we have image-specific errors?  Could put them here.  See [[Processing]] documentation on how we handle failures.
#mediagoblin/media_types/image/util.py Maybe we have some generally fun and useful utilities we could dump in here.

... or we could just dump it all in
mediagoblin/media_types/image/__init__.py ;)

Also in this particular case, since bundled with mediagoblin, putting
templates in the right place probably easy:
* mediagoblin/templates/mediagoblin/media_displays/image.html

... see the Display section for more on this.

Also there should be some sort of meta media manager which collects
information on all activated media managers and routes things
appropriately... maybe? :)


= Submission =

How does submission work?

Not too much here, basically we could either let users submit whatever
and auto-figure-out what media type it is.  Maybe later.  For now if
there are multiple media types enabled, we'll take the easier (for us)
route and give a dropdown asking for what media type it is.

Anyway, so you submitted, what next?

First, check to see if we have a registered and enabled media manager
for the type selected.  If not, throw an error.  If so, check that
this media manager actually supports the mimetype or extension given.
No need to open and actually do a file integrity check... that happens
next in processing.  Anyway, if not supported from this loose check,
throw an error here.

Otherwise, mark with the parent submodule of the MEDIA_MANAGER (so
like, mediagoblin.media_types.image) in media_type and pass it off to
processing.



= Processing =

So, processing!

mediagoblin.process_media:ProcessMedia().run() has code that looks
like so:

        # Try to process, and handle expected errors.
        try:
            process_image(entry)
        except BaseProcessingFail, exc:
            mark_entry_failed(entry[u'_id'], exc)
            return

Instead of just passing off to process_image, we should find the
MEDIA_MANAGER associated with the media_type specified in the
MediaEntry.  Then we should pass in the entry to the 'processor'
function specified there.

So, this processor function should do a few things.
* Check the integrity of the media (might happen just as a side effect of trying to process it)
* Process the media

If for some reason the media fails to open in some reasonable way, we
should run it through the failure system.  See
mediagoblin.process_media.errors for more info, possibly also look at
the current image processing code as an example of how to raise these
problems.  Oh yeah, and read the [[Processing]] docs on the wiki. :)

At this point, er... process the media.  Make a workbench if you
like.  Convert the media if necessary, move it to the public store.

One thing that maybe should be changed is that the queued_file doesn't
need to be wiped by the processor function... every processor will
need to do that.  It can be wiped by the ProcessMedia.run() method
after successful execution.

= Display =

Display might be easier than you might think!

Basically we can take advantage of Jinja2's template inheritance
stuff.

We should figure out what template to display based on the
value of 'display_template' in the relevant MEDIA_MANAGER.  Load that
template!

Basically, we should refactor the current user_pages/media.html
template and put a {% block media_display %}{% endblock %} where the
current media display'ing gets done.

Then our template from display_template should extend this base
template and fill in the media_display block.

That should make things nice and easy and extensible.  We could even
give a block on the sidebar for other media-specific stuff to be
dumped in there, too. :)

(What would we do in the event that we have several external-package
media types registered?  I guess we'd have to extend the templating
loader to be able to load from their template paths.)

= Thumbnails =

When you save your things to the public_store you should also record
them in the 'media_files' section of the MediaEntry.

You're welcome to put anything you want in any key, excepting that one
is reserved... thumb :) You should make a maximum-width 180 by
maximum-width 180 thumbnail and put this here.

Sometimes thumbnails might not be able to be extracted, or might not
be appropriate.  It would be a good idea to include a "default"
thumbnail then.  Provide a path to a file that can be staticdirect'ed.

(... This is all good and well for multimedia types that come bundled
with mediagoblin, but I have no idea where we'll put stuff from
external packages that can't be staticdirected.  We'll have to think
about this more later.)

= Media metadata =

You might need to store some media-specific information with your
mediaentry.  Where to put it?

Luckily, that's already been thought out.  See the 'media_data' key in
the mediagoblin documentation:

     - media_data: Extra information that's media-format-dependent.
       For example, images might contain some EXIF data that's not appropriate
       to other formats.  You might store it like:
     
         mediaentry['media_data']['exif'] = {
             'manufacturer': 'CASIO',
             'model': 'QV-4000',
             'exposure_time': .659}
     
       Alternately for video you might store:
     
         # play length in seconds
         mediaentry['media_data']['play_length'] = 340

= Enabling / disabling media types for an application =

To start with, we could just make it so that MEDIA_MANAGERs which are
registered are always enabled.  But maybe we have a few videos in the
system, but later on we decided we don't want videos but want to keep
the old ones up, or etc.  So maybe we deserve separate lists in the
config file for which media types are enabled?  Not sure.