Multiple media support

The general idea

So, MediaGoblin supporting multiple media types! This has been thought about in the designs since the beginning. But what does that mean?

As far as I can tell, we need to support multiple media types in a few core ways:

• Submitting: Is there an extra step for a user to select what media type they have? Should there be an extra step? Should we figure it out for users? If so what does that mean when two media handlers can handle the same type of media, but differently?
• Processing: One of the most important aspects, we should be able to define special functions to process the media. Each media might need to be resized/transcoded... generally processed!... in different ways. A certain amount of error handling will happen here too.
• Displaying: How to display the media? Videos and images clearly don't share the same display code.
• Feeds: An exciting, perhaps surprising aspect of multiple media types is that we can specify different file types to be available to media views.
• Filtering: If a user wants to look at just-video, shouldn't we let them?
• Per-media metadata: We'll need to store different extra data about each type of media. Where to put that?

The structure of a Media Manager

Unlike the StorageInterface, each media type won't be a subclass of some abstracted class... instead, it will be a dictionary instance. That's because we don't really need individual instnaces of these things... they just point to stuff.

The base MEDIA_MANAGER dict looks like:

from mediagoblin.media_types.image.processing import start_process

MEDIA_MANAGER = {
    "human_readable": "Image",
    "processor": start_process, # alternately a string,
                                # 'mediagoblin.media_types.image.processing'?
    "display_template": "mediagoblin/media_displays/image.html",
    "default_thumb": "images/media_thumbs/image.jpg",
    "accepted_extensions": ["jpg", "jpeg", "png", "gif", "tiff"],
    "accepted_mimetypes": [
        "image/jpeg", "image/png", "image/gif", "image/tiff"]}

Possible extra parameters can be figured out as needed... for now we'll also assume "extra_templates_dir" is a possibility of sorts. That's explained later.

The MEDIA_MANAGER variable is expected to be accessable at the toplevel of some submodule. For example, our MediaEntry might have type:

 'media_type': 'mediagoblin.media_types.image'

Seeing this, our handlers expect to find the MEDIA_MANAGER at:

 mediagoblin.media_types.image.MEDIA_MANAGER

This is looked at for all future directives here.

An examble subdirectory layout for this might looks something like:

1. mediagoblin/media_types/image/__init__.py contains MEDIA_MANAGER, possibly a few other things
2. mediagoblin/media_types/image/processing.py contains most processing-relevant code
3. mediagoblin/media_types/image/errors.py Do we have image-specific errors? Could put them here. See Processing documentation on how we handle failures.
4. mediagoblin/media_types/image/util.py Maybe we have some generally fun and useful utilities we could dump in here.

... or we could just dump it all in mediagoblin/media_types/image/__init__.py ;)

Also in this particular case, since bundled with mediagoblin, putting templates in the right place probably easy:

• mediagoblin/templates/mediagoblin/media_displays/image.html

... see the Display section for more on this.

Also there should be some sort of meta media manager which collects information on all activated media managers and routes things appropriately... maybe? :)

Submission

How does submission work?

Not too much here, basically we could either let users submit whatever and auto-figure-out what media type it is. Maybe later. For now if there are multiple media types enabled, we'll take the easier (for us) route and give a dropdown asking for what media type it is.

Anyway, so you submitted, what next?

First, check to see if we have a registered and enabled media manager for the type selected. If not, throw an error. If so, check that this media manager actually supports the mimetype or extension given. No need to open and actually do a file integrity check... that happens next in processing. Anyway, if not supported from this loose check, throw an error here.

Otherwise, mark with the parent submodule of the MEDIA_MANAGER (so like, mediagoblin.media_types.image) in media_type and pass it off to processing.

Processing

So, processing!

mediagoblin.process_media:ProcessMedia().run() has code that looks like so:

       # Try to process, and handle expected errors.
       try:
           process_image(entry)
       except BaseProcessingFail, exc:
           mark_entry_failed(entry[u'_id'], exc)
           return

Instead of just passing off to process_image, we should find the MEDIA_MANAGER associated with the media_type specified in the MediaEntry. Then we should pass in the entry to the 'processor' function specified there.

So, this processor function should do a few things.

• Check the integrity of the media (might happen just as a side effect of trying to process it)
• Process the media

If for some reason the media fails to open in some reasonable way, we should run it through the failure system. See mediagoblin.process_media.errors for more info, possibly also look at the current image processing code as an example of how to raise these problems. Oh yeah, and read the processing docs on the wiki. :)

At this point, er... process the media. Make a workbench if you like. Convert the media if necessary, move it to the public store.

One thing that maybe should be changed is that the queued_file doesn't need to be wiped by the processor function... every processor will need to do that. It can be wiped by the ProcessMedia.run() method after successful execution.

Display

Display might be easier than you might think!

Basically we can take advantage of Jinja2's template inheritance stuff.

We should figure out what template to display based on the value of 'display_template' in the relevant MEDIA_MANAGER. Load that template!

Basically, we should refactor the current user_pages/media.html template and put a {% block media_display %}{% endblock %} where the current media display'ing gets done.

Then our template from display_template should extend this base template and fill in the media_display block.

That should make things nice and easy and extensible. We could even give a block on the sidebar for other media-specific stuff to be dumped in there, too. :)

(What would we do in the event that we have several external-package media types registered? I guess we'd have to extend the templating loader to be able to load from their template paths.)

Thumbnails

When you save your things to the public_store you should also record them in the 'media_files' section of the MediaEntry.

You're welcome to put anything you want in any key, excepting that one is reserved... thumb :) You should make a maximum-width 180 by maximum-width 180 thumbnail and put this here.

Sometimes thumbnails might not be able to be extracted, or might not be appropriate. It would be a good idea to include a "default" thumbnail then. Provide a path to a file that can be staticdirect'ed.

(... This is all good and well for multimedia types that come bundled with mediagoblin, but I have no idea where we'll put stuff from external packages that can't be staticdirected. We'll have to think about this more later.)

Media metadata

You might need to store some media-specific information with your mediaentry. Where to put it?

Luckily, that's already been thought out. See the 'media_data' key in the mediagoblin documentation:

    - media_data: Extra information that's media-format-dependent.
      For example, images might contain some EXIF data that's not appropriate
      to other formats.  You might store it like:

        mediaentry['media_data']['exif'] = {
            'manufacturer': 'CASIO',
            'model': 'QV-4000',
            'exposure_time': .659}

      Alternately for video you might store:

        # play length in seconds
        mediaentry['media_data']['play_length'] = 340

Enabling / disabling media types for an application

To start with, we could just make it so that MEDIA_MANAGERs which are registered are always enabled. But maybe we have a few videos in the system, but later on we decided we don't want videos but want to keep the old ones up, or etc. So maybe we deserve separate lists in the config file for which media types are enabled? Not sure.