= Media Types Design =

This document is based on requirements/wishes listed in http://wiki.mediagoblin.org/MediaTypeRefactor

*Note*: * Attachments have deliberately been excluded from this document to reduce complexity

= Publishing clients =

These are described optimal behaviours of different types of publishing clients.

== CLI ==

=== Single-file ===

{{{
gmg submit FILE
}}}

=== *n*-file ===

{{{
gmg submit FILE_1 FILE_2 [...] FILE_N
}}}

=== *n*-file with media metadata ===

GStreamer `gst-launch` inspired exclamation-mark-separated "elements" of media entries:

{{{
gmg submit FILE_1 title=TITLE_1 description=DESCRIPTION_1 ! \
FILE_2 title=TITLE_2 ! \
FILE_3 title=TITLE_3 tags=TAGS_3[...]
}}}

== WWW ==

=== *n*-file ===

*Note*: Requires browser [http://www.whatwg.org/specs/web-apps/current-work/multipage/dnd.html#dnd drag and drop] support.

{{{
,-----------------------------------------------.
| S U B M I T F I L E S |
| |
| Drag media from your desktop |
| and drop it here |
| |
`-----------------------------------------------Â´
}}}

*Note*: * What if there's folders in there?
* What about publishing-metadata?

=== Single-file ===

Could be used as fallback for browsers not supporting [http://www.whatwg.org/specs/web-apps/current-work/multipage/dnd.html#dnd drag and drop].

{{{
,-----------------------------------------------.
| S U B M I T F I L E |
| |
| Source file: [ /home/joar/Pictures... Browse] |
| |
| [Submit] |
`-----------------------------------------------Â´
}}}

== API ==

TBD

= Media types =

== Installation ==

{{{
pip install mediagoblin-video
echo "[[mediagoblin.media_types.video]]" >> mediagoblin_local.ini
gmg dbupdate
}}}

*Note*: * We could build a wrapper around pip that performs the installation steps automatically.
* What about dependencies such as GStreamer and related packages? We could include that in our wrapper, but it'd have to take into account all the different distributions' package managers out there.

== Negotiation ==

Media type negotiation is done per-file by iterating through a registry of enabled media type plugins. Several negotiation mechanisms may exists, such as a cheap file-extension based one and a more expensive file content sniffing negotiation step.

== Output files ==

Some media types such as ASCII art and images might have a very simple data structure:

{{{
This illustrates the output of the ``mediagoblin.media_types.image`` media
type:

ORIGINAL IMAGE -> THUMBNAIL JPEG
MEDIUM SIZED JPEG
}}}

Media types such as `video` and `3D object` might have more complex output files:

{{{
ORIGINAL VIDEO -> THUMBNAIL JPEG
640px-wide WEBM # Displayed on media page
720p WEBM # Displayed in full-screen player
1080p WEBM # Displayed in full-screen player
640px-wide H.264 # Alternate container/stream formats
720p H.264
1080p H.264
}}}

we should therefore allow a lot of flexibility for the files

= Media processing =

Processing takes the original file and creates files of lesser size/quality that The pipeline is as follows:

# The application receives a bytestream, optional byte stream metadata (i.e. filename) and publishing-metadata. The bytestream and bytestream metadata is sourced from the *original file*.
* Possibly we could have "adapters" of some sort that provides this data consistently across publishing clients.
*Note*: Some users have expressed a wish for mediagoblin to simply read a folder with files and create a "web album" from them.

* The media type negotiation mechanism decides one or more suitable media type plugins to handle the conversion of the *original file*, pushes task data to the task broker and creates a *MediaEntry* in the database and populates it with at least the media type.
* The task handler now checks the task broker for a new task when received the task handler looks up the processing method for the selected media type and runs it.
* The media processor converts the *original file* into smaller, displayable media. These media files are then placed in the *public storage*.

*Note*: * Are there cases where we should accept more than one input file?

2012-11-03T11:24:54Z

Joar:

This is the proposed default usecase for newly deployed GNU MediaGoblin applications.

== Gavroche ==

Gavroche is an independent artist living in a suburb of Paris, France. Gavroche creates audible, text, visual and moving pictures often under the CC-BY-SA license.

Gavroche is ''currently'' a GNOME3 user. Gavroche uses desktop applications such as

* Blender, for animations and VFX
* Shotwell/digiKam, for digital photo development
* GIMP, for photo editing
* Vim/Emacs/$FOO

Gavroche would like to be able to submit stuff directly from any of these applications, but he should not be required to launch any of them to be able to upload anything.

<blockquote>
<p>Desktop application integration is certainly possible sine the API. The main limitation would be the desktop application itself.</p>
--[[User:Joar|Joar]] ([[User talk:Joar|talk]]) 07:24, 3 November 2012 (EDT)
</blockquote>

=== Submitting from a recent browser ===

* Gavroche uses Firefox or Chromium/Chrome, he might use IE10, but that's a rare occasion. He avoids IE <= 9 because he has a feeling that they might not be usable everywhere on the web.

User:Joar/Proposed usecase

2012-11-03T09:50:03Z

Joar:

User:Joar/Proposed usecase

2012-11-03T09:49:47Z

Joar:

2012-08-05T22:39:58Z

Joar:

[https://gobblin.se gobblin.se] ''(previously mg.wandborg.se)'' - GNU MediaGoblin master branch instance managed & owned by [[User:Joar|Joar]]

== Registration ==
* Registration on the mg.wandborg.se instance is open.

== Uptime ==
mg.wandborg.se is hosted on a linode 512 instance and will be online as long as there are no necessary extra costs. Details about the linode 512 plan can be found at the bottom of http://www.linode.com/.

As of 2011-11-06 22:31 CET the linode 512 plan includes

* 20 GB data storage
* 200 GB monthly bandwidth ''[http://blog.linode.com/2011/09/01/linode-reduces-transfer-pricing/ Only outbound]''
* 512 MB RAM

I have email alerts for CPU, bandwidth and IO enabled.

=== Usage as of 2011-11-06 22:31 CET ===

* 2.3 GB data storage (including the OS)
* 321 MB bandwidth
* Load: mostly idle

=== Usage as of 2012-03-29 12:40 CET ===

* 7.9 GB data storage (including the OS)
* 7.49 GB outgoing bandwidth (current month)
* 2.53 GB incoming bandwidth (current month)
* RAM is an issue. ~438/512M RAM and almost 255M swap full.
* Load: 0.06 0.11 0.13

=== Usage as of 2012-08-06 ===

* RAM has been upgraded to 1024M

=== Persistance of data ===
The data on the instance is backed with linode's backup service.

Database migrations provided by the master branch will run whenever I pull new code from the MediaGoblin master.

I reserve the right to remove the mg.wandborg.se instance.

I reserve the right to moderate the content of the mg.wandborg.se instance.

In the case that the instance is removed, an environment dump of the entire instance will be provided via this page in a .torrent file. The environment dump will be cleaned from sensitive information such as password hashes and email addresses.

== Abuse ==
To report abuse, contact me via http://wandborg.se

[[User:Joar|Joar]] 16:43, 6 November 2011 (EST)

2012-06-23T00:14:25Z

Joar:

Media types are plugin-like

== File structure ==

<pre>
mediagoblin/media_types/video
├── __init__.py # Contains the MEDIA_MANAGER
├── migrations.py # DB migrations
├── models.py # DB models
├── processing.py # De-facto standard main file
└── transcoders.py # Additional python models for video
# specifically, name it appropriately
# for your media type.
</pre>

== MEDIA_MANAGER ==

The media manager contains metadata about the media type,
along with a reference to the method used to process the media.

<pre>
MEDIA_MANAGER = {
"human_readable": "Video",
"processor": process_video,
"sniff_handler": sniff_handler,
"display_template": "mediagoblin/media_displays/video.html",
"default_thumb": "images/media_thumbs/video.jpg",
"accepted_extensions": [
"mp4", "mov", "webm", "avi", "3gp", "3gpp", "mkv", "ogv", "m4v"]}
</pre>

==== human_readable ====

A human-readable name for the media type.

==== processor ====

A reference to the media processing method for the media type.

==== sniff_handler ====

A reference to the sniffing handler for the media type.

==== display_template ====

A path to the [[Media Types/Display|display template]], relative to <code>mediagoblin/templates</code>.

==== defailt_thumb ====

No current use.

==== accepted_extensions ====

A list containing file extensions that should be associated to this media type. This is used as a fast way to get the media type based on an uploaded file before MediaGoblin falls back to sniffing.

== Negotiating media type based on a file upload ==

To decide which of the available media types that should handle a file upload, we first match the file extension to extensions in the available [[Media Types#MEDIA_MANAGER|MEDIA_MANAGERS]], then we resort to sniffing.

=== Sniffing ===

As a second step in media type negotiation, if the file-extension based lookup fails, <code>mediagoblin.media_types.get_media_type_and_manager</code> will call <code>mediagoblin.media_types.sniff_media</code>, which will iterate and run all the enabled media types's sniffing handlers.

The sniffing handlers can respond with either true or false, responding with true indicates that the media type believes it can parse the file.

This is mediagoblin.media_types.video.processing.sniff_handler.

<pre>
def sniff_handler(media_file, **kw):
transcoder = transcoders.VideoTranscoder()
data = transcoder.discover(media_file.name)

_log.debug('Discovered: {0}'.format(data))

if not data:
_log.error('Could not discover {0}'.format(
kw.get('media')))
return False

if data['is_video'] == True:
return True

return False
</pre>

Media Types

2012-06-22T22:33:28Z

Joar: /* File structure */

Media Types/Display

2012-06-22T22:32:42Z

Joar:

Media type display pages are media type-specific
[http://jinja.pocoo.org/docs/ Jinja2] templates defined in the media type's <code>MEDIA_MANAGER['display_template']</code>.

== Set the template path ==

The path for the media type display template is configured in
the <code>MEDIA_MANAGER</code> object for each media type.
The <code>MEDIA_MANAGER</code> is configured in
<code>(mediagoblin/)media_types/$MEDIA_TYPE/__init__.py</code>.

* The path to the template should be relative to the <code>mediagoblin/templates/</code> directory.
* The standard directory for media templates is <code>mediagoblin/templates/mediagoblin/media_displays</code>

This is an example of a <code>MEDIA_MANAGER</code> dict, see the <code>display_template</code> key.

MEDIA_MANAGER = {
"human_readable": "Video",
"processor": process_video,
"sniff_handler": sniff_handler,
"display_template": "mediagoblin/media_displays/video.html",
"default_thumb": "images/media_thumbs/video.jpg",
"accepted_extensions": [
"mp4", "mov", "webm", "avi", "3gp", "3gpp", "mkv", "ogv", "m4v"]}

== Create the template ==

The medie templates are [http://jinja.pocoo.org/docs/ Jinja2]
templates that extend upon the
<code>mediagoblin/templates/mediagoblin/user_pages/media.html</code>
template. They contain media-type-specifig markup used to display
the media.

This is an example of such a template.

<pre>
{#
# GNU MediaGoblin -- federated, autonomous media hosting
# Copyright (C) 2011, 2012 MediaGoblin contributors. See AUTHORS.
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
#
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#}

{% extends 'mediagoblin/user_pages/media.html' %}

{% block mediagoblin_head %}
{{ super() }}
<script type="text/javascript"
src="{{ request.staticdirect('/js/extlib/video-js/video.js') }}"></script>
<link href="{{ request.staticdirect('/css/vjs-mg-skin.css') }}" rel="stylesheet">
{% endblock %}

{% block mediagoblin_media %}
<div class="video-player" style="position: relative;">
<video class="video-js vjs-mg-skin"
width="{{ media.media_data.width }}"
height="{{ media.media_data.height }}"
controls="controls"
preload="metadata"
data-setup="">
<source src="{{ request.app.public_store.file_url(
media.media_files['webm_640']) }}"
type="video/webm; codecs="vp8, vorbis"" />
<div class="no_html5">
{%- trans -%}Sorry, this video will not work because
your web browser does not support HTML5
video.{%- endtrans -%}<br/>
{%- trans -%}You can get a modern web browser that
can play this video at <a href="http://getfirefox.com">
http://getfirefox.com</a>!{%- endtrans -%}
</div>
</video>
</div>
{% if 'original' in media.media_files %}
<p>
<a href="{{ request.app.public_store.file_url(
media.media_files['original']) }}">
{%- trans -%}
Original
{%- endtrans -%}
</a>
</p>
{% endif %}
{% endblock %}
</pre>