API: Difference between revisions

From GNU MediaGoblin Wiki
Jump to navigation Jump to search
(Created a page with a proposed start for the MediaGoblin API)
 
m (→‎Possible design problems.: fullstop & spacing ; punctuation.)
 
(7 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This page describes a JSON-based API for MediaGoblin. It is a '''WORK IN PROGRESS''' at the moment while it is being drafted before it gets implemented. Currently it only includes a submission API, based on a specific scenario which might not work in general for MediaGoblin but can be used as a springboard for further API improvements and specifications.
This page describes a JSON-based API for MediaGoblin. It is a '''WORK IN PROGRESS''' at the moment while it is being drafted before it gets implemented. Currently it only includes a submission API, based on a specific scenario which might not work in general for MediaGoblin but can be used as a springboard for further API improvements and specifications.


= Scenario =
= Scenario. =


A MediaGoblin server (''MGS'') is used for media storage/repository on behalf of a few other servers, called user servers (''US'') . A user of one ''US'' submits a multimedia file. The file is sent to the ''MGS'' and categorized on the ''US'' in some server specific way. All future use of the multimedia file on the ''US'' is processed by the ''MGS''.
A MediaGoblin server (''MGS'') is used for media storage/repository on behalf of a few other servers, called user servers (''US'') . A user of one ''US'' submits a multimedia file. The file is sent to the ''MGS'' and categorized on the ''US'' in some server specific way. All future use of the multimedia file on the ''US'' is processed by the ''MGS''.
Line 7: Line 7:
The collection of multimedia files from all of the ''US'''s is accessible through the ''MGS'' interface and through specific categorizations on each ''US'' via API or direct links to the content on the ''MGS''.
The collection of multimedia files from all of the ''US'''s is accessible through the ''MGS'' interface and through specific categorizations on each ''US'' via API or direct links to the content on the ''MGS''.


== Submission ==
== Submission. ==


Since all files are processed automatically when they are uploaded there is no need for a specific submission api. Media files should therefore just be uploaded using a normal HTTP POST which is made available via the API using some sort of an authentication mechanism. An extra variable, containing a webhook/callback URL, can be sent alongside the file upload. The callback provides a URL to the file in the public store. The ''MGS'' allows the ''US'' to send metadata (for example license information) for the file.
For submission files are sent directly to the ''MGS'' for processing. Each ''US'' has to claim the uploaded multimedia file to start processing. The process is as follows:

The process is as follows:


# User fills in a form on a ''US''. The form is in fact based up of two forms
# User fills in a form on a ''US''. The form is in fact based up of two forms
#* One for categorization on the ''US''
#* One for categorization on the ''US''
#* One is a file form for the ''MGS''
#* One is a file form for the ''MGS''
# File is sent to the ''MGS'' with a unique name
#* The name distinction should be designed to ensure uniqueness (perhaps uuid?)
# ''US'' saves a pending object with the categorization
# ''US'' saves a pending object with the categorization
# ''US'' claims the file through the unique name via an API which start the processing of the file
# The ''MGS'' processes the file as usual
# The ''MGS'' processes the file as usual
# When the ''MGS'' has finished processing it notifies the ''US'' through a special callback url provided that the processing is finished and the file is available in the public store through a specific url
# When the ''MGS'' has finished processing it notifies the ''US'' through a special callback url provided that the processing is finished and the file is available in the public store through a specific url
# ''US'' moves the object from a pending state to a real object with direct links (or api specific information which might be designed later) to the ''MGS''
# ''US'' moves the object from a pending state to a real object with direct links (or api specific information which might be designed later) to the ''MGS''
# ''US'' sends metadata to attach to the file.


=== File claim/start processing API ===
=== Callback/finished processing API. ===

A proposed json-based api to claim a file uploaded to a MediaGoblin instance. When a file is claimed, the necessary variables are the unique filename, the callback url and the uploader (uses OStatus WebFinger id). The request can of course also contains other metadata and in this scenario we add Creative Commons metadata like original author and license.


The callback function of the ''US'' should read and accept MediaGoblin JSON objects according to the following proposed schema when sent to the callback url provided alongside the original upload:
The following is the JSON Schema for the file claim:

<source lang="javascript">
{
"name":"Claim",
"properties":
{
"filename":
{
"type":"string",
"description":"Unique, claimable filename of an uploaded file on the MediaGoblin server",
"required":true
},
"callback":
{
"type":"string",
"format":"url",
"description":"URL to call when processing is completed/failed for notification",
"required":true
},
"uploader":
{
"type":"string",
"format":"email",
"description":"Webfinger email address of the uploader for identification",
"required":true
},
"creativecommons":
{
"type":"object",
"required":false,
"properties":
{
"author":
{
"type":"string",
"description":"Original author of the work, used for attribution",
"required":true,
},
"license":
{
"type":"string",
"format":"url",
"description":"URL pointing to the Creative Commons license used",
"required":true,
}
}
}
}
}
</source>

An example of a Claim in JSON would be:

<source lang="javascript">
{
"filename":"b5d9f282-08ce-11e1-93a3-b32377194cc9.png",
"callback":"http://example.com/pending/b5d9f282-08ce-11e1-93a3-b32377194cc9/",
"uploader":"user@example.com",
"creativecommons":
{
"author":"Claude Monet",
"licence":"http://creativecommons.org/publicdomain/mark/1.0/"
}
}
</source>

=== Callback/finished processing API ===

The callback function of the ''US'' should read and accept MediaGoblin JSON objects according to the following proposed schema when sent to the callback url provided in the original claim:


<source lang="javascript">
<source lang="javascript">
Line 124: Line 53:
"properties":
"properties":
{
{
"mediatype":
"mimetype":
{
{
"type":"string",
"type":"string",
"description":"Optional string to categorize the file, e.g. ogv, webm",
"description":"Mimetype to categorize the file",
"required":false,
"required":true,
},
},
"location":
"location":
Line 139: Line 68:
}
}
}
}
"mimetype":
"mediatype":
{
{
"type":"string",
"type":"string",
"description":"Mimetype of the file to allow receiver to categorize pending object",
"description":"Media type of the file categorization, e.g. video, audio, image",
"required":false
}
}
}
</source>

The MediaGoblin expects status codes 204 (No content) on success (to imitate pubsubhubbub protocol) or normal error codes (4xx or 5xx) to indicate failures. It is possible to further define JSON objects in responses but probably unnecessary.

== Metadata. ==

It should be possible to add metadata to a specific file at any time. This could be information such as license information, related files (e.g. derivative work), views on site or something. In this case we only start with a Creative Commons license.

=== Metadata API. ===

A proposed json-based api to attach metadata to a file uploaded to a MediaGoblin instance. Other metadata information can be added later. This is just something to start with. It is necessary to check if the calling application is the same as the uploading application (to restrict others from setting metadata to other people's files).

The following is the JSON Schema for metadata:

<source lang="javascript">
{
"name":"Metadata",
"properties":
{
"file":
{
"type":"string",
"format":"url",
"description":"URL to the resource (file) to which metadata should be attached",
"required":true
"required":true
},
"metadata":
{
"type":"array",
"description":"URL to the resource (file) to which metadata should be attached",
"required":true,
"items":
{
"type":"object",
"properties":
{
"creativecommons":
{
"type":"object",
"required":false,
"properties":
{
"author":
{
"type":"string",
"description":"Original author of the work, used for attribution",
"required":true,
},
"license":
{
"type":"string",
"format":"url",
"description":"URL pointing to the Creative Commons license used",
"required":true,
}
}
}
}
}
}
}
}
}
Line 149: Line 140:
</source>
</source>


An example of a Claim in JSON would be (it remains to decide how to handle user uploads from sites where users aren't registerd on MediaGoblin):
=== Possible design problems ===

<source lang="javascript">
{
"file":"http://media.goblin.com/u/artcollection/water-lilies.jpg",
"metadata":
[
"creativecommons":
{
"author":"Claude Monet",
"licence":"http://creativecommons.org/publicdomain/mark/1.0/"
}
]
}
</source>

== Possible design problems. ==


* MediaGoblin is user based but who is the user when files are sent in by users on a specific site ? Use OAuth/OpenID ? Perhaps tie closely with OStatus federation. Probably best to follow Twitter API where applications must be registered and get an API key for uploading.
* Requires a separate upload (or possibly a rewrite of the current upload) to enable ''file claims''
* Mediatype for callback needs a better definition in the API (what can be sent, e.g. .).
* A secure and unique filename to be claimed is needed
* MediaGoblin is user based but who is the user when files are sent in by users on a specific site? Use OAuth/OpenID? Perhaps tie closely with OStatus federation.
* Need a service to remove old unclaimed files (and what would be the time window?)
* Mediatype needs a better definition in the API (what can be sent, e.g. webm or WebM etc.)

Latest revision as of 00:34, 16 February 2016

This page describes a JSON-based API for MediaGoblin. It is a WORK IN PROGRESS at the moment while it is being drafted before it gets implemented. Currently it only includes a submission API, based on a specific scenario which might not work in general for MediaGoblin but can be used as a springboard for further API improvements and specifications.

Scenario.

A MediaGoblin server (MGS) is used for media storage/repository on behalf of a few other servers, called user servers (US) . A user of one US submits a multimedia file. The file is sent to the MGS and categorized on the US in some server specific way. All future use of the multimedia file on the US is processed by the MGS.

The collection of multimedia files from all of the US's is accessible through the MGS interface and through specific categorizations on each US via API or direct links to the content on the MGS.

Submission.

Since all files are processed automatically when they are uploaded there is no need for a specific submission api. Media files should therefore just be uploaded using a normal HTTP POST which is made available via the API using some sort of an authentication mechanism. An extra variable, containing a webhook/callback URL, can be sent alongside the file upload. The callback provides a URL to the file in the public store. The MGS allows the US to send metadata (for example license information) for the file.

The process is as follows:

  1. User fills in a form on a US. The form is in fact based up of two forms
    • One for categorization on the US
    • One is a file form for the MGS
  2. US saves a pending object with the categorization
  3. The MGS processes the file as usual
  4. When the MGS has finished processing it notifies the US through a special callback url provided that the processing is finished and the file is available in the public store through a specific url
  5. US moves the object from a pending state to a real object with direct links (or api specific information which might be designed later) to the MGS
  6. US sends metadata to attach to the file.

Callback/finished processing API.

The callback function of the US should read and accept MediaGoblin JSON objects according to the following proposed schema when sent to the callback url provided alongside the original upload:

{
        "name":"Callback",
        "properties":
        {
                "success":
                {
                        "type":"boolean",
                        "description":"Boolean to indicate whether the processing of the file was successful or not",
                        "required":true
                },
                "message":
                {
                        "type":"string",
                        "description":"Optional message used when processing was unsuccessful to explain the reason for the failure",
                        "required":false
                },
                "location":
                {
                        "type":"array",
                        "required":true,
                        "description":"Array of locations for the processed file (or files, e.g. when video file is encoded into the three possible HTML5 video formats",
                        "items":
                        {
                                "type":"object",
                                "properties":
                                {
                                        "mimetype":
                                        {
                                                "type":"string",
                                                "description":"Mimetype to categorize the file",
                                                "required":true,
                                        },
                                        "location":
                                        {
                                                "type":"string",
                                                "format":"url",
                                                "description":"URL of the file in the public store",
                                                "required":true,
                                        },
                        }
                }
                "mediatype":
                {
                        "type":"string",
                        "description":"Media type of the file categorization, e.g. video, audio, image",
                        "required":false
                }
        }
}

The MediaGoblin expects status codes 204 (No content) on success (to imitate pubsubhubbub protocol) or normal error codes (4xx or 5xx) to indicate failures. It is possible to further define JSON objects in responses but probably unnecessary.

Metadata.

It should be possible to add metadata to a specific file at any time. This could be information such as license information, related files (e.g. derivative work), views on site or something. In this case we only start with a Creative Commons license.

Metadata API.

A proposed json-based api to attach metadata to a file uploaded to a MediaGoblin instance. Other metadata information can be added later. This is just something to start with. It is necessary to check if the calling application is the same as the uploading application (to restrict others from setting metadata to other people's files).

The following is the JSON Schema for metadata:

{
        "name":"Metadata",
        "properties":
        {
                "file":
                {
                        "type":"string",
                        "format":"url",
                        "description":"URL to the resource (file) to which metadata should be attached",
                        "required":true
                },
                "metadata":
                {
                        "type":"array",
                        "description":"URL to the resource (file) to which metadata should be attached",
                        "required":true,
                        "items":
                        {
                                "type":"object",
                                "properties":
                                {
                                        "creativecommons":
                                        {
                                                "type":"object",
                                                "required":false,
                                                "properties":
                                                {
                                                        "author":
                                                        {
                                                                "type":"string",
                                                                "description":"Original author of the work, used for attribution",
                                                                "required":true,
                                                        },
                                                        "license":
                                                        {
                                                                "type":"string",
                                                                "format":"url",
                                                                "description":"URL pointing to the Creative Commons license used",
                                                                "required":true,
                                                        }
                                                }
                                        }
                                }
                        }
                }
        }
}

An example of a Claim in JSON would be (it remains to decide how to handle user uploads from sites where users aren't registerd on MediaGoblin):

{
        "file":"http://media.goblin.com/u/artcollection/water-lilies.jpg",
        "metadata":
        [
                "creativecommons":
                {
                        "author":"Claude Monet",
                        "licence":"http://creativecommons.org/publicdomain/mark/1.0/"
                }
        ]
}

Possible design problems.

  • MediaGoblin is user based but who is the user when files are sent in by users on a specific site ? Use OAuth/OpenID ? Perhaps tie closely with OStatus federation. Probably best to follow Twitter API where applications must be registered and get an API key for uploading.
  • Mediatype for callback needs a better definition in the API (what can be sent, e.g. .).