2013-03-17 09:47:14 +00:00
|
|
|
# tusd
|
2013-03-16 21:23:25 +00:00
|
|
|
|
2013-03-17 14:06:34 +00:00
|
|
|
If content is king, you must make no mistakes acquiring it. tus provides the
|
|
|
|
infrastructure for fast and reliable file uploads for your website or mobile
|
|
|
|
app.
|
2013-03-16 21:23:25 +00:00
|
|
|
|
2013-03-16 23:36:51 +00:00
|
|
|
Sounds interesting? Get notified when it's ready: http://tus.io/
|
|
|
|
|
2013-03-17 14:10:33 +00:00
|
|
|
## Motivation
|
|
|
|
|
|
|
|
It's 2013, and file uploading on the web is still an unsolved problem. There is
|
2013-03-17 21:08:59 +00:00
|
|
|
a distinct lack of full stack open source software that allows developers to
|
2013-03-17 21:08:05 +00:00
|
|
|
provide their users with the experience they deserve.
|
2013-03-17 11:44:58 +00:00
|
|
|
|
|
|
|
The tus mission is to make file uploading more reliable, faster and a better
|
2013-03-17 11:46:12 +00:00
|
|
|
user experience. Instead of building yet another black box service, we are
|
2013-03-17 21:08:05 +00:00
|
|
|
dedicated to providing an open source solution to this problem.
|
2013-03-17 11:44:58 +00:00
|
|
|
|
2013-03-16 21:25:33 +00:00
|
|
|
## Roadmap
|
|
|
|
|
2013-03-17 11:06:53 +00:00
|
|
|
The initial goal for this project to come up with a good and simple solution
|
|
|
|
for resumable file uploads over http.
|
2013-03-16 21:23:25 +00:00
|
|
|
|
2013-03-17 12:08:15 +00:00
|
|
|
* Defining a good http API (first proposal created)
|
|
|
|
* Implementing a minimal and robust server for it (in progress)
|
2013-03-16 21:25:33 +00:00
|
|
|
* Creating an HTML5 client
|
2013-03-16 21:23:25 +00:00
|
|
|
* Setting up an online demo
|
|
|
|
* Integrating Amazon S3 for storage
|
|
|
|
* Creating an iOS client
|
|
|
|
|
2013-03-17 11:06:53 +00:00
|
|
|
Future features will be based on [your
|
|
|
|
feedback](https://github.com/tus/tusd/issues/new). A few potential ideas:
|
|
|
|
|
|
|
|
* Alternative transfer mechanisms: FTP, UDP, E-Mail, etc.
|
|
|
|
* Security: Authentication Tokens, HTTPS, etc.
|
|
|
|
* Support for running tusd instances in a geographically distributed cluster
|
|
|
|
(reverse CDN)
|
|
|
|
* Alternative storage backends: Cloud Files, Dropbox, etc.
|
|
|
|
* More clients: Android, PhoneGap, etc.
|
2013-03-18 16:05:44 +00:00
|
|
|
* Service integrations: Transloadit, Zencoder, Encoding.com, Youtube, Vimeo, Facebook, AWS
|
2013-03-17 14:03:02 +00:00
|
|
|
Transcoder, etc.
|
2013-03-17 11:11:36 +00:00
|
|
|
* File meta data analysis
|
|
|
|
* Thumbnail generation
|
2013-03-17 11:06:53 +00:00
|
|
|
|
|
|
|
Once the project matures, we plan to offer a hosted service and support
|
2013-03-17 11:08:24 +00:00
|
|
|
contracts. However, all code will continue to be released as open source, and
|
|
|
|
you'll always be able to run your own deployments easily. There will be no bait
|
|
|
|
and switch.
|
2013-03-16 21:23:25 +00:00
|
|
|
|
2013-03-17 15:45:09 +00:00
|
|
|
|
|
|
|
## Getting started
|
|
|
|
|
|
|
|
**Requirements:**
|
|
|
|
|
|
|
|
* [Go 1.0](http://golang.org/)
|
|
|
|
|
2013-03-17 15:45:47 +00:00
|
|
|
**Running tusd:**
|
2013-03-17 15:45:09 +00:00
|
|
|
|
|
|
|
Clone the git repository, and `cd` into it. Then simply:
|
|
|
|
|
|
|
|
```bash
|
|
|
|
go run src/cmd/tusd/*.go
|
|
|
|
```
|
|
|
|
|
2013-03-16 23:32:37 +00:00
|
|
|
## HTTP API
|
|
|
|
|
2013-03-17 09:41:17 +00:00
|
|
|
Below is the proposed HTTP API for resumable file uploading.
|
|
|
|
|
2013-03-16 23:44:23 +00:00
|
|
|
### POST /files
|
2013-03-16 23:32:37 +00:00
|
|
|
|
2013-03-17 10:45:36 +00:00
|
|
|
Used to create a resumable file upload. You may send parts or all of your file
|
|
|
|
along with this request, but this is discouraged as you will not be able to
|
|
|
|
resume the request if something goes wrong.
|
|
|
|
|
|
|
|
**Request Example:**
|
2013-03-16 23:32:37 +00:00
|
|
|
|
|
|
|
```
|
|
|
|
POST /files HTTP/1.1
|
|
|
|
Host: tus.example.com
|
|
|
|
Content-Length: 0
|
|
|
|
Content-Range: bytes */100
|
2013-03-17 10:41:45 +00:00
|
|
|
Content-Type: image/jpg
|
2013-03-16 23:32:37 +00:00
|
|
|
```
|
2013-03-17 10:45:36 +00:00
|
|
|
```
|
|
|
|
<empty body>
|
|
|
|
```
|
2013-03-16 23:32:37 +00:00
|
|
|
|
2013-03-17 10:45:36 +00:00
|
|
|
**Response Example:**
|
2013-03-16 23:32:37 +00:00
|
|
|
|
|
|
|
```
|
|
|
|
HTTP/1.1 201 Created
|
2013-03-17 15:40:19 +00:00
|
|
|
Location: http://tus.example.com/files/24e533e02ec3bc40c387f1a0e460e216
|
2013-03-17 10:41:45 +00:00
|
|
|
Content-Length: 0
|
2013-03-16 23:32:37 +00:00
|
|
|
```
|
|
|
|
|
2013-03-17 10:50:32 +00:00
|
|
|
The `Location` header returns the `<fileUrl>` to use for interacting with the
|
|
|
|
file upload.
|
|
|
|
|
|
|
|
### PUT \<fileUrl\>
|
2013-03-16 23:42:41 +00:00
|
|
|
|
2013-03-17 10:45:36 +00:00
|
|
|
**Request Example:**
|
2013-03-17 11:50:08 +00:00
|
|
|
|
2013-03-16 23:42:41 +00:00
|
|
|
```
|
2013-03-17 15:40:19 +00:00
|
|
|
PUT /files/24e533e02ec3bc40c387f1a0e460e216 HTTP/1.1
|
2013-03-16 23:42:41 +00:00
|
|
|
Host: tus.example.com
|
|
|
|
Content-Length: 100
|
|
|
|
Content-Range: bytes 0-99/100
|
|
|
|
```
|
|
|
|
```
|
2013-03-17 10:45:36 +00:00
|
|
|
<bytes 0-99>
|
2013-03-16 23:42:41 +00:00
|
|
|
```
|
|
|
|
|
2013-03-17 10:45:36 +00:00
|
|
|
**Response Example:**
|
2013-03-16 23:42:41 +00:00
|
|
|
```
|
|
|
|
HTTP/1.1 200 Ok
|
|
|
|
```
|
|
|
|
|
2013-03-17 10:50:32 +00:00
|
|
|
### HEAD \<fileUrl\>
|
2013-03-17 10:41:45 +00:00
|
|
|
|
2013-03-17 11:50:08 +00:00
|
|
|
**Request Example:**
|
|
|
|
|
|
|
|
```
|
2013-03-17 15:40:19 +00:00
|
|
|
HEAD /files/24e533e02ec3bc40c387f1a0e460e216 HTTP/1.1
|
2013-03-17 11:50:08 +00:00
|
|
|
Host: tus.example.com
|
|
|
|
```
|
|
|
|
|
|
|
|
**Response Example:**
|
|
|
|
```
|
|
|
|
HTTP/1.1 200 Ok
|
|
|
|
Content-Length: 100
|
|
|
|
Content-Type: image/jpg
|
2013-03-18 15:51:24 +00:00
|
|
|
Range: bytes=0-20,40-99
|
2013-03-17 11:50:08 +00:00
|
|
|
```
|
|
|
|
|
2013-03-18 15:51:24 +00:00
|
|
|
The `Range` header holds a [byte
|
2013-03-17 11:50:08 +00:00
|
|
|
range](http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35.1) that
|
2013-03-18 15:51:24 +00:00
|
|
|
informs the client which parts of the file have been received so far. It is
|
2013-03-17 11:50:08 +00:00
|
|
|
up to the client to choose appropiate `PUT` requests to complete the upload.
|
|
|
|
|
2013-03-18 15:51:24 +00:00
|
|
|
A completed upload will be indicated by a single range covering the entire file
|
|
|
|
size (e.g. `Range: bytes=0-99` for a 100 byte file).
|
2013-03-17 11:50:08 +00:00
|
|
|
|
2013-03-17 10:50:32 +00:00
|
|
|
### GET \<fileUrl\>
|
2013-03-17 10:41:45 +00:00
|
|
|
|
|
|
|
Used to download an uploaded file.
|
2013-03-16 23:32:37 +00:00
|
|
|
|
|
|
|
**Request:**
|
2013-03-17 10:41:45 +00:00
|
|
|
|
2013-03-16 23:32:37 +00:00
|
|
|
```
|
2013-03-17 15:40:19 +00:00
|
|
|
GET /files/24e533e02ec3bc40c387f1a0e460e216 HTTP/1.1
|
2013-03-16 23:32:37 +00:00
|
|
|
Host: tus.example.com
|
|
|
|
```
|
|
|
|
|
2013-03-17 10:41:45 +00:00
|
|
|
**Response:**
|
2013-03-16 23:32:37 +00:00
|
|
|
|
|
|
|
```
|
2013-03-17 10:41:45 +00:00
|
|
|
HTTP/1.1 200 Ok
|
|
|
|
Content-Length: 100
|
|
|
|
Content-Type: image/jpg
|
|
|
|
```
|
|
|
|
```
|
|
|
|
[file data]
|
2013-03-16 23:32:37 +00:00
|
|
|
```
|
|
|
|
|
2013-03-17 14:12:20 +00:00
|
|
|
### Prior art:
|
|
|
|
|
2013-03-18 15:12:49 +00:00
|
|
|
* [YouTube Data API - Resumable Upload](https://developers.google.com/youtube/v3/guides/using_resumable_upload_protocol)
|
2013-03-17 14:12:20 +00:00
|
|
|
* [Google Drive - Upload Files](https://developers.google.com/drive/manage-uploads)
|
|
|
|
* [Resumable Media Uploads in the Google Data Protocol](https://developers.google.com/gdata/docs/resumable_upload) (deprecated)
|
|
|
|
* [ResumableHttpRequestsProposal from Gears](http://code.google.com/p/gears/wiki/ResumableHttpRequestsProposal) (deprecated)
|
|
|
|
|
2013-03-17 14:03:02 +00:00
|
|
|
## FAQ
|
|
|
|
|
2013-03-17 14:03:41 +00:00
|
|
|
### Who is behind this?
|
2013-03-17 14:03:02 +00:00
|
|
|
|
|
|
|
[Transloadit Ltd](http://transloadit.com/) is funding the initial development.
|
|
|
|
However, our goal is to build an active community around this project, so
|
|
|
|
contributions and feedback are more than welcome!
|
|
|
|
|
|
|
|
### Why not upload to Amazon S3 directly?
|
|
|
|
|
|
|
|
Amazon S3 has several limitations that we consider problematic:
|
|
|
|
|
|
|
|
* The minimum chunk size for multipart uploads is 5 MB. This is by far too
|
|
|
|
large for use under bad network conditions.
|
|
|
|
* Throughput to S3 is often too slow for high bandwidth clients.
|
2013-03-17 14:04:32 +00:00
|
|
|
* S3 is a proprietary service. Having an open, vendor agnostic API allows
|
2013-03-17 14:44:18 +00:00
|
|
|
you to treat storage as an implementation detail.
|
2013-03-17 14:03:02 +00:00
|
|
|
* The lack of uniform HTML5, iOS and Android clients that can be easily used
|
|
|
|
to add reliable file uploading to any application.
|
|
|
|
* While there is some support, S3 was not designed to be used in a browser
|
|
|
|
environment.
|
|
|
|
|
2013-03-17 14:57:26 +00:00
|
|
|
S3 is an incredible offering, but we feel that it leaves much to be desired
|
|
|
|
when it comes to offering the best file uploading experience to your users. We
|
|
|
|
can build something much better.
|
2013-03-17 10:41:45 +00:00
|
|
|
|
2013-03-16 21:23:25 +00:00
|
|
|
## License
|
|
|
|
|
|
|
|
This project is licensed under the AGPL v3.
|
|
|
|
|
|
|
|
```
|
|
|
|
Copyright (C) 2013 Transloadit Limited
|
|
|
|
http://transloadit.com/
|
|
|
|
|
|
|
|
This program is free software: you can redistribute it and/or modify
|
|
|
|
it under the terms of the GNU Affero General Public License as published by
|
|
|
|
the Free Software Foundation, either version 3 of the License, or
|
|
|
|
(at your option) any later version.
|
|
|
|
|
|
|
|
This program is distributed in the hope that it will be useful,
|
|
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
|
|
GNU Affero General Public License for more details.
|
|
|
|
|
|
|
|
You should have received a copy of the GNU Affero General Public License
|
|
|
|
along with this program. If not, see <http://www.gnu.org/licenses/>.
|
|
|
|
```
|