tusd/README.md

219 lines
6.0 KiB
Markdown
Raw Normal View History

2013-03-17 09:47:14 +00:00
# tusd
2013-03-16 21:23:25 +00:00
2013-03-17 14:06:34 +00:00
If content is king, you must make no mistakes acquiring it. tus provides the
infrastructure for fast and reliable file uploads for your website or mobile
app.
2013-03-16 21:23:25 +00:00
2013-03-16 23:36:51 +00:00
Sounds interesting? Get notified when it's ready: http://tus.io/
2013-03-17 14:10:33 +00:00
## Motivation
It's 2013, and file uploading on the web is still an unsolved problem. There is
2013-03-17 21:08:59 +00:00
a distinct lack of full stack open source software that allows developers to
2013-03-17 21:08:05 +00:00
provide their users with the experience they deserve.
2013-03-17 11:44:58 +00:00
The tus mission is to make file uploading more reliable, faster and a better
2013-03-17 11:46:12 +00:00
user experience. Instead of building yet another black box service, we are
2013-03-17 21:08:05 +00:00
dedicated to providing an open source solution to this problem.
2013-03-17 11:44:58 +00:00
2013-03-16 21:25:33 +00:00
## Roadmap
2013-03-17 11:06:53 +00:00
The initial goal for this project to come up with a good and simple solution
for resumable file uploads over http.
2013-03-16 21:23:25 +00:00
2013-03-17 12:08:15 +00:00
* Defining a good http API (first proposal created)
* Implementing a minimal and robust server for it (in progress)
2013-03-16 21:25:33 +00:00
* Creating an HTML5 client
2013-03-16 21:23:25 +00:00
* Setting up an online demo
* Integrating Amazon S3 for storage
* Creating an iOS client
2013-03-17 11:06:53 +00:00
Future features will be based on [your
feedback](https://github.com/tus/tusd/issues/new). A few potential ideas:
* Alternative transfer mechanisms: FTP, UDP, E-Mail, etc.
* Security: Authentication Tokens, HTTPS, etc.
* Support for running tusd instances in a geographically distributed cluster
(reverse CDN)
* Alternative storage backends: Cloud Files, Dropbox, etc.
* More clients: Android, PhoneGap, etc.
2013-03-17 14:03:02 +00:00
* Service integrations: Zencoder, Encoding.com, Youtube, Vimeo, Facebook, AWS
Transcoder, etc.
2013-03-17 11:11:36 +00:00
* File meta data analysis
* Thumbnail generation
2013-03-17 11:06:53 +00:00
Once the project matures, we plan to offer a hosted service and support
2013-03-17 11:08:24 +00:00
contracts. However, all code will continue to be released as open source, and
you'll always be able to run your own deployments easily. There will be no bait
and switch.
2013-03-16 21:23:25 +00:00
2013-03-17 15:45:09 +00:00
## Getting started
**Requirements:**
* [Go 1.0](http://golang.org/)
2013-03-17 15:45:47 +00:00
**Running tusd:**
2013-03-17 15:45:09 +00:00
Clone the git repository, and `cd` into it. Then simply:
```bash
go run src/cmd/tusd/*.go
```
2013-03-16 23:32:37 +00:00
## HTTP API
2013-03-17 09:41:17 +00:00
Below is the proposed HTTP API for resumable file uploading.
2013-03-16 23:44:23 +00:00
### POST /files
2013-03-16 23:32:37 +00:00
2013-03-17 10:45:36 +00:00
Used to create a resumable file upload. You may send parts or all of your file
along with this request, but this is discouraged as you will not be able to
resume the request if something goes wrong.
**Request Example:**
2013-03-16 23:32:37 +00:00
```
POST /files HTTP/1.1
Host: tus.example.com
Content-Length: 0
Content-Range: bytes */100
2013-03-17 10:41:45 +00:00
Content-Type: image/jpg
2013-03-16 23:32:37 +00:00
```
2013-03-17 10:45:36 +00:00
```
<empty body>
```
2013-03-16 23:32:37 +00:00
2013-03-17 10:45:36 +00:00
**Response Example:**
2013-03-16 23:32:37 +00:00
```
HTTP/1.1 201 Created
2013-03-17 15:40:19 +00:00
Location: http://tus.example.com/files/24e533e02ec3bc40c387f1a0e460e216
2013-03-17 10:41:45 +00:00
Content-Length: 0
2013-03-16 23:32:37 +00:00
```
2013-03-17 10:50:32 +00:00
The `Location` header returns the `<fileUrl>` to use for interacting with the
file upload.
### PUT \<fileUrl\>
2013-03-16 23:42:41 +00:00
2013-03-17 10:45:36 +00:00
**Request Example:**
2013-03-17 11:50:08 +00:00
2013-03-16 23:42:41 +00:00
```
2013-03-17 15:40:19 +00:00
PUT /files/24e533e02ec3bc40c387f1a0e460e216 HTTP/1.1
2013-03-16 23:42:41 +00:00
Host: tus.example.com
Content-Length: 100
Content-Range: bytes 0-99/100
```
```
2013-03-17 10:45:36 +00:00
<bytes 0-99>
2013-03-16 23:42:41 +00:00
```
2013-03-17 10:45:36 +00:00
**Response Example:**
2013-03-16 23:42:41 +00:00
```
HTTP/1.1 200 Ok
```
2013-03-17 10:50:32 +00:00
### HEAD \<fileUrl\>
2013-03-17 10:41:45 +00:00
2013-03-17 11:50:08 +00:00
**Request Example:**
```
2013-03-17 15:40:19 +00:00
HEAD /files/24e533e02ec3bc40c387f1a0e460e216 HTTP/1.1
2013-03-17 11:50:08 +00:00
Host: tus.example.com
```
**Response Example:**
```
HTTP/1.1 200 Ok
Content-Length: 100
Content-Type: image/jpg
2013-03-18 11:24:44 +00:00
X-Missing: bytes=20-50,60-99
2013-03-17 11:50:08 +00:00
```
2013-03-18 11:24:44 +00:00
The `X-Missing` header holds a [byte
2013-03-17 11:50:08 +00:00
range](http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35.1) that
informs the client which parts of the file have not been received yet. It is
up to the client to choose appropiate `PUT` requests to complete the upload.
2013-03-18 11:24:44 +00:00
The absence of a `X-Missing` header means that the the entire file has been
2013-03-17 11:50:08 +00:00
received by the server.
2013-03-17 10:50:32 +00:00
### GET \<fileUrl\>
2013-03-17 10:41:45 +00:00
Used to download an uploaded file.
2013-03-16 23:32:37 +00:00
**Request:**
2013-03-17 10:41:45 +00:00
2013-03-16 23:32:37 +00:00
```
2013-03-17 15:40:19 +00:00
GET /files/24e533e02ec3bc40c387f1a0e460e216 HTTP/1.1
2013-03-16 23:32:37 +00:00
Host: tus.example.com
```
2013-03-17 10:41:45 +00:00
**Response:**
2013-03-16 23:32:37 +00:00
```
2013-03-17 10:41:45 +00:00
HTTP/1.1 200 Ok
Content-Length: 100
Content-Type: image/jpg
```
```
[file data]
2013-03-16 23:32:37 +00:00
```
2013-03-17 14:12:20 +00:00
### Prior art:
2013-03-18 15:12:49 +00:00
* [YouTube Data API - Resumable Upload](https://developers.google.com/youtube/v3/guides/using_resumable_upload_protocol)
2013-03-17 14:12:20 +00:00
* [Google Drive - Upload Files](https://developers.google.com/drive/manage-uploads)
* [Resumable Media Uploads in the Google Data Protocol](https://developers.google.com/gdata/docs/resumable_upload) (deprecated)
* [ResumableHttpRequestsProposal from Gears](http://code.google.com/p/gears/wiki/ResumableHttpRequestsProposal) (deprecated)
2013-03-17 14:03:02 +00:00
## FAQ
2013-03-17 14:03:41 +00:00
### Who is behind this?
2013-03-17 14:03:02 +00:00
[Transloadit Ltd](http://transloadit.com/) is funding the initial development.
However, our goal is to build an active community around this project, so
contributions and feedback are more than welcome!
### Why not upload to Amazon S3 directly?
Amazon S3 has several limitations that we consider problematic:
* The minimum chunk size for multipart uploads is 5 MB. This is by far too
large for use under bad network conditions.
* Throughput to S3 is often too slow for high bandwidth clients.
2013-03-17 14:04:32 +00:00
* S3 is a proprietary service. Having an open, vendor agnostic API allows
2013-03-17 14:44:18 +00:00
you to treat storage as an implementation detail.
2013-03-17 14:03:02 +00:00
* The lack of uniform HTML5, iOS and Android clients that can be easily used
to add reliable file uploading to any application.
* While there is some support, S3 was not designed to be used in a browser
environment.
2013-03-17 14:57:26 +00:00
S3 is an incredible offering, but we feel that it leaves much to be desired
when it comes to offering the best file uploading experience to your users. We
can build something much better.
2013-03-17 10:41:45 +00:00
2013-03-16 21:23:25 +00:00
## License
This project is licensed under the AGPL v3.
```
Copyright (C) 2013 Transloadit Limited
http://transloadit.com/
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
```