Commit Graph

742 Commits

Author SHA1 Message Date
Ridho Azhar 477ef689d3
Merge branch 'master' into master 2019-05-27 15:56:20 +07:00
ridhozhr 82c50f9364 add test file with prefix 2019-05-27 13:30:57 +07:00
ridhozhr aa8a29866f remove object prefix gcs from parameter 2019-05-27 13:18:08 +07:00
Marius b89c337b1b cli: Log full HTTP address for uploads 2019-05-26 22:28:14 +02:00
Marius d23be46d7a
core: Add ability to stop upload from post-receive hook (#279)
* First implementation of stopping an upload from the server

* Remove unnecessary json tag

* Use golang.org/x/net/context for support in Go < 1.7
2019-05-26 20:56:51 +01:00
Marius 14faaafc67
chore: Upgrade Docker image to Go 1.12 2019-05-26 21:38:19 +02:00
ridhozhr e25b36c5e9 add flags gcs object prefix validation 2019-05-22 22:19:01 +07:00
ridhozhr 53762be170 integrate prefix with store method 2019-05-22 22:04:17 +07:00
ridhozhr fe62533f1e add prefix in test file gcs store 2019-05-22 21:03:25 +07:00
ridhozhr e824008fe2 integrate flags with composer gcs object prefix 2019-05-22 20:57:32 +07:00
ridhozhr bb2ee4cf41 add gcs-object-prefix flag 2019-05-22 20:54:38 +07:00
ridhozhr 600f4fc939 add object prefix in gcs store 2019-05-22 20:54:14 +07:00
Marius fd2926cf60 chore: Deploy tusd on Go 1.12 2019-05-20 22:53:39 +02:00
Marius 12914032e5 s3store: Exclude aws-sdk-go examples from build
These examples are not compatible with old Go versions.
See https://travis-ci.org/tus/tusd/jobs/535001180#L508
2019-05-20 22:52:15 +02:00
Marius 774cd3800d etcd3locker: Do not run tests on Go < 1.11 2019-05-20 22:38:53 +02:00
Marius 0dd8efd606 filestore: Do not error out on unexpected EOF 2019-05-16 00:03:14 +02:00
Marius 6de723ec8b core: Do not emit post-receive hook when the offset didn't change 2019-05-15 23:57:32 +02:00
Rémy Sanchez 966711019b Create CLI option to bind to UNIX socket (#265) 2019-05-12 22:12:28 +01:00
kiloreux 9d693c93a3 Require it to not run on preemptible 2019-05-08 04:06:26 +01:00
kiloreux cce12757d4 Remove request for 2GB of memory from machine and the requirements to run on a preemptible machine @kvz @Acconut 2019-05-08 04:02:32 +01:00
Marius 12f102abf6
docs: add note about hook extensions on Windows
See https://github.com/tus/tusd/issues/267
2019-04-24 22:48:56 +02:00
Marius acc99bfecf
Only run a single tusd instance at master.tus.io 2019-04-11 21:25:00 +02:00
Marius c02acb8fdf
Decrease read timeout to 6sec for master.tus.io 2019-04-11 19:49:19 +02:00
Marius 2c19a92316
ci: Update helm installation script
In the hope that it will fix the `curl: (3) Illegal characters found in URL` error inside the deployment step:
https://travis-ci.org/tus/tusd/jobs/518477872#L7357
2019-04-10 22:53:35 +02:00
Marius fbbd768a0d consullocker: Update tests to use new import path 2019-04-10 22:28:10 +02:00
Anders Chen b6dc60d2dc docs: Add tusd-dynamo-locker link to README.md (#263) 2019-04-10 21:31:55 +02:00
Marius 589189ba67
docs: Stress the trailing slash in README 2019-04-09 21:47:28 +02:00
Marius 25f64ee00a
Extend FAQ entry about lockfile errors 2019-04-01 22:26:45 +02:00
Rija Ménagé 82bd6c796a docker: Enable the sample hook files to execute without error (#256)
- Include jq package in the Dockerfile for parsing JSON
- Use sh instead of bash which is not available in Alpine Linux
2019-03-30 18:16:53 +01:00
Marius 56acce823d
misc: Add templates for GitHub issues 2019-03-24 19:28:09 +01:00
Kirill Shvakov 495068b5a4 core: Remove unnecessary goroutines (#253)
* deleted unnecessary goroutines

* remove unnecessary goroutines

* restore NotifyTerminatedUploads

* fix merge

* restore NotifyTerminatedUploads

fix merge

revert upload progress event
2019-03-21 20:04:01 +01:00
Acconut b1a657049e limitedstore: Remove unused package fmt 2019-03-15 22:01:43 +01:00
Acconut 0116e4e15b limitedstore: Claim back space from already deleted uploads
Fixes https://github.com/tus/tusd/issues/249
2019-03-15 21:57:47 +01:00
Marius 1612a8ca46
Add FAQ entry about disabling downloads 2019-03-15 21:23:46 +01:00
Acconut c0651e77fc gcsstore: Do not ignore errors when writing an object
Previously, any error returned from w.Close() would be ignored. This is
a problem since this function is responsible for returning any error
that occurred during the actual GCS transaction.

This fix also uncovered an issue in the corresponding test routine.

Furthermore, a "404 Not Found" error when writing an object is now
interpreted as if the bucket does not exist to ease debugging
(see https://github.com/tus/tusd/issues/241).

/cc @tab1293
2019-03-10 17:01:29 +01:00
Adam Jensen 5e06fc54b0 s3store: Use DeleteObject and GetObject to fix IAM issues for incomplete parts (#233)
* Use GetObject instead of HeadObject to locate incomplete part

This avoids confusion around the errors that are returned by HeadObject, especially when the request has limited permissions for the bucket.

* Remove unused HeadObject function

* Add DeleteObject to S3API interface

* Use DeleteObject to remove .part objects

* Update tests
2019-02-23 21:24:38 +01:00
oliverpool 0baa9eaf48 Forward status code and body from pre-create HTTP hook (#236)
* Forward status code from Pre-create http hook

Fix #170

* Allow the returned body to difer from the logged error

* Update HTTP Hooks documentation
2019-02-12 22:45:08 +01:00
Jonas Thelemann df77e34fed Correct "around" Spelling (#242) 2019-02-12 22:11:11 +01:00
Anders Chen 057bb9e2c7 Docs: consullocker/etcd3locker - correct UnlockUpload method docs (#237) 2019-02-04 22:33:17 +01:00
Acconut fa58f91bb4 Store data on S3 even for io.ErrUnexpectedEOF
Previously, some data would be discarded if the user pauseed the upload.
Pausing causes the connection to be interrupted which makes Go's
net/http return an io.ErrUnexpectedEOF.
Before https://github.com/tus/tusd/pull/219 from @acj this would
not be noticed since data is discarded anyway. However, after this
PR, every byte can now be saved on S3 even if we don't have enough
data for a multipart upload part.
2019-01-28 21:41:50 +01:00
kiloreux 5919636d6f Switch master.tus.io storage to AWS S3 (#232)
* Added reference to s3 secrets

* Change s3

* Adjust indentation
2019-01-17 12:06:05 +01:00
Acconut 3b745ad85d Expose number of failed hook executions 2019-01-17 09:52:25 +01:00
Marius 7eb95254a5
Cleanup Travis CI instructions (#231) 2019-01-14 16:19:11 +01:00
Marius 5bc494a989
Install new Docker version on CI (#230) 2019-01-14 13:10:18 +01:00
Acconut 05f79b33ac Only test the prometheuscollector on supported versions 2019-01-14 00:22:12 +01:00
Adam Jensen 204044e4b9 Handle "NotFound" error code from HeadObject (#228)
* Handle "NotFound" error code from HeadObject

This accommodates third party implementations of the S3 interface, such as Minio, that may return a different error string.

* Check NotFound error string in test

This also fixes an incorrect return value.
2019-01-14 00:16:40 +01:00
Acconut daa2250530 Add CA certificates to final Docker image
Fixes https://github.com/tus/tusd/issues/227
2019-01-14 00:08:03 +01:00
Adam Jensen 33d12533e7 S3Store: Add support for creation-defer-length extension (#219)
* Add HeadObject function to S3API

* Regenerate S3API mock

* Include incomplete part size in the offset

* Add CRUD functions for managing incomplete parts

* Account for incomplete parts in S3Store's Terminate

* Account for incomplete parts in S3Store's WriteChunk

* Factor out writeInfo function

* Declare support for deferred length in S3Store

* Add test for S3Store's DeclareLength

* Adapt S3Store tests to new implementation

* Add PutObjectInputMatcher test helper

* Add test for prepending incomplete parts

* Add GetInfo test for incomplete parts

* Update S3Store docs

* Consistently handle NoSuchKey errors from S3

* Handle both 403 and 404 responses from HeadObject

If the IAM role doesn't have permission to list the contents of the bucket, then HEAD requests will return 403 for nonexistent objects.
2019-01-05 09:02:42 +01:00
Rio Kierkels 36526ef03d Switch to a multistage build for the container image (#225)
This makes it easier to reason about tusd's run environment. It also
significantly lowers the docker image size from 280MB for the current 0.11.0
image to only 26MB. Also not having the entire build environment in the
shipped images makes it quite attractive to use this image as a base
for other images. Those could contain some custom hooks for example.
2019-01-04 22:40:22 +01:00
Marius f46d660073
Merge pull request #220 from lucab85/20181121
Fix typos
2018-11-23 11:38:38 +00:00