Commit Graph

841 Commits

Author SHA1 Message Date
Marius 8ad39ff466 gcsstore: Cancel context in tests to please 'go vet' 2019-06-11 16:52:36 +02:00
Marius 18c24f83f4 misc: Drop support for versions before Go 1.12 2019-06-11 16:33:12 +02:00
Marius b0be7f4cb3 core: Remove fallback for subtests before Go 1.7 2019-06-11 16:29:31 +02:00
Marius 02c5942843 core: Use context package from Go stdlib 2019-06-11 16:16:02 +02:00
Marius 05d9a0ba98 cli: Add option for run hooks from Go plugin
Squashed commit of the following:

commit 1b80f51f94cf860ba8516baed4b65e9ded6441fe
Author: Marius <maerious@gmail.com>
Date:   Mon Jun 10 11:41:30 2019 +0200

    Minor improvements

commit 98daad5f9fa55895a7ae6397b5fcaa353e240954
Author: Marius <maerious@gmail.com>
Date:   Fri Jun 7 13:26:14 2019 +0200

    Extract File and Http hooks into own structs
2019-06-10 11:42:51 +02:00
Juanjo Rodriguez 10e0bb1fb9 core: Add timestamps to tusd log (#276)
* Add timestamp to log

* tus/tusd#274 Add timestamp to log
2019-06-02 15:17:25 +01:00
Marius 79fcf60e79 chore: Cleanup failures from 53b86abc35 2019-06-02 16:09:24 +02:00
ridhozhr 53b86abc35 gcsstore: Add ability to set custom object prefix (#275)
Squashed commit of the following:

commit e48ca3f3fe086504aa1a97d26e2f4fe263880664
Author: Marius <maerious@gmail.com>
Date:   Sun Jun 2 15:54:39 2019 +0200

    Format Go source code

commit 477ef689d3
Merge: 82c50f9 b89c337
Author: Ridho Azhar <azharridho42@gmail.com>
Date:   Mon May 27 15:56:20 2019 +0700

    Merge branch 'master' into master

commit 82c50f9364
Author: ridhozhr <ridho@nodeflux.io>
Date:   Mon May 27 13:30:57 2019 +0700

    add test file with prefix

commit aa8a29866f
Author: ridhozhr <ridho@nodeflux.io>
Date:   Mon May 27 13:18:08 2019 +0700

    remove object prefix gcs from parameter

commit e25b36c5e9
Author: ridhozhr <ridho@nodeflux.io>
Date:   Wed May 22 22:19:01 2019 +0700

    add flags gcs object prefix validation

commit 53762be170
Author: ridhozhr <ridho@nodeflux.io>
Date:   Wed May 22 22:04:17 2019 +0700

    integrate prefix with store method

commit fe62533f1e
Author: ridhozhr <ridho@nodeflux.io>
Date:   Wed May 22 21:03:25 2019 +0700

    add prefix in test file gcs store

commit e824008fe2
Author: ridhozhr <ridho@nodeflux.io>
Date:   Wed May 22 20:57:32 2019 +0700

    integrate flags with composer gcs object prefix

commit bb2ee4cf41
Author: ridhozhr <ridho@nodeflux.io>
Date:   Wed May 22 20:54:38 2019 +0700

    add gcs-object-prefix flag

commit 600f4fc939
Author: ridhozhr <ridho@nodeflux.io>
Date:   Wed May 22 20:54:14 2019 +0700

    add object prefix in gcs store
2019-06-02 15:58:33 +02:00
Marius b89c337b1b cli: Log full HTTP address for uploads 2019-05-26 22:28:14 +02:00
Marius d23be46d7a
core: Add ability to stop upload from post-receive hook (#279)
* First implementation of stopping an upload from the server

* Remove unnecessary json tag

* Use golang.org/x/net/context for support in Go < 1.7
2019-05-26 20:56:51 +01:00
Marius 14faaafc67
chore: Upgrade Docker image to Go 1.12 2019-05-26 21:38:19 +02:00
Marius fd2926cf60 chore: Deploy tusd on Go 1.12 2019-05-20 22:53:39 +02:00
Marius 12914032e5 s3store: Exclude aws-sdk-go examples from build
These examples are not compatible with old Go versions.
See https://travis-ci.org/tus/tusd/jobs/535001180#L508
2019-05-20 22:52:15 +02:00
Marius 774cd3800d etcd3locker: Do not run tests on Go < 1.11 2019-05-20 22:38:53 +02:00
Marius 0dd8efd606 filestore: Do not error out on unexpected EOF 2019-05-16 00:03:14 +02:00
Marius 6de723ec8b core: Do not emit post-receive hook when the offset didn't change 2019-05-15 23:57:32 +02:00
Rémy Sanchez 966711019b Create CLI option to bind to UNIX socket (#265) 2019-05-12 22:12:28 +01:00
kiloreux 9d693c93a3 Require it to not run on preemptible 2019-05-08 04:06:26 +01:00
kiloreux cce12757d4 Remove request for 2GB of memory from machine and the requirements to run on a preemptible machine @kvz @Acconut 2019-05-08 04:02:32 +01:00
Marius 12f102abf6
docs: add note about hook extensions on Windows
See https://github.com/tus/tusd/issues/267
2019-04-24 22:48:56 +02:00
Marius acc99bfecf
Only run a single tusd instance at master.tus.io 2019-04-11 21:25:00 +02:00
Marius c02acb8fdf
Decrease read timeout to 6sec for master.tus.io 2019-04-11 19:49:19 +02:00
Marius 2c19a92316
ci: Update helm installation script
In the hope that it will fix the `curl: (3) Illegal characters found in URL` error inside the deployment step:
https://travis-ci.org/tus/tusd/jobs/518477872#L7357
2019-04-10 22:53:35 +02:00
Marius fbbd768a0d consullocker: Update tests to use new import path 2019-04-10 22:28:10 +02:00
Anders Chen b6dc60d2dc docs: Add tusd-dynamo-locker link to README.md (#263) 2019-04-10 21:31:55 +02:00
Marius 589189ba67
docs: Stress the trailing slash in README 2019-04-09 21:47:28 +02:00
Marius 25f64ee00a
Extend FAQ entry about lockfile errors 2019-04-01 22:26:45 +02:00
Rija Ménagé 82bd6c796a docker: Enable the sample hook files to execute without error (#256)
- Include jq package in the Dockerfile for parsing JSON
- Use sh instead of bash which is not available in Alpine Linux
2019-03-30 18:16:53 +01:00
Marius 56acce823d
misc: Add templates for GitHub issues 2019-03-24 19:28:09 +01:00
Kirill Shvakov 495068b5a4 core: Remove unnecessary goroutines (#253)
* deleted unnecessary goroutines

* remove unnecessary goroutines

* restore NotifyTerminatedUploads

* fix merge

* restore NotifyTerminatedUploads

fix merge

revert upload progress event
2019-03-21 20:04:01 +01:00
Acconut b1a657049e limitedstore: Remove unused package fmt 2019-03-15 22:01:43 +01:00
Acconut 0116e4e15b limitedstore: Claim back space from already deleted uploads
Fixes https://github.com/tus/tusd/issues/249
2019-03-15 21:57:47 +01:00
Marius 1612a8ca46
Add FAQ entry about disabling downloads 2019-03-15 21:23:46 +01:00
Acconut c0651e77fc gcsstore: Do not ignore errors when writing an object
Previously, any error returned from w.Close() would be ignored. This is
a problem since this function is responsible for returning any error
that occurred during the actual GCS transaction.

This fix also uncovered an issue in the corresponding test routine.

Furthermore, a "404 Not Found" error when writing an object is now
interpreted as if the bucket does not exist to ease debugging
(see https://github.com/tus/tusd/issues/241).

/cc @tab1293
2019-03-10 17:01:29 +01:00
Adam Jensen 5e06fc54b0 s3store: Use DeleteObject and GetObject to fix IAM issues for incomplete parts (#233)
* Use GetObject instead of HeadObject to locate incomplete part

This avoids confusion around the errors that are returned by HeadObject, especially when the request has limited permissions for the bucket.

* Remove unused HeadObject function

* Add DeleteObject to S3API interface

* Use DeleteObject to remove .part objects

* Update tests
2019-02-23 21:24:38 +01:00
oliverpool 0baa9eaf48 Forward status code and body from pre-create HTTP hook (#236)
* Forward status code from Pre-create http hook

Fix #170

* Allow the returned body to difer from the logged error

* Update HTTP Hooks documentation
2019-02-12 22:45:08 +01:00
Jonas Thelemann df77e34fed Correct "around" Spelling (#242) 2019-02-12 22:11:11 +01:00
Anders Chen 057bb9e2c7 Docs: consullocker/etcd3locker - correct UnlockUpload method docs (#237) 2019-02-04 22:33:17 +01:00
Acconut fa58f91bb4 Store data on S3 even for io.ErrUnexpectedEOF
Previously, some data would be discarded if the user pauseed the upload.
Pausing causes the connection to be interrupted which makes Go's
net/http return an io.ErrUnexpectedEOF.
Before https://github.com/tus/tusd/pull/219 from @acj this would
not be noticed since data is discarded anyway. However, after this
PR, every byte can now be saved on S3 even if we don't have enough
data for a multipart upload part.
2019-01-28 21:41:50 +01:00
kiloreux 5919636d6f Switch master.tus.io storage to AWS S3 (#232)
* Added reference to s3 secrets

* Change s3

* Adjust indentation
2019-01-17 12:06:05 +01:00
Acconut 3b745ad85d Expose number of failed hook executions 2019-01-17 09:52:25 +01:00
Marius 7eb95254a5
Cleanup Travis CI instructions (#231) 2019-01-14 16:19:11 +01:00
Marius 5bc494a989
Install new Docker version on CI (#230) 2019-01-14 13:10:18 +01:00
Acconut 05f79b33ac Only test the prometheuscollector on supported versions 2019-01-14 00:22:12 +01:00
Adam Jensen 204044e4b9 Handle "NotFound" error code from HeadObject (#228)
* Handle "NotFound" error code from HeadObject

This accommodates third party implementations of the S3 interface, such as Minio, that may return a different error string.

* Check NotFound error string in test

This also fixes an incorrect return value.
2019-01-14 00:16:40 +01:00
Acconut daa2250530 Add CA certificates to final Docker image
Fixes https://github.com/tus/tusd/issues/227
2019-01-14 00:08:03 +01:00
Adam Jensen 33d12533e7 S3Store: Add support for creation-defer-length extension (#219)
* Add HeadObject function to S3API

* Regenerate S3API mock

* Include incomplete part size in the offset

* Add CRUD functions for managing incomplete parts

* Account for incomplete parts in S3Store's Terminate

* Account for incomplete parts in S3Store's WriteChunk

* Factor out writeInfo function

* Declare support for deferred length in S3Store

* Add test for S3Store's DeclareLength

* Adapt S3Store tests to new implementation

* Add PutObjectInputMatcher test helper

* Add test for prepending incomplete parts

* Add GetInfo test for incomplete parts

* Update S3Store docs

* Consistently handle NoSuchKey errors from S3

* Handle both 403 and 404 responses from HeadObject

If the IAM role doesn't have permission to list the contents of the bucket, then HEAD requests will return 403 for nonexistent objects.
2019-01-05 09:02:42 +01:00
Rio Kierkels 36526ef03d Switch to a multistage build for the container image (#225)
This makes it easier to reason about tusd's run environment. It also
significantly lowers the docker image size from 280MB for the current 0.11.0
image to only 26MB. Also not having the entire build environment in the
shipped images makes it quite attractive to use this image as a base
for other images. Those could contain some custom hooks for example.
2019-01-04 22:40:22 +01:00
Marius f46d660073
Merge pull request #220 from lucab85/20181121
Fix typos
2018-11-23 11:38:38 +00:00
Luca Berton 0b2312020f
Fix typos 2018-11-21 21:43:55 +01:00