Commit Graph

104 Commits

Author SHA1 Message Date
Marius 387b04a2e2 Merge branch 'master' of github.com:tus/tusd into v2 2021-10-15 22:05:01 +02:00
Ole-Martin Bratteng 1b11885823
azurestore: Add implementation
* Add azure-storage-blob-go dependency

* Implement Azure BlobStorage store

* Add AzureStore Mock test

* Refactor Blob interfaces to use uppercase fields

* Refactor and remove the Create function
When getting the offset, and we get the status code BlobNotFound, we can say the offset is 0, and start from the beginning

* Update the mock

* Refactor error checking of GetOffset to actually check the service code

* Begin testing azurestore

* Write more tests

* New feature allows to set access type on new containers and blob access tier

* Write more docs

* Upgrade azure-storage-blob-go to v0.13.0

* Remove AzError, not needed

* Update link to container access type information

* Remove ?toc from link in comments

* Remove trailing spaces from workflow

* Run tests with go1.15 and 1.16

* Don't fail fast
This lets all other tests complete, and makes it easier to see if it's just a one-off fail, or on different OSes and versions

* Remove darwin 386 from `build_all.sh` script
Removed in go1.15 https://github.com/golang/go/issues/37610

* Update go version in `Dockerfile`

* Compile for Apple Silicone (darwin arm64)
Only go1.16 supports it
2021-07-29 01:14:50 +02:00
Marius 934265dd38 Merge branch 'master' of github.com:tus/tusd into v2 2021-07-01 18:15:47 +02:00
Marius ccdfe8e604 cli: Add flag to customize the S3 part buffer size 2021-06-21 13:17:36 +02:00
Juanjo Rodriguez b50c2936a1
cli: Change timestamp to microseconds precision in log (#489) 2021-06-14 14:16:39 +02:00
Marius 946539c3b9 cli: Add option to expose Go's pprof 2021-05-28 13:26:13 +02:00
Marius f4314dd360 s3store: Do not register metrics to default registry 2021-05-24 23:45:54 +02:00
Marius ce54ff8b1f fixup! s3store: Parallelize part uploads and information retrieval (#478) 2021-05-23 12:54:53 +02:00
Marius 578731ab0b cli: Add metric for number of hook invocations 2021-05-20 21:38:19 +02:00
Marius 8fd18364e7
s3store: Parallelize part uploads and information retrieval (#478)
* Add first draft of parallel upload queue

* s3store: Use queue for parallel uploads

* Revert "Add first draft of parallel upload queue"

This reverts commit 86a329cef2.

* Revert "s3store: Use queue for parallel uploads"

This reverts commit 29b59a2c90.

* s3store: Cache results from listing parts and checking incomplete object

* s3store: Remove debugging output`

* s3store: Make requests for fetching info concurrently

* s3store: Make parallel uploads work and tests pass

* s3store: Add semaphore package

* s3store: Add comments to semaphore package

* s3store: Encapsulate more logic into s3PartProducer

* s3store: Refactor WriteChunk

* s3store: Remove TODO

* s3store: Acquire lock before uploading

* cli: Add flag for setting concurrency limit

* s3store: One more comment
2021-05-18 10:29:18 +02:00
Marius 8031aabb7e cli: Add flag to disable SSL for S3 2021-03-29 15:12:17 +02:00
Marius 67fd74e129 Add flag to disable calculating expensive hashes for S3 2021-01-18 13:31:07 +01:00
Joey Coleman f4d1b7c443
cli: Allow accepting requests over HTTPS (#423)
* Add TLS support to tusd

* Adds `-tls-certificate`, `-tls-key`, and `-tls-mode` flags
* Alter printed URL to reflect protocol in use
* For non-TLS, do an eary-exit if http.Serve() returns
* Configure TLS for the following modes
    * TLSv1.3-only
    * (default) TLSv1.3+TLSv1.2, with recommended ciphersuites from Mozilla + matching RSA key transport modes
    * TLSv1.2 only, with only 256-bit AES ciphersuites
* All modes disable HTTP/2, given that it’s not supported in non-TLS mode
* Update documentation

* * Remove RSA-based key transport ciphersuites as they don’t support forward secrecy

* Improve the TLS/HTTPS example in the usage documentation

Signed-off-by: Joey Coleman <joey.coleman@kirasystems.com>

* Update docs further to a) record that the key file must be unencrypted, and b) clean up the RSA-based ciphersuite comments.
2020-09-29 15:19:10 +02:00
Kevin McGee 90e208a286
cli: Add flag to enable S3 transfer acceleration (#411)
* Enable S3 transfer acceleration in SDK

* Format better

* Setup for flag changes

* Place feature behind new flag

* Fix a docs issue

Co-authored-by: Cloud User <centos@ip-10-0-0-184.us-west-2.compute.internal>
2020-08-01 15:07:03 +02:00
Marius 9831c8b4da
s3store: Increase default part size (#410)
* s3store: Increase default part size

* Update tests and add flag

* Fix minor issues
2020-08-01 14:58:31 +02:00
Vincenzo Demasi c822a3afb1
cli: Fix help description for "hooks-enabled-events" (#399)
* Update flags.go

* change hooks-enabled-events documentation
2020-06-19 17:32:34 +02:00
Marius 52181920c2 cli: Fix upload creation when using -basepath=/
Closes https://github.com/tus/tusd/issues/387
2020-05-25 09:25:55 +02:00
Hamish Forbes fdf168fbb6
cli: Add pre-finish hook (#382)
* core: add new synchronous event: pre-finish

* docs: pre-finish hook event

* docs: Added information about CORS to the FAQ. (#384)

* Added information about CORS to the FAQ.

* docs: Expand explanation of CORS

Co-authored-by: Marius <marius@transloadit.com>

* cli: add header forwarding in HTTP hooks (#371)

* cli: add header forwarding in HTTP hooks

* docs: Reword header forwarding flag, add documentation

* Don't enable pre-finish hooks by default

* Enable pre-finish hooks for plugins

* docs: default enabled hooks

* Rename callback

Co-authored-by: josh-marshall-jax <52457971+josh-marshall-jax@users.noreply.github.com>
Co-authored-by: Marius <marius@transloadit.com>
2020-05-15 17:27:09 +02:00
Hamish Forbes a4a733fb39
cli: add header forwarding in HTTP hooks (#371)
* cli: add header forwarding in HTTP hooks

* docs: Reword header forwarding flag, add documentation
2020-05-07 14:31:44 +02:00
Marius cfebf1778e cli: Fix build time error of gRPC hooks 2020-04-06 13:46:58 +02:00
Clement Peyrabere acfa974c10
cli: Add hook name to gRPC message (#364)
* Make use of gRPC metadatas to pass on Hook-Name. Fixes #362

* Add Hook name in proto.

* Pass the hook name into the hook proto instead of gRPC metadatas.

Co-authored-by: Clément Peyrabère <clement.peyrabere@idcapture.fr>
2020-03-27 20:48:02 +01:00
Iñigo 1f0b0f515d
cli: Fix gRPC Hook reference (#358)
close #357
2020-03-16 21:15:04 +01:00
Iñigo 8ef7648713
cli: add gRPC hooks (#316)
* add grpc hook

* add retry/backoff params
make streaming RPC call

* Update cmd/tusd/cli/flags.go

Co-Authored-By: Márk Sági-Kazár <sagikazarmark@users.noreply.github.com>

* move one time grpc configuration to `Setup`

* remove stream grpc

Co-authored-by: Márk Sági-Kazár <sagikazarmark@users.noreply.github.com>
2020-02-06 17:35:37 +01:00
Marius 14aebe2985 cli: Fix error when setting basepath to /
Closes https://github.com/tus/tusd/issues/325
2019-11-10 20:18:24 +01:00
Marius d778999213 cli: Tune default timeout to decrease lock times 2019-09-29 20:04:26 +02:00
Marius e7dfc9405d cli: Move printing of hook flags to correct file 2019-09-20 14:33:05 +02:00
Marius 6b21772107 core: Provide HTTP request details to hooks
Closes https://github.com/tus/tusd/issues/185
2019-09-19 11:15:48 +02:00
Marius d2be5e82bd cli: Remove unneeded flag validation check
Closes https://github.com/tus/tusd/issues/229
2019-09-16 14:12:00 +02:00
Marius 485c21d72e core: Add context to DataStores
Closes https://github.com/tus/tusd/issues/288
2019-09-15 13:43:59 +02:00
Marius 7d639b930d core: Replace old LockerDataStore with new Locker 2019-09-12 12:37:43 +02:00
Marius 241c458184 cli: Print out available tus extensions 2019-09-10 15:27:47 +02:00
Marius d01d878e29 gcsstore: Adjust to new DataStore interfaces 2019-09-10 11:09:07 +02:00
Marius e591c9ea69 cli: Update to new DataStore interfaces 2019-09-10 10:23:39 +02:00
Marius 12114b1ae8 Merge branch 'master' of github.com:tus/tusd into v1 2019-09-09 13:37:53 +02:00
Marius f1fe5e2603 cli: Add option to whitelist hooks (#302) 2019-08-26 20:34:37 +02:00
Marius 141a49a38d cli: Allow uploads without trailing slash 2019-08-19 12:03:14 +02:00
Marius c0c3a4d1d0 filestore: Add information about saved upload 2019-08-19 09:29:56 +02:00
Marius e8fb3a431b Merge branch 'master' of github.com:tus/tusd into v1 2019-08-18 14:34:37 +02:00
Marius 571bc4df75 cli: Add flag to reduce verbosity of output 2019-08-13 10:51:03 +02:00
Marius ab641b6f89 Merge branch 'master' of github.com:tus/tusd into v1 2019-06-16 17:00:10 +02:00
Thomas c48512798e cli: Replace deprecated function for getting Prometheus handler (#281) 2019-06-13 10:34:28 +01:00
Marius 42bfe35457 core: Move packages into pkg/ and internal/ folders 2019-06-11 18:23:20 +02:00
Marius 32cf95aefe limitedstore: Remove package due to bad functionality 2019-06-11 17:06:33 +02:00
Marius 05d9a0ba98 cli: Add option for run hooks from Go plugin
Squashed commit of the following:

commit 1b80f51f94cf860ba8516baed4b65e9ded6441fe
Author: Marius <maerious@gmail.com>
Date:   Mon Jun 10 11:41:30 2019 +0200

    Minor improvements

commit 98daad5f9fa55895a7ae6397b5fcaa353e240954
Author: Marius <maerious@gmail.com>
Date:   Fri Jun 7 13:26:14 2019 +0200

    Extract File and Http hooks into own structs
2019-06-10 11:42:51 +02:00
Marius 79fcf60e79 chore: Cleanup failures from 53b86abc35 2019-06-02 16:09:24 +02:00
ridhozhr 53b86abc35 gcsstore: Add ability to set custom object prefix (#275)
Squashed commit of the following:

commit e48ca3f3fe086504aa1a97d26e2f4fe263880664
Author: Marius <maerious@gmail.com>
Date:   Sun Jun 2 15:54:39 2019 +0200

    Format Go source code

commit 477ef689d3
Merge: 82c50f9 b89c337
Author: Ridho Azhar <azharridho42@gmail.com>
Date:   Mon May 27 15:56:20 2019 +0700

    Merge branch 'master' into master

commit 82c50f9364
Author: ridhozhr <ridho@nodeflux.io>
Date:   Mon May 27 13:30:57 2019 +0700

    add test file with prefix

commit aa8a29866f
Author: ridhozhr <ridho@nodeflux.io>
Date:   Mon May 27 13:18:08 2019 +0700

    remove object prefix gcs from parameter

commit e25b36c5e9
Author: ridhozhr <ridho@nodeflux.io>
Date:   Wed May 22 22:19:01 2019 +0700

    add flags gcs object prefix validation

commit 53762be170
Author: ridhozhr <ridho@nodeflux.io>
Date:   Wed May 22 22:04:17 2019 +0700

    integrate prefix with store method

commit fe62533f1e
Author: ridhozhr <ridho@nodeflux.io>
Date:   Wed May 22 21:03:25 2019 +0700

    add prefix in test file gcs store

commit e824008fe2
Author: ridhozhr <ridho@nodeflux.io>
Date:   Wed May 22 20:57:32 2019 +0700

    integrate flags with composer gcs object prefix

commit bb2ee4cf41
Author: ridhozhr <ridho@nodeflux.io>
Date:   Wed May 22 20:54:38 2019 +0700

    add gcs-object-prefix flag

commit 600f4fc939
Author: ridhozhr <ridho@nodeflux.io>
Date:   Wed May 22 20:54:14 2019 +0700

    add object prefix in gcs store
2019-06-02 15:58:33 +02:00
Marius b89c337b1b cli: Log full HTTP address for uploads 2019-05-26 22:28:14 +02:00
Marius d23be46d7a
core: Add ability to stop upload from post-receive hook (#279)
* First implementation of stopping an upload from the server

* Remove unnecessary json tag

* Use golang.org/x/net/context for support in Go < 1.7
2019-05-26 20:56:51 +01:00
Rémy Sanchez 966711019b Create CLI option to bind to UNIX socket (#265) 2019-05-12 22:12:28 +01:00
oliverpool 0baa9eaf48 Forward status code and body from pre-create HTTP hook (#236)
* Forward status code from Pre-create http hook

Fix #170

* Allow the returned body to difer from the logged error

* Update HTTP Hooks documentation
2019-02-12 22:45:08 +01:00