Commit Graph

120 Commits

Author SHA1 Message Date
Kaltenbach, Jonas 01dd4c6290 Added Azure SAS Token Authentication 2022-09-15 10:18:41 +02:00
Marius 920deb3df7 cli: Add flag to specify progress interval 2022-08-15 23:26:58 +02:00
Marius eec6a14d4a cli: Do not show metrics in greeting, if disabled
Closes https://github.com/tus/tusd/issues/760
2022-08-04 11:39:55 +02:00
Marius be6f50f14f s3store: Only disable signatures for UploadPart, if -s3-disable-content-hashes is used 2022-07-25 13:48:08 +02:00
Marius efe8c9ce05 s3store: Add option for switching to Minio SDK 2022-07-11 13:43:09 +02:00
Marius aace4704d7 cli: Add fgprof and authentication to pprof endpoint 2022-04-10 19:29:50 +02:00
Marius 9508fd3910 Merge branch 'master' of github.com:tus/tusd into v2 2022-04-10 18:24:30 +02:00
Ole-Martin Bratteng 3feef174fd
handler, cli: Add a flag to disable downloads and termination of uploads (#668)
* Add a flag to disable downloads of uploaded files

* Add a flag to disable deletion of uploaded files

* Fix spelling error

* Rename `DisableDelete` to `DisableTermination`

* Also rename CLI flag

* Conditionally build the `Access-Control-Allow-Methods` headers

* Update tests with new order of allowed methods header

* Add test for checking conditional `Access-Control-Allow-Methods` header
2022-03-28 23:43:35 +02:00
Marius 12c10bf62f
v2: Rework hooks system (#516)
* ci: Remove plugin hook handler

* Rework error type from interface to struct

* Avoid writing to http.ResponseWriter directly

* Allow hooks to modify response

* Add example for HTTP hooks using Python

* Implement new plugin system using Hashicorp/go-plugin

* Enable returning partial HTTPResponses

* Remove some (unnecessary) error handling

* Forward stdout and stderr from plugin to tusd

* docs: Update examples

* cli: Update filehooks to new system

* cli: Renovate gRPC hooks

* docs: Correct casing of gRPC

* misc: Documentation, better examples, and code structure
2022-03-02 00:36:49 +01:00
Marius a05c090d05 Merge branch 'master' into v2 2022-02-23 16:03:41 +01:00
Marius b47294267a cli: Correct environment variables needed for azure store
Closes https://github.com/tus/tusd/issues/660
2022-02-21 12:11:27 +01:00
Marius 965a011b1b cli: Add memorylocker when azurestore is used
Closes https://github.com/tus/tusd/issues/662
2022-02-21 12:07:08 +01:00
Marius 92d704f43f Merge branch 'master' of github.com:tus/tusd into v2 2021-10-25 10:28:51 +02:00
Ole-Martin Bratteng a8d8b0fea6
cli: Add flag to disable the greeting page (#530)
* feat: add flag to disable the greeting page
Related to #473

* remove `ShowGreeting` from `handler.Config`
2021-10-23 23:56:37 +02:00
Marius bc51cb05c0 Merge branch 'master' of github.com:tus/tusd into v2 2021-10-19 17:30:55 +02:00
Tinco Andringa bae0ffb5e5
gcsstore: Allow object prefix to contain underscores (#495)
* Only determine object type based on name after last separator

* modify test to keep in mind directory prefixes with underscores in them

* update documentation to reflect support of underscores in gcs object prefix
2021-10-17 23:48:53 +02:00
Marius 387b04a2e2 Merge branch 'master' of github.com:tus/tusd into v2 2021-10-15 22:05:01 +02:00
Ole-Martin Bratteng 1b11885823
azurestore: Add implementation
* Add azure-storage-blob-go dependency

* Implement Azure BlobStorage store

* Add AzureStore Mock test

* Refactor Blob interfaces to use uppercase fields

* Refactor and remove the Create function
When getting the offset, and we get the status code BlobNotFound, we can say the offset is 0, and start from the beginning

* Update the mock

* Refactor error checking of GetOffset to actually check the service code

* Begin testing azurestore

* Write more tests

* New feature allows to set access type on new containers and blob access tier

* Write more docs

* Upgrade azure-storage-blob-go to v0.13.0

* Remove AzError, not needed

* Update link to container access type information

* Remove ?toc from link in comments

* Remove trailing spaces from workflow

* Run tests with go1.15 and 1.16

* Don't fail fast
This lets all other tests complete, and makes it easier to see if it's just a one-off fail, or on different OSes and versions

* Remove darwin 386 from `build_all.sh` script
Removed in go1.15 https://github.com/golang/go/issues/37610

* Update go version in `Dockerfile`

* Compile for Apple Silicone (darwin arm64)
Only go1.16 supports it
2021-07-29 01:14:50 +02:00
Marius 934265dd38 Merge branch 'master' of github.com:tus/tusd into v2 2021-07-01 18:15:47 +02:00
Marius ccdfe8e604 cli: Add flag to customize the S3 part buffer size 2021-06-21 13:17:36 +02:00
Juanjo Rodriguez b50c2936a1
cli: Change timestamp to microseconds precision in log (#489) 2021-06-14 14:16:39 +02:00
Marius 946539c3b9 cli: Add option to expose Go's pprof 2021-05-28 13:26:13 +02:00
Marius f4314dd360 s3store: Do not register metrics to default registry 2021-05-24 23:45:54 +02:00
Marius ce54ff8b1f fixup! s3store: Parallelize part uploads and information retrieval (#478) 2021-05-23 12:54:53 +02:00
Marius 578731ab0b cli: Add metric for number of hook invocations 2021-05-20 21:38:19 +02:00
Marius 8fd18364e7
s3store: Parallelize part uploads and information retrieval (#478)
* Add first draft of parallel upload queue

* s3store: Use queue for parallel uploads

* Revert "Add first draft of parallel upload queue"

This reverts commit 86a329cef2.

* Revert "s3store: Use queue for parallel uploads"

This reverts commit 29b59a2c90.

* s3store: Cache results from listing parts and checking incomplete object

* s3store: Remove debugging output`

* s3store: Make requests for fetching info concurrently

* s3store: Make parallel uploads work and tests pass

* s3store: Add semaphore package

* s3store: Add comments to semaphore package

* s3store: Encapsulate more logic into s3PartProducer

* s3store: Refactor WriteChunk

* s3store: Remove TODO

* s3store: Acquire lock before uploading

* cli: Add flag for setting concurrency limit

* s3store: One more comment
2021-05-18 10:29:18 +02:00
Marius 8031aabb7e cli: Add flag to disable SSL for S3 2021-03-29 15:12:17 +02:00
Marius 67fd74e129 Add flag to disable calculating expensive hashes for S3 2021-01-18 13:31:07 +01:00
Joey Coleman f4d1b7c443
cli: Allow accepting requests over HTTPS (#423)
* Add TLS support to tusd

* Adds `-tls-certificate`, `-tls-key`, and `-tls-mode` flags
* Alter printed URL to reflect protocol in use
* For non-TLS, do an eary-exit if http.Serve() returns
* Configure TLS for the following modes
    * TLSv1.3-only
    * (default) TLSv1.3+TLSv1.2, with recommended ciphersuites from Mozilla + matching RSA key transport modes
    * TLSv1.2 only, with only 256-bit AES ciphersuites
* All modes disable HTTP/2, given that it’s not supported in non-TLS mode
* Update documentation

* * Remove RSA-based key transport ciphersuites as they don’t support forward secrecy

* Improve the TLS/HTTPS example in the usage documentation

Signed-off-by: Joey Coleman <joey.coleman@kirasystems.com>

* Update docs further to a) record that the key file must be unencrypted, and b) clean up the RSA-based ciphersuite comments.
2020-09-29 15:19:10 +02:00
Kevin McGee 90e208a286
cli: Add flag to enable S3 transfer acceleration (#411)
* Enable S3 transfer acceleration in SDK

* Format better

* Setup for flag changes

* Place feature behind new flag

* Fix a docs issue

Co-authored-by: Cloud User <centos@ip-10-0-0-184.us-west-2.compute.internal>
2020-08-01 15:07:03 +02:00
Marius 9831c8b4da
s3store: Increase default part size (#410)
* s3store: Increase default part size

* Update tests and add flag

* Fix minor issues
2020-08-01 14:58:31 +02:00
Vincenzo Demasi c822a3afb1
cli: Fix help description for "hooks-enabled-events" (#399)
* Update flags.go

* change hooks-enabled-events documentation
2020-06-19 17:32:34 +02:00
Marius 52181920c2 cli: Fix upload creation when using -basepath=/
Closes https://github.com/tus/tusd/issues/387
2020-05-25 09:25:55 +02:00
Hamish Forbes fdf168fbb6
cli: Add pre-finish hook (#382)
* core: add new synchronous event: pre-finish

* docs: pre-finish hook event

* docs: Added information about CORS to the FAQ. (#384)

* Added information about CORS to the FAQ.

* docs: Expand explanation of CORS

Co-authored-by: Marius <marius@transloadit.com>

* cli: add header forwarding in HTTP hooks (#371)

* cli: add header forwarding in HTTP hooks

* docs: Reword header forwarding flag, add documentation

* Don't enable pre-finish hooks by default

* Enable pre-finish hooks for plugins

* docs: default enabled hooks

* Rename callback

Co-authored-by: josh-marshall-jax <52457971+josh-marshall-jax@users.noreply.github.com>
Co-authored-by: Marius <marius@transloadit.com>
2020-05-15 17:27:09 +02:00
Hamish Forbes a4a733fb39
cli: add header forwarding in HTTP hooks (#371)
* cli: add header forwarding in HTTP hooks

* docs: Reword header forwarding flag, add documentation
2020-05-07 14:31:44 +02:00
Marius cfebf1778e cli: Fix build time error of gRPC hooks 2020-04-06 13:46:58 +02:00
Clement Peyrabere acfa974c10
cli: Add hook name to gRPC message (#364)
* Make use of gRPC metadatas to pass on Hook-Name. Fixes #362

* Add Hook name in proto.

* Pass the hook name into the hook proto instead of gRPC metadatas.

Co-authored-by: Clément Peyrabère <clement.peyrabere@idcapture.fr>
2020-03-27 20:48:02 +01:00
Iñigo 1f0b0f515d
cli: Fix gRPC Hook reference (#358)
close #357
2020-03-16 21:15:04 +01:00
Iñigo 8ef7648713
cli: add gRPC hooks (#316)
* add grpc hook

* add retry/backoff params
make streaming RPC call

* Update cmd/tusd/cli/flags.go

Co-Authored-By: Márk Sági-Kazár <sagikazarmark@users.noreply.github.com>

* move one time grpc configuration to `Setup`

* remove stream grpc

Co-authored-by: Márk Sági-Kazár <sagikazarmark@users.noreply.github.com>
2020-02-06 17:35:37 +01:00
Marius 14aebe2985 cli: Fix error when setting basepath to /
Closes https://github.com/tus/tusd/issues/325
2019-11-10 20:18:24 +01:00
Marius d778999213 cli: Tune default timeout to decrease lock times 2019-09-29 20:04:26 +02:00
Marius e7dfc9405d cli: Move printing of hook flags to correct file 2019-09-20 14:33:05 +02:00
Marius 6b21772107 core: Provide HTTP request details to hooks
Closes https://github.com/tus/tusd/issues/185
2019-09-19 11:15:48 +02:00
Marius d2be5e82bd cli: Remove unneeded flag validation check
Closes https://github.com/tus/tusd/issues/229
2019-09-16 14:12:00 +02:00
Marius 485c21d72e core: Add context to DataStores
Closes https://github.com/tus/tusd/issues/288
2019-09-15 13:43:59 +02:00
Marius 7d639b930d core: Replace old LockerDataStore with new Locker 2019-09-12 12:37:43 +02:00
Marius 241c458184 cli: Print out available tus extensions 2019-09-10 15:27:47 +02:00
Marius d01d878e29 gcsstore: Adjust to new DataStore interfaces 2019-09-10 11:09:07 +02:00
Marius e591c9ea69 cli: Update to new DataStore interfaces 2019-09-10 10:23:39 +02:00
Marius 12114b1ae8 Merge branch 'master' of github.com:tus/tusd into v1 2019-09-09 13:37:53 +02:00