Commit Graph

111 Commits

Author SHA1 Message Date
Marius Kleidl 7b9030c33a
chore: Address misc linting warnings 2023-06-09 11:07:05 +02:00
Marius Kleidl d5eca08944
handler: Refactor logging of UploadFinished and UploadTerminated into handler 2023-06-09 08:30:42 +02:00
Marius a75c24996f Add v2 to module name
See https://github.com/tus/tusd/pull/672#issuecomment-1082737523 for context
2023-03-08 13:03:18 +01:00
Marius aff352c413 fixup! Merge branch 'master' of github.com:tus/tusd into v2 2022-10-10 20:54:44 +02:00
Marius f0faa8e556 Merge branch 'master' of github.com:tus/tusd into v2 2022-10-10 20:48:47 +02:00
Marius a349a9296a handler: Allow quotes Forwarded header
Fixes https://github.com/tus/tusd/issues/809
2022-10-05 00:06:17 +02:00
genofire 306a031953
filestore: keep ID passed to NewUpload, if available (#820)
* fix(filestore): keep info.ID

* fix
2022-10-02 00:25:40 +02:00
Shun 7e6cd89114
cli: Change timestamp to microseconds in log (#794)
Co-authored-by: 内藤峻 <naito.shun@nttcom.co.jp>
2022-09-09 18:34:18 +02:00
Marius f898cd2b25 Revert "handler: Avoid calling WriteChunk if no data is available (#746)"
This reverts commit b5cbafdbd5.

See https://github.com/tus/tusd/issues/798
2022-09-09 18:21:09 +02:00
Marius 920deb3df7 cli: Add flag to specify progress interval 2022-08-15 23:26:58 +02:00
Marius 870c434485 s3store: Add metrics for demand on upload semaphore 2022-08-11 23:31:33 +02:00
Marius cab456900a Merge branch 'master' of github.com:tus/tusd into v2 2022-07-25 13:02:05 +02:00
Roberto Villalba e423e30135
s3storeL Add prefix to upload concat source (#767) 2022-07-16 14:52:53 +02:00
Marius f680b9f1ff s3store: Fix tests by adding ContentLength property 2022-07-11 15:01:24 +02:00
Marius efe8c9ce05 s3store: Add option for switching to Minio SDK 2022-07-11 13:43:09 +02:00
Walter Pesce 884cf6ecc1
handler: Fix typo preventing video/ogg from being inlined (#758)
fixed typo to allow video/ogg content-disposition inline.
fixed comment typos.
2022-07-01 23:29:20 +02:00
Marius 1038298a79 core: Change GetReader to return io.ReadCloser (#739)
Squashed commit of the following:

commit e8b5b3751a86d86cae10e0bcf89caa481e5c3de6
Author: Marius <marius@transloadit.com>
Date:   Sun Jun 19 12:15:22 2022 +0200

    Fix generated mocks

commit 736e2e7bb6
Merge: 9d7096f 1e69d9b
Author: Stefan Scheidewig <stefan.scheidewig@staffbase.com>
Date:   Sat Jun 18 07:53:29 2022 +0200

    Merge branch 'v2' into readcloser_in_getreader

commit 9d7096fcb3
Author: Stefan Scheidewig <stefan.scheidewig@staffbase.com>
Date:   Tue May 24 14:16:01 2022 +0200

    Return ReadCloser in getReader
2022-06-19 12:18:02 +02:00
André Fontenele b5cbafdbd5
handler: Avoid calling WriteChunk if no data is available (#746)
* fix: avoiding `writeChunk` no-op

* fix: formatting

* fix: added return statement

* fix: preserving side effects

* refactor: smaller change
2022-06-17 14:04:26 +02:00
Marius 1e69d9ba68 Merge branch 'master' of github.com:tus/tusd into v2 2022-06-17 13:21:59 +02:00
Marius 63830b35d1 handler: Send pre-finish hook before post-finish
Fixes https://github.com/tus/tusd/issues/702
2022-06-17 13:05:38 +02:00
Tom Berger ebf767b2e9
gcsstore: Avoid panic when composing with no passed objects (#360) 2022-05-04 19:18:41 +02:00
Marius c0f2026e96 s3store: Implement temporary support for buffering in memory 2022-04-10 21:13:07 +02:00
Marius 9508fd3910 Merge branch 'master' of github.com:tus/tusd into v2 2022-04-10 18:24:30 +02:00
Ole-Martin Bratteng 3feef174fd
handler, cli: Add a flag to disable downloads and termination of uploads (#668)
* Add a flag to disable downloads of uploaded files

* Add a flag to disable deletion of uploaded files

* Fix spelling error

* Rename `DisableDelete` to `DisableTermination`

* Also rename CLI flag

* Conditionally build the `Access-Control-Allow-Methods` headers

* Update tests with new order of allowed methods header

* Add test for checking conditional `Access-Control-Allow-Methods` header
2022-03-28 23:43:35 +02:00
Marius 7eae867ec1 azurestore: Work around error being not comparable 2022-03-19 23:41:54 +01:00
Marius e77cc64063 Merge branch 'v2' of github.com:tus/tusd into v2 2022-03-19 23:25:03 +01:00
Marius 211feb9ab9 Merge branch 'master' of github.com:tus/tusd into v2 2022-03-19 23:23:03 +01:00
Marius e52139f977
v2: Implement cancelable lock mechanism (#667)
* handler: Implement prototype of new locking back-end

* memorylocker2: Switch to channel for release notification

* handler: Update locker interface

* handler: Add method to close body with error

* memorylocker: Replace with new implementation

* filelocker: Adjust methods to match interface

* handler: Introduce new httpContext

* handler: Implement upload interruption

* handler: Adjust tests to new inferfaces

* handler, memorylocker: Cancel context to avoid leaks
2022-03-19 23:21:17 +01:00
Ole-Martin Bratteng 35dbca86f1
Fix leakage of info when no azure blob is found (#664)
* Create `isAzureError` function

* Throw 404 is info file is not found

* Use `isAzureError` in GetOffset function

* Make `GetUpload` match others without named returns

* Follow code style for `isAzureError` not returning but setting `err` to value

* Return not found when the blob is not found
This will also be triggered when the blob is being uploaded but not yet committed

* Return not found if blob is not found when getting offset

* Don't return error in GetUpload if GetOffset returns `handler.ErrNotFound`
2022-03-17 17:29:15 +01:00
Marius 12c10bf62f
v2: Rework hooks system (#516)
* ci: Remove plugin hook handler

* Rework error type from interface to struct

* Avoid writing to http.ResponseWriter directly

* Allow hooks to modify response

* Add example for HTTP hooks using Python

* Implement new plugin system using Hashicorp/go-plugin

* Enable returning partial HTTPResponses

* Remove some (unnecessary) error handling

* Forward stdout and stderr from plugin to tusd

* docs: Update examples

* cli: Update filehooks to new system

* cli: Renovate gRPC hooks

* docs: Correct casing of gRPC

* misc: Documentation, better examples, and code structure
2022-03-02 00:36:49 +01:00
Marius a05c090d05 Merge branch 'master' into v2 2022-02-23 16:03:41 +01:00
Anatoly 6828cbdce1
s3store: stricter character replacement regexp for x-amz-metadata (#598)
* stricter character replacement rule for x-amz-metadata

* misspell fixed
2021-12-30 17:17:45 +01:00
Marius aca18332d1 core: Add TODO for metrics work 2021-11-15 13:23:59 +01:00
Marius b2273d4153 core: Add error constants
/cc @kvz
2021-10-25 11:54:02 +02:00
Marius c1eddef26a core: Use Go's builtin status code property 2021-10-25 10:38:10 +02:00
Marius 92d704f43f Merge branch 'master' of github.com:tus/tusd into v2 2021-10-25 10:28:51 +02:00
Marius fa09e71262
deps: Update dependencies for GCSStore (#535)
* build(deps): bump google.golang.org/grpc from 1.28.0 to 1.41.0

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.28.0 to 1.41.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.28.0...v1.41.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump google.golang.org/api from 0.6.0 to 0.59.0

Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.6.0 to 0.59.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/master/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.6.0...v0.59.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Regenerate GCS API mock

* update test

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-24 10:58:03 +02:00
Ole-Martin Bratteng 511ebaed0e
s3store: relax test to prevent failure from race conditions (#529)
* fix: if a race condition occur, we check for it in the test

* fix: windows sometimes only has 1 file

* Trigger test runs

* Trigger test runs

* Trigger test runs

* Trigger test runs

* Trigger test runs
2021-10-23 23:28:33 +02:00
Ole-Martin Bratteng 2d27132938
chore: update azure-storage-blob-go to v0.14.0 (#527)
* chore: update azure-storage-blob-go to v0.14.0

* fix: convert the block size from int32 to int64
2021-10-21 14:28:58 +02:00
Marius bc51cb05c0 Merge branch 'master' of github.com:tus/tusd into v2 2021-10-19 17:30:55 +02:00
Marius 84faa14987 core: Allow non-tus HEAD requests, Add Length to HEAD responses
See https://github.com/tus/tusd/pull/480

Squashed commit of the following:

commit 7439fd84a6103afdedaf94701a65ce4376789380
Author: Marius <marius@transloadit.com>
Date:   Mon Oct 18 00:27:12 2021 +0200

    Docs and test

commit 16d9dc67e8c8eefc328b1ce12d7e7ca01a49f9f6
Merge: bae0ffb bea5183
Author: Marius <marius@transloadit.com>
Date:   Mon Oct 18 00:23:13 2021 +0200

    Merge branch 'head_header_check' of https://github.com/s3rius/tusd into s3rius-head_header_check

commit bea5183ec3
Author: Pavel Kirilin <win10@list.ru>
Date:   Thu May 20 19:53:36 2021 +0400

    Fixed "Tus-Resumable" header check for HEAD request.

    Signed-off-by: Pavel Kirilin <win10@list.ru>
2021-10-18 00:29:13 +02:00
Tinco Andringa bae0ffb5e5
gcsstore: Allow object prefix to contain underscores (#495)
* Only determine object type based on name after last separator

* modify test to keep in mind directory prefixes with underscores in them

* update documentation to reflect support of underscores in gcs object prefix
2021-10-17 23:48:53 +02:00
Marius 387b04a2e2 Merge branch 'master' of github.com:tus/tusd into v2 2021-10-15 22:05:01 +02:00
benito 16a3747ec4
s3store: Accept alternate error response from DigitalOcean Spaces (#507)
* fix digitalocean spaces fetch

* remove log

* remove log

* remove info.Size check

* Add comment

* Delete .gitignore

Co-authored-by: Marius <marius.kleidl@gmail.com>
2021-10-15 21:50:17 +02:00
Marius 0ad435b4c8 core: Fix wrong offset in upload progress notifications
Closes https://github.com/tus/tusd/issues/500
2021-10-13 21:08:09 +02:00
Ole-Martin Bratteng 1b11885823
azurestore: Add implementation
* Add azure-storage-blob-go dependency

* Implement Azure BlobStorage store

* Add AzureStore Mock test

* Refactor Blob interfaces to use uppercase fields

* Refactor and remove the Create function
When getting the offset, and we get the status code BlobNotFound, we can say the offset is 0, and start from the beginning

* Update the mock

* Refactor error checking of GetOffset to actually check the service code

* Begin testing azurestore

* Write more tests

* New feature allows to set access type on new containers and blob access tier

* Write more docs

* Upgrade azure-storage-blob-go to v0.13.0

* Remove AzError, not needed

* Update link to container access type information

* Remove ?toc from link in comments

* Remove trailing spaces from workflow

* Run tests with go1.15 and 1.16

* Don't fail fast
This lets all other tests complete, and makes it easier to see if it's just a one-off fail, or on different OSes and versions

* Remove darwin 386 from `build_all.sh` script
Removed in go1.15 https://github.com/golang/go/issues/37610

* Update go version in `Dockerfile`

* Compile for Apple Silicone (darwin arm64)
Only go1.16 supports it
2021-07-29 01:14:50 +02:00
Marius 675e767ee6 Merge branch 'master' into v2 2021-07-01 18:49:09 +02:00
Marius 781324986d s3store: Return 400 instead of 500 if upload cannot be streamed
Closes https://github.com/tus/tusd/issues/490
2021-07-01 18:48:58 +02:00
Marius ccdfe8e604 cli: Add flag to customize the S3 part buffer size 2021-06-21 13:17:36 +02:00
Marius 36f12b1d18 s3store: Fix failing tests due to missing argument 2021-06-21 13:15:21 +02:00