Commit Graph

942 Commits

Author SHA1 Message Date
Marius 93187d760c Rework error type from interface to struct 2021-11-27 21:38:26 +01:00
Marius 0513a59e0b Merge branch 'v2' into feat/new-hooks 2021-11-20 17:08:44 +01:00
Marius aca18332d1 core: Add TODO for metrics work 2021-11-15 13:23:59 +01:00
Marius b2273d4153 core: Add error constants
/cc @kvz
2021-10-25 11:54:02 +02:00
Marius c1eddef26a core: Use Go's builtin status code property 2021-10-25 10:38:10 +02:00
Marius 92d704f43f Merge branch 'master' of github.com:tus/tusd into v2 2021-10-25 10:28:51 +02:00
dependabot[bot] 9e91baf19d
build(deps): bump cloud.google.com/go/storage from 1.10.0 to 1.18.2 (#537)
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.10.0 to 1.18.2.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/master/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.10.0...storage/v1.18.2)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-25 10:23:38 +02:00
dependabot[bot] c153ad4f26
build(deps): bump github.com/aws/aws-sdk-go from 1.20.1 to 1.41.9 (#536)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.20.1 to 1.41.9.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.20.1...v1.41.9)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-25 10:23:06 +02:00
Marius fa09e71262
deps: Update dependencies for GCSStore (#535)
* build(deps): bump google.golang.org/grpc from 1.28.0 to 1.41.0

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.28.0 to 1.41.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.28.0...v1.41.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build(deps): bump google.golang.org/api from 0.6.0 to 0.59.0

Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.6.0 to 0.59.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/master/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.6.0...v0.59.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Regenerate GCS API mock

* update test

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-24 10:58:03 +02:00
Ole-Martin Bratteng 0f4f81a1fc
docs: Update build status badges (#534) 2021-10-24 10:04:14 +02:00
Ole-Martin Bratteng a8d8b0fea6
cli: Add flag to disable the greeting page (#530)
* feat: add flag to disable the greeting page
Related to #473

* remove `ShowGreeting` from `handler.Config`
2021-10-23 23:56:37 +02:00
dependabot[bot] 968bb1ddcc
build(deps): bump gopkg.in/h2non/gock.v1 from 1.0.14 to 1.1.2 (#532)
Bumps [gopkg.in/h2non/gock.v1](https://github.com/h2non/gock) from 1.0.14 to 1.1.2.
- [Release notes](https://github.com/h2non/gock/releases)
- [Changelog](https://github.com/h2non/gock/blob/master/History.md)
- [Commits](https://github.com/h2non/gock/compare/v1.0.14...v1.1.2)

---
updated-dependencies:
- dependency-name: gopkg.in/h2non/gock.v1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-23 23:50:35 +02:00
dependabot[bot] 75f2e5a439
build(deps): bump github.com/golang/mock from 1.3.1 to 1.6.0 (#533)
Bumps [github.com/golang/mock](https://github.com/golang/mock) from 1.3.1 to 1.6.0.
- [Release notes](https://github.com/golang/mock/releases)
- [Changelog](https://github.com/golang/mock/blob/master/.goreleaser.yml)
- [Commits](https://github.com/golang/mock/compare/1.3.1...v1.6.0)

---
updated-dependencies:
- dependency-name: github.com/golang/mock
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-23 23:48:58 +02:00
Ole-Martin Bratteng 511ebaed0e
s3store: relax test to prevent failure from race conditions (#529)
* fix: if a race condition occur, we check for it in the test

* fix: windows sometimes only has 1 file

* Trigger test runs

* Trigger test runs

* Trigger test runs

* Trigger test runs

* Trigger test runs
2021-10-23 23:28:33 +02:00
dependabot[bot] d1e710d065
build(deps): bump github.com/grpc-ecosystem/go-grpc-middleware (#522)
Bumps [github.com/grpc-ecosystem/go-grpc-middleware](https://github.com/grpc-ecosystem/go-grpc-middleware) from 1.1.0 to 1.3.0.
- [Release notes](https://github.com/grpc-ecosystem/go-grpc-middleware/releases)
- [Changelog](https://github.com/grpc-ecosystem/go-grpc-middleware/blob/master/CHANGELOG.md)
- [Commits](https://github.com/grpc-ecosystem/go-grpc-middleware/compare/v1.1.0...v1.3.0)

---
updated-dependencies:
- dependency-name: github.com/grpc-ecosystem/go-grpc-middleware
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-23 23:26:28 +02:00
dependabot[bot] 13ae6d2a3a
build(deps): bump github.com/stretchr/testify from 1.4.0 to 1.7.0 (#531)
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.4.0 to 1.7.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.4.0...v1.7.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-23 23:03:35 +02:00
dependabot[bot] 2b794e4b9b
build(deps): bump github.com/prometheus/client_golang (#523)
Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.0.0 to 1.11.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/master/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.0.0...v1.11.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-22 14:08:33 +02:00
dependabot[bot] 7e93a8abaa
build(deps): bump github.com/golang/protobuf from 1.3.5 to 1.5.2 (#521)
Bumps [github.com/golang/protobuf](https://github.com/golang/protobuf) from 1.3.5 to 1.5.2.
- [Release notes](https://github.com/golang/protobuf/releases)
- [Commits](https://github.com/golang/protobuf/compare/v1.3.5...v1.5.2)

---
updated-dependencies:
- dependency-name: github.com/golang/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-21 14:45:17 +02:00
Ole-Martin Bratteng c758437a8a
ci: fix non-failing windows test when it should fail (#526)
* chore: make `continuous-integration.yml` match `release.yml` workflow

* fix: move `go vet` to separate step
2021-10-21 14:29:57 +02:00
Ole-Martin Bratteng 2d27132938
chore: update azure-storage-blob-go to v0.14.0 (#527)
* chore: update azure-storage-blob-go to v0.14.0

* fix: convert the block size from int32 to int64
2021-10-21 14:28:58 +02:00
dependabot[bot] 025ebe47c8
build(deps): bump github.com/vimeo/go-util from 1.2.0 to 1.4.1 (#520)
Bumps [github.com/vimeo/go-util](https://github.com/vimeo/go-util) from 1.2.0 to 1.4.1.
- [Release notes](https://github.com/vimeo/go-util/releases)
- [Commits](https://github.com/vimeo/go-util/compare/v1.2...v1.4.1)

---
updated-dependencies:
- dependency-name: github.com/vimeo/go-util
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-20 16:04:47 +02:00
dependabot[bot] 00bdee0835
build(deps): bump akhileshns/heroku-deploy from 3.4.6 to 3.12.12 (#518)
Bumps [akhileshns/heroku-deploy](https://github.com/akhileshns/heroku-deploy) from 3.4.6 to 3.12.12.
- [Release notes](https://github.com/akhileshns/heroku-deploy/releases)
- [Commits](https://github.com/akhileshns/heroku-deploy/compare/v3.4.6...v3.12.12)

---
updated-dependencies:
- dependency-name: akhileshns/heroku-deploy
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2021-10-20 16:00:49 +02:00
Ole-Martin Bratteng 1401222528
ci: Rework GitHub actions flow (#515)
* feat: separate test job into separate workflow

* feat: add dependabot for github actions, docker and go

* feat: new releaes workflow

* feat: add docker build step

* feat: add docker build step

* chore: remove unused steps in the Dockerfile
Since it is multistage, we don't need to remove stuff like `git` since we only copy the binary

* refactor: Dockerfile now cache better
The layers that have a less chance of being updated, is added earlier

* build: use golang `1.17.2` as builder stage

* build: use alpine `3.14.2` as runtime stage

* chore: remove `gcc` from runtime stage

* feat: add heroku step

* chore: remove `main.yaml` workflow

* fix: remove `latest` flavor, the action handles it
2021-10-19 18:17:21 +02:00
Marius 754aee8c3f ci: Remove plugin hook handler 2021-10-19 17:38:27 +02:00
Marius bc51cb05c0 Merge branch 'master' of github.com:tus/tusd into v2 2021-10-19 17:30:55 +02:00
Marius 21b3de4c71
ci: Replace version test with working regex 2021-10-18 15:04:48 +02:00
Marius cd717e580c
ci: Tag docker image with tag name
See https://github.com/tus/tusd/issues/511
2021-10-18 14:43:37 +02:00
Marius 84faa14987 core: Allow non-tus HEAD requests, Add Length to HEAD responses
See https://github.com/tus/tusd/pull/480

Squashed commit of the following:

commit 7439fd84a6103afdedaf94701a65ce4376789380
Author: Marius <marius@transloadit.com>
Date:   Mon Oct 18 00:27:12 2021 +0200

    Docs and test

commit 16d9dc67e8c8eefc328b1ce12d7e7ca01a49f9f6
Merge: bae0ffb bea5183
Author: Marius <marius@transloadit.com>
Date:   Mon Oct 18 00:23:13 2021 +0200

    Merge branch 'head_header_check' of https://github.com/s3rius/tusd into s3rius-head_header_check

commit bea5183ec3
Author: Pavel Kirilin <win10@list.ru>
Date:   Thu May 20 19:53:36 2021 +0400

    Fixed "Tus-Resumable" header check for HEAD request.

    Signed-off-by: Pavel Kirilin <win10@list.ru>
2021-10-18 00:29:13 +02:00
Tinco Andringa bae0ffb5e5
gcsstore: Allow object prefix to contain underscores (#495)
* Only determine object type based on name after last separator

* modify test to keep in mind directory prefixes with underscores in them

* update documentation to reflect support of underscores in gcs object prefix
2021-10-17 23:48:53 +02:00
Marius 387b04a2e2 Merge branch 'master' of github.com:tus/tusd into v2 2021-10-15 22:05:01 +02:00
benito 16a3747ec4
s3store: Accept alternate error response from DigitalOcean Spaces (#507)
* fix digitalocean spaces fetch

* remove log

* remove log

* remove info.Size check

* Add comment

* Delete .gitignore

Co-authored-by: Marius <marius.kleidl@gmail.com>
2021-10-15 21:50:17 +02:00
Marius dd44267c08
ci: Upgrade Go version for tests and releases 2021-10-13 22:05:19 +02:00
Marius 5614cecb60
misc: Specify Go version for Heroku
See https://github.com/tus/tusd/issues/509
2021-10-13 21:52:30 +02:00
Marius d973f9a81a misc: Increase Go version in go.mod
The current version Go 1.12 is too old, so Heroku does not compile tusd anymore: https://github.com/tus/tusd/runs/3886723478
We use Go 1.16 and not 1.17 because we always support the two latest major releases.
2021-10-13 21:40:39 +02:00
CharlyBr 484e506cd3
misc: Ignore tusd binary (#505) 2021-10-13 21:33:49 +02:00
Marius 0ad435b4c8 core: Fix wrong offset in upload progress notifications
Closes https://github.com/tus/tusd/issues/500
2021-10-13 21:08:09 +02:00
Ole-Martin Bratteng 1b11885823
azurestore: Add implementation
* Add azure-storage-blob-go dependency

* Implement Azure BlobStorage store

* Add AzureStore Mock test

* Refactor Blob interfaces to use uppercase fields

* Refactor and remove the Create function
When getting the offset, and we get the status code BlobNotFound, we can say the offset is 0, and start from the beginning

* Update the mock

* Refactor error checking of GetOffset to actually check the service code

* Begin testing azurestore

* Write more tests

* New feature allows to set access type on new containers and blob access tier

* Write more docs

* Upgrade azure-storage-blob-go to v0.13.0

* Remove AzError, not needed

* Update link to container access type information

* Remove ?toc from link in comments

* Remove trailing spaces from workflow

* Run tests with go1.15 and 1.16

* Don't fail fast
This lets all other tests complete, and makes it easier to see if it's just a one-off fail, or on different OSes and versions

* Remove darwin 386 from `build_all.sh` script
Removed in go1.15 https://github.com/golang/go/issues/37610

* Update go version in `Dockerfile`

* Compile for Apple Silicone (darwin arm64)
Only go1.16 supports it
2021-07-29 01:14:50 +02:00
Marius 5be2afa2f8 Merge branch 'master' of github.com:tus/tusd 2021-07-01 18:49:42 +02:00
Marius 675e767ee6 Merge branch 'master' into v2 2021-07-01 18:49:09 +02:00
Marius 781324986d s3store: Return 400 instead of 500 if upload cannot be streamed
Closes https://github.com/tus/tusd/issues/490
2021-07-01 18:48:58 +02:00
Marius 934265dd38 Merge branch 'master' of github.com:tus/tusd into v2 2021-07-01 18:15:47 +02:00
Marius ccdfe8e604 cli: Add flag to customize the S3 part buffer size 2021-06-21 13:17:36 +02:00
Marius 36f12b1d18 s3store: Fix failing tests due to missing argument 2021-06-21 13:15:21 +02:00
Juanjo Rodriguez b50c2936a1
cli: Change timestamp to microseconds precision in log (#489) 2021-06-14 14:16:39 +02:00
Marius 946539c3b9 cli: Add option to expose Go's pprof 2021-05-28 13:26:13 +02:00
Marius f4314dd360 s3store: Do not register metrics to default registry 2021-05-24 23:45:54 +02:00
Marius 0f24a80ea5 s3store: Expose metrics about request durations 2021-05-24 12:00:20 +02:00
Marius ce54ff8b1f fixup! s3store: Parallelize part uploads and information retrieval (#478) 2021-05-23 12:54:53 +02:00
Marius 578731ab0b cli: Add metric for number of hook invocations 2021-05-20 21:38:19 +02:00
Marius 8fd18364e7
s3store: Parallelize part uploads and information retrieval (#478)
* Add first draft of parallel upload queue

* s3store: Use queue for parallel uploads

* Revert "Add first draft of parallel upload queue"

This reverts commit 86a329cef2.

* Revert "s3store: Use queue for parallel uploads"

This reverts commit 29b59a2c90.

* s3store: Cache results from listing parts and checking incomplete object

* s3store: Remove debugging output`

* s3store: Make requests for fetching info concurrently

* s3store: Make parallel uploads work and tests pass

* s3store: Add semaphore package

* s3store: Add comments to semaphore package

* s3store: Encapsulate more logic into s3PartProducer

* s3store: Refactor WriteChunk

* s3store: Remove TODO

* s3store: Acquire lock before uploading

* cli: Add flag for setting concurrency limit

* s3store: One more comment
2021-05-18 10:29:18 +02:00