tusd/s3store/s3store_test.go

698 lines
20 KiB
Go
Raw Normal View History

s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
package s3store
2016-01-05 17:21:53 +00:00
import (
"bytes"
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
"fmt"
2016-01-05 17:21:53 +00:00
"io/ioutil"
"testing"
"github.com/golang/mock/gomock"
"github.com/stretchr/testify/assert"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/service/s3"
"github.com/tus/tusd"
)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
//go:generate mockgen -destination=./s3store_mock_test.go -package=s3store github.com/tus/tusd/s3store S3API
2016-01-05 17:21:53 +00:00
// Test interface implementations
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
var _ tusd.DataStore = S3Store{}
var _ tusd.GetReaderDataStore = S3Store{}
var _ tusd.TerminaterDataStore = S3Store{}
var _ tusd.FinisherDataStore = S3Store{}
var _ tusd.ConcaterDataStore = S3Store{}
2016-01-05 17:21:53 +00:00
func TestNewUpload(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
2016-01-05 17:21:53 +00:00
assert.Equal("bucket", store.Bucket)
assert.Equal(s3obj, store.Service)
2016-01-05 17:21:53 +00:00
2016-01-19 20:39:24 +00:00
s1 := "hello"
s2 := "men?"
2016-01-19 20:39:24 +00:00
2016-01-05 17:21:53 +00:00
gomock.InOrder(
s3obj.EXPECT().CreateMultipartUpload(&s3.CreateMultipartUploadInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
2016-01-19 20:39:24 +00:00
Metadata: map[string]*string{
"foo": &s1,
"bar": &s2,
},
2016-01-05 17:21:53 +00:00
}).Return(&s3.CreateMultipartUploadOutput{
UploadId: aws.String("multipartId"),
}, nil),
s3obj.EXPECT().PutObject(&s3.PutObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId.info"),
Body: bytes.NewReader([]byte(`{"ID":"uploadId+multipartId","Size":500,"SizeIsDeferred":false,"Offset":0,"MetaData":{"bar":"menü","foo":"hello"},"IsPartial":false,"IsFinal":false,"PartialUploads":null}`)),
ContentLength: aws.Int64(int64(171)),
}),
2016-01-05 17:21:53 +00:00
)
info := tusd.FileInfo{
ID: "uploadId",
Size: 500,
2016-01-19 20:39:24 +00:00
MetaData: map[string]string{
"foo": "hello",
"bar": "menü",
2016-01-19 20:39:24 +00:00
},
2016-01-05 17:21:53 +00:00
}
id, err := store.NewUpload(info)
assert.Nil(err)
assert.Equal("uploadId+multipartId", id)
2016-01-05 17:21:53 +00:00
}
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
func TestNewUploadLargerMaxObjectSize(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
store := New("bucket", s3obj)
assert.Equal("bucket", store.Bucket)
assert.Equal(s3obj, store.Service)
info := tusd.FileInfo{
ID: "uploadId",
Size: store.MaxObjectSize + 1,
}
id, err := store.NewUpload(info)
assert.NotNil(err)
assert.EqualError(err, fmt.Sprintf("s3store: upload size of %v bytes exceeds MaxObjectSize of %v bytes", info.Size, store.MaxObjectSize))
assert.Equal("", id)
}
2016-01-05 17:21:53 +00:00
func TestGetInfoNotFound(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
2016-01-05 17:21:53 +00:00
s3obj.EXPECT().GetObject(&s3.GetObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId.info"),
}).Return(nil, awserr.New("NoSuchKey", "The specified key does not exist.", nil))
_, err := store.GetInfo("uploadId+multipartId")
assert.Equal(tusd.ErrNotFound, err)
2016-01-05 17:21:53 +00:00
}
func TestGetInfo(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
2016-01-05 17:21:53 +00:00
gomock.InOrder(
s3obj.EXPECT().GetObject(&s3.GetObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId.info"),
}).Return(&s3.GetObjectOutput{
Body: ioutil.NopCloser(bytes.NewReader([]byte(`{"ID":"uploadId+multipartId","Size":500,"Offset":0,"MetaData":{"bar":"menü","foo":"hello"},"IsPartial":false,"IsFinal":false,"PartialUploads":null}`))),
2016-01-05 17:21:53 +00:00
}, nil),
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumberMarker: aws.Int64(0),
2016-01-05 17:21:53 +00:00
}).Return(&s3.ListPartsOutput{
Parts: []*s3.Part{
{
Size: aws.Int64(100),
},
{
Size: aws.Int64(200),
},
},
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
NextPartNumberMarker: aws.Int64(2),
IsTruncated: aws.Bool(true),
}, nil),
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumberMarker: aws.Int64(2),
}).Return(&s3.ListPartsOutput{
Parts: []*s3.Part{
{
Size: aws.Int64(100),
},
},
2016-01-05 17:21:53 +00:00
}, nil),
)
info, err := store.GetInfo("uploadId+multipartId")
assert.Nil(err)
assert.Equal(int64(500), info.Size)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
assert.Equal(int64(400), info.Offset)
2016-03-26 17:23:37 +00:00
assert.Equal("uploadId+multipartId", info.ID)
assert.Equal("hello", info.MetaData["foo"])
assert.Equal("menü", info.MetaData["bar"])
2016-01-05 17:21:53 +00:00
}
func TestGetInfoFinished(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
2016-01-05 17:21:53 +00:00
gomock.InOrder(
s3obj.EXPECT().GetObject(&s3.GetObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId.info"),
}).Return(&s3.GetObjectOutput{
Body: ioutil.NopCloser(bytes.NewReader([]byte(`{"ID":"uploadId","Size":500,"Offset":0,"MetaData":null,"IsPartial":false,"IsFinal":false,"PartialUploads":null}`))),
}, nil),
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumberMarker: aws.Int64(0),
2016-01-05 17:21:53 +00:00
}).Return(nil, awserr.New("NoSuchUpload", "The specified upload does not exist.", nil)),
)
info, err := store.GetInfo("uploadId+multipartId")
assert.Nil(err)
assert.Equal(int64(500), info.Size)
assert.Equal(int64(500), info.Offset)
}
func TestGetReader(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
2016-01-05 17:21:53 +00:00
s3obj.EXPECT().GetObject(&s3.GetObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
}).Return(&s3.GetObjectOutput{
Body: ioutil.NopCloser(bytes.NewReader([]byte(`hello world`))),
}, nil)
content, err := store.GetReader("uploadId+multipartId")
assert.Nil(err)
assert.Equal(ioutil.NopCloser(bytes.NewReader([]byte(`hello world`))), content)
2016-01-05 17:21:53 +00:00
}
func TestGetReaderNotFound(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
2016-01-05 17:21:53 +00:00
gomock.InOrder(
s3obj.EXPECT().GetObject(&s3.GetObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
}).Return(nil, awserr.New("NoSuchKey", "The specified key does not exist.", nil)),
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
MaxParts: aws.Int64(0),
}).Return(nil, awserr.New("NoSuchUpload", "The specified upload does not exist.", nil)),
)
content, err := store.GetReader("uploadId+multipartId")
assert.Nil(content)
assert.Equal(tusd.ErrNotFound, err)
2016-01-05 17:21:53 +00:00
}
func TestGetReaderNotFinished(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
2016-01-05 17:21:53 +00:00
gomock.InOrder(
s3obj.EXPECT().GetObject(&s3.GetObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
}).Return(nil, awserr.New("NoSuchKey", "The specified key does not exist.", nil)),
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
MaxParts: aws.Int64(0),
}).Return(&s3.ListPartsOutput{
Parts: []*s3.Part{},
}, nil),
)
content, err := store.GetReader("uploadId+multipartId")
assert.Nil(content)
assert.Equal("cannot stream non-finished upload", err.Error())
2016-01-05 17:21:53 +00:00
}
func TestFinishUpload(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
2016-01-05 17:21:53 +00:00
gomock.InOrder(
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumberMarker: aws.Int64(0),
2016-01-05 17:21:53 +00:00
}).Return(&s3.ListPartsOutput{
Parts: []*s3.Part{
{
Size: aws.Int64(100),
ETag: aws.String("foo"),
PartNumber: aws.Int64(1),
},
{
Size: aws.Int64(200),
ETag: aws.String("bar"),
PartNumber: aws.Int64(2),
},
},
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
NextPartNumberMarker: aws.Int64(2),
IsTruncated: aws.Bool(true),
}, nil),
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumberMarker: aws.Int64(2),
}).Return(&s3.ListPartsOutput{
Parts: []*s3.Part{
{
Size: aws.Int64(100),
ETag: aws.String("foobar"),
PartNumber: aws.Int64(3),
},
},
2016-01-05 17:21:53 +00:00
}, nil),
s3obj.EXPECT().CompleteMultipartUpload(&s3.CompleteMultipartUploadInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
MultipartUpload: &s3.CompletedMultipartUpload{
Parts: []*s3.CompletedPart{
{
ETag: aws.String("foo"),
PartNumber: aws.Int64(1),
},
{
ETag: aws.String("bar"),
PartNumber: aws.Int64(2),
},
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
{
ETag: aws.String("foobar"),
PartNumber: aws.Int64(3),
},
2016-01-05 17:21:53 +00:00
},
},
}).Return(nil, nil),
)
err := store.FinishUpload("uploadId+multipartId")
assert.Nil(err)
}
func TestWriteChunk(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
store.MaxPartSize = 8
store.MinPartSize = 4
store.MaxMultipartParts = 10000
store.MaxObjectSize = 5 * 1024 * 1024 * 1024 * 1024
2016-01-05 17:21:53 +00:00
gomock.InOrder(
s3obj.EXPECT().GetObject(&s3.GetObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId.info"),
}).Return(&s3.GetObjectOutput{
Body: ioutil.NopCloser(bytes.NewReader([]byte(`{"ID":"uploadId","Size":500,"Offset":0,"MetaData":null,"IsPartial":false,"IsFinal":false,"PartialUploads":null}`))),
}, nil),
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumberMarker: aws.Int64(0),
2016-01-05 17:21:53 +00:00
}).Return(&s3.ListPartsOutput{
Parts: []*s3.Part{
{
Size: aws.Int64(100),
},
{
Size: aws.Int64(200),
},
},
}, nil),
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumberMarker: aws.Int64(0),
2016-01-05 17:21:53 +00:00
}).Return(&s3.ListPartsOutput{
Parts: []*s3.Part{
{
Size: aws.Int64(100),
},
{
Size: aws.Int64(200),
},
},
}, nil),
s3obj.EXPECT().UploadPart(NewUploadPartInputMatcher(&s3.UploadPartInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumber: aws.Int64(3),
Body: bytes.NewReader([]byte("1234")),
})).Return(nil, nil),
s3obj.EXPECT().UploadPart(NewUploadPartInputMatcher(&s3.UploadPartInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumber: aws.Int64(4),
Body: bytes.NewReader([]byte("5678")),
})).Return(nil, nil),
s3obj.EXPECT().UploadPart(NewUploadPartInputMatcher(&s3.UploadPartInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumber: aws.Int64(5),
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
Body: bytes.NewReader([]byte("90AB")),
2016-01-05 17:21:53 +00:00
})).Return(nil, nil),
)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
// The last bytes "CD" will be ignored, as they are not the last bytes of the
// upload (500 bytes total) and not of full part-size.
bytesRead, err := store.WriteChunk("uploadId+multipartId", 300, bytes.NewReader([]byte("1234567890ABCD")))
2016-01-05 17:21:53 +00:00
assert.Nil(err)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
assert.Equal(int64(12), bytesRead)
2016-01-05 17:21:53 +00:00
}
func TestWriteChunkDropTooSmall(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
2016-01-05 17:21:53 +00:00
gomock.InOrder(
s3obj.EXPECT().GetObject(&s3.GetObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId.info"),
}).Return(&s3.GetObjectOutput{
Body: ioutil.NopCloser(bytes.NewReader([]byte(`{"ID":"uploadId","Size":500,"Offset":0,"MetaData":null,"IsPartial":false,"IsFinal":false,"PartialUploads":null}`))),
}, nil),
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumberMarker: aws.Int64(0),
2016-01-05 17:21:53 +00:00
}).Return(&s3.ListPartsOutput{
Parts: []*s3.Part{
{
Size: aws.Int64(100),
},
{
Size: aws.Int64(200),
},
},
}, nil),
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumberMarker: aws.Int64(0),
2016-01-05 17:21:53 +00:00
}).Return(&s3.ListPartsOutput{
Parts: []*s3.Part{
{
Size: aws.Int64(100),
},
{
Size: aws.Int64(200),
},
},
}, nil),
)
bytesRead, err := store.WriteChunk("uploadId+multipartId", 300, bytes.NewReader([]byte("1234567890")))
assert.Nil(err)
assert.Equal(int64(0), bytesRead)
}
func TestWriteChunkAllowTooSmallLast(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
2016-01-05 17:21:53 +00:00
store.MinPartSize = 20
gomock.InOrder(
s3obj.EXPECT().GetObject(&s3.GetObjectInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId.info"),
}).Return(&s3.GetObjectOutput{
Body: ioutil.NopCloser(bytes.NewReader([]byte(`{"ID":"uploadId","Size":500,"Offset":0,"MetaData":null,"IsPartial":false,"IsFinal":false,"PartialUploads":null}`))),
}, nil),
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumberMarker: aws.Int64(0),
2016-01-05 17:21:53 +00:00
}).Return(&s3.ListPartsOutput{
Parts: []*s3.Part{
{
Size: aws.Int64(400),
},
{
Size: aws.Int64(90),
},
},
}, nil),
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumberMarker: aws.Int64(0),
2016-01-05 17:21:53 +00:00
}).Return(&s3.ListPartsOutput{
Parts: []*s3.Part{
{
Size: aws.Int64(400),
},
{
Size: aws.Int64(90),
},
},
}, nil),
s3obj.EXPECT().UploadPart(NewUploadPartInputMatcher(&s3.UploadPartInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumber: aws.Int64(3),
Body: bytes.NewReader([]byte("1234567890")),
})).Return(nil, nil),
)
// 10 bytes are missing for the upload to be finished (offset at 490 for 500
// bytes file) but the minimum chunk size is higher (20). The chunk is
// still uploaded since the last part may be smaller than the minimum.
bytesRead, err := store.WriteChunk("uploadId+multipartId", 490, bytes.NewReader([]byte("1234567890")))
assert.Nil(err)
assert.Equal(int64(10), bytesRead)
}
func TestTerminate(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
// Order is not important in this situation.
s3obj.EXPECT().AbortMultipartUpload(&s3.AbortMultipartUploadInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
}).Return(nil, nil)
s3obj.EXPECT().DeleteObjects(&s3.DeleteObjectsInput{
Bucket: aws.String("bucket"),
Delete: &s3.Delete{
Objects: []*s3.ObjectIdentifier{
{
Key: aws.String("uploadId"),
},
{
Key: aws.String("uploadId.info"),
},
},
Quiet: aws.Bool(true),
},
}).Return(&s3.DeleteObjectsOutput{}, nil)
err := store.Terminate("uploadId+multipartId")
assert.Nil(err)
}
func TestTerminateWithErrors(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
// Order is not important in this situation.
// NoSuchUpload errors should be ignored
s3obj.EXPECT().AbortMultipartUpload(&s3.AbortMultipartUploadInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
}).Return(nil, awserr.New("NoSuchUpload", "The specified upload does not exist.", nil))
s3obj.EXPECT().DeleteObjects(&s3.DeleteObjectsInput{
Bucket: aws.String("bucket"),
Delete: &s3.Delete{
Objects: []*s3.ObjectIdentifier{
{
Key: aws.String("uploadId"),
},
{
Key: aws.String("uploadId.info"),
},
},
Quiet: aws.Bool(true),
},
}).Return(&s3.DeleteObjectsOutput{
Errors: []*s3.Error{
{
Code: aws.String("hello"),
Key: aws.String("uploadId"),
Message: aws.String("it's me."),
},
},
}, nil)
err := store.Terminate("uploadId+multipartId")
2016-09-27 20:10:16 +00:00
assert.Equal("Multiple errors occurred:\n\tAWS S3 Error (hello) for object uploadId: it's me.\n", err.Error())
}
func TestConcatUploads(t *testing.T) {
mockCtrl := gomock.NewController(t)
defer mockCtrl.Finish()
assert := assert.New(t)
s3obj := NewMockS3API(mockCtrl)
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
store := New("bucket", s3obj)
s3obj.EXPECT().UploadPartCopy(&s3.UploadPartCopyInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
CopySource: aws.String("bucket/aaa"),
PartNumber: aws.Int64(1),
}).Return(nil, nil)
s3obj.EXPECT().UploadPartCopy(&s3.UploadPartCopyInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
CopySource: aws.String("bucket/bbb"),
PartNumber: aws.Int64(2),
}).Return(nil, nil)
s3obj.EXPECT().UploadPartCopy(&s3.UploadPartCopyInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
CopySource: aws.String("bucket/ccc"),
PartNumber: aws.Int64(3),
}).Return(nil, nil)
// Output from s3Store.FinishUpload
gomock.InOrder(
s3obj.EXPECT().ListParts(&s3.ListPartsInput{
s3store: calculate part size based on upload size See https://github.com/tus/tusd/issues/149 and https://github.com/tus/tusd/pull/150 for more details. Squashed commit of the following: commit 78312ab26ea7ee664038e5b5d362bd534bfe0e37 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:49:48 2017 +0200 Correct error assertions for exceeding max part size commit 9350712c0a46651e6a7a91d8819307ba4b08ec7e Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:44:28 2017 +0200 Make CalcOptimalPartSize unexported commit 593f3b2d37d16c51f229572c1d6b39fc2a234079 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:38:46 2017 +0200 Add more output for debugging tests commit b7193bfe67b535c9b9dd441610b41af11fe4538f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:35:48 2017 +0200 Extract size assertions into own function commit 7521de23194652519fbbf3d61a41ef0b44b005fa Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:26:48 2017 +0200 Move tests for CalcPartSize into own file commit 6c483de7710cc119c870271ccad629c98c15c9a3 Author: Marius <maerious@gmail.com> Date: Fri Sep 1 19:13:02 2017 +0200 Use same assertions in AllUploadSizes test commit 7b0290a07e7def09ea8ed982e7817a2ea7cd468a Author: Marius <maerious@gmail.com> Date: Fri Sep 1 18:30:02 2017 +0200 Split negative test case from TestCalcOptimalPartSize into own test commit 79c0a20d7bc71b494bc0824ad2aa8879b0c2900b Merge: 5240f9b 997961f Author: Marius <maerious@gmail.com> Date: Fri Sep 1 17:32:31 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 997961ff5cc3d59c9055e54335926929f9ed570a Author: Markus Kienast <mark@rickkiste.at> Date: Fri Sep 1 00:59:38 2017 +0200 TestNewUploadLargerMaxObjectSize commit 0831bd79f8d34c0d5d079d1750852754db36ecdf Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 31 23:08:03 2017 +0200 fmt.Sprintf removed, range from 0 - MaxObjectSize+1 commit 1be708152409f6a918a8dee86d704d1a9f25690b Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:23:50 2017 +0200 turn off debug mode commit be9a9bec10e0681572c6a1aa588fd158d913c008 Author: Markus Kienast <mark@rickkiste.at> Date: Tue Aug 29 10:12:20 2017 +0200 moved MaxObjectSize check to NewUpload, refined tests * moved MaxObjectSize check to NewUpload * removed MaxObjectSize check from CalcOptimalPartSize * switched to assert in tests * added TestAllPartSizes, excluded in short mode TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size commit 7c22847a451c8f70ef3b312111267b699ff00693 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:55:07 2017 +0200 adding debug code to TestCalcOptimalPartSize commit 5240f9b549000fac34be79ddfbe6e82404387f6b Merge: 63c011e 5b116e7 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:51 2017 +0200 Merge branch 'f-s3-part-size' of https://github.com/flaneurtv/tusd into flaneurtv-f-s3-part-size commit 63c011ef768db42e99004df921c2b9e5c4776fd2 Author: Marius <maerious@gmail.com> Date: Sat Aug 26 12:50:45 2017 +0200 Format s3store_test commit 5b116e70875789eff95e490353b5d232c21f8660 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:24:22 2017 +0200 restructuring tests to accommodate optimalPartSize of 0 commit 93134a5696e3755d35e3ab360ba393cd3e52b6f0 Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 12:03:18 2017 +0200 moving MaxObjectSize check to top commit 68e6bb8c41f92e29dfbf37bcb391bc5256ebb90a Author: Markus Kienast <mark@rickkiste.at> Date: Sat Aug 26 02:31:27 2017 +0200 enhance readability, comments and errors commit 8831a98c34d0a702fe24b6450928830569dac351 Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 02:27:57 2017 +0200 separated partsize calc and error handling commit f059acc7ccc443405cf9d465f619319483ed947a Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 24 01:29:26 2017 +0200 fixed edge cases; pre-cleanup commit e2e3b9ffe4aeef70ed44bc63b6afc5e4c353c159 Author: Markus Kienast <mark@rickkiste.at> Date: Wed Aug 23 13:28:59 2017 +0200 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt commit 381d3326cb17a173b13eced781a1ae6efc88773b Author: Markus Kienast <mark@rickkiste.at> Date: Thu Aug 17 16:32:25 2017 +0200 calculating PartSize based on size of upload simplified algorithm, respect MaxObjectSize, updated tests, go fmt commit 1ad6187d6dc199018605e3e3a7d9d4c7d2c37cf8 Author: koenvo <info@koenvossen.nl> Date: Thu Aug 17 21:31:37 2017 +0200 Take IsTruncated field of S3 ListParts API response into account (#148) * Take IsTruncated field of S3 ListParts API response into account * Rename s3store.ListParts to ListAllParts * Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo * Update TestFinishUpload to also test paged ListParts response
2017-09-03 08:57:06 +00:00
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
PartNumberMarker: aws.Int64(0),
}).Return(&s3.ListPartsOutput{
Parts: []*s3.Part{
{
ETag: aws.String("foo"),
PartNumber: aws.Int64(1),
},
{
ETag: aws.String("bar"),
PartNumber: aws.Int64(2),
},
{
ETag: aws.String("baz"),
PartNumber: aws.Int64(3),
},
},
}, nil),
s3obj.EXPECT().CompleteMultipartUpload(&s3.CompleteMultipartUploadInput{
Bucket: aws.String("bucket"),
Key: aws.String("uploadId"),
UploadId: aws.String("multipartId"),
MultipartUpload: &s3.CompletedMultipartUpload{
Parts: []*s3.CompletedPart{
{
ETag: aws.String("foo"),
PartNumber: aws.Int64(1),
},
{
ETag: aws.String("bar"),
PartNumber: aws.Int64(2),
},
{
ETag: aws.String("baz"),
PartNumber: aws.Int64(3),
},
},
},
}).Return(nil, nil),
)
err := store.ConcatUploads("uploadId+multipartId", []string{
"aaa+AAA",
"bbb+BBB",
"ccc+CCC",
})
assert.Nil(err)
}