Commit Graph

31 Commits

Author SHA1 Message Date
Markus Kienast 1be7081524 turn off debug mode 2017-08-29 10:23:50 +02:00
Markus Kienast be9a9bec10 moved MaxObjectSize check to NewUpload, refined tests
* moved MaxObjectSize check to NewUpload
* removed MaxObjectSize check from CalcOptimalPartSize
* switched to assert in tests
* added TestAllPartSizes, excluded in short mode

TODO: TestNewUploadLargerMaxObjectSize needs to fail if MaxObjectSize > size
2017-08-29 10:12:20 +02:00
Markus Kienast 7c22847a45 adding debug code to TestCalcOptimalPartSize 2017-08-26 12:55:07 +02:00
Markus Kienast 5b116e7087 restructuring tests to accommodate optimalPartSize of 0 2017-08-26 12:24:22 +02:00
Markus Kienast 93134a5696 moving MaxObjectSize check to top 2017-08-26 12:03:18 +02:00
Markus Kienast 68e6bb8c41 enhance readability, comments and errors 2017-08-26 02:31:27 +02:00
Markus Kienast 8831a98c34 separated partsize calc and error handling 2017-08-24 02:27:57 +02:00
Markus Kienast f059acc7cc fixed edge cases; pre-cleanup 2017-08-24 01:29:26 +02:00
Markus Kienast e2e3b9ffe4 added error, when size > MaxObjectSize; additional case in algorithm + tests; go fmt 2017-08-23 13:28:59 +02:00
Markus Kienast 381d3326cb calculating PartSize based on size of upload
simplified algorithm, respect MaxObjectSize, updated tests, go fmt
2017-08-20 02:07:58 +02:00
koenvo 1ad6187d6d Take IsTruncated field of S3 ListParts API response into account (#148)
* Take IsTruncated field of S3 ListParts API response into account

* Rename s3store.ListParts to ListAllParts

* Use proper formatting + make listAllParts private + test listAllParts through TestGetInfo

* Update TestFinishUpload to also test paged ListParts response
2017-08-17 21:31:37 +02:00
Markus Kienast 028ba57206 Squashed commit of the following:
commit 5a268dbafb9318b888142931ea27a1af10b9a8e7
Author: Marius <maerious@gmail.com>
Date:   Wed Jul 19 11:47:26 2017 +0200

    Remove manual assignment of upload ID in S3Store

commit a37e149090ee7fd5f170d24ccc33b8af9ae18fae
Author: Marius <maerious@gmail.com>
Date:   Wed Jul 19 11:42:00 2017 +0200

    Format Go code

commit 6643a9be62
Author: Markus Kienast <mark@rickkiste.at>
Date:   Sun Jul 16 17:08:24 2017 +0200

    fixed ID value in .info; adjusted tests; fixed assert(expected, received) swap
2017-07-19 11:54:26 +02:00
Marius d48e4ac577 Correct when S3Store write loop is ended 2017-03-23 12:06:15 +01:00
Marius 37fe99b37d Define own, minimal S3API interface 2016-12-20 17:13:02 +01:00
Marius 513fe9fcf3 Correct linting issue and misspellings 2016-09-27 22:10:16 +02:00
Marius 1bf1c942e0 Update mocked S3API implementation for tests 2016-09-19 19:30:56 +02:00
Marius 3eefdbe2c6 Update mocked S3API implementation for tests 2016-08-26 17:14:33 +02:00
Marius 015ae39d18 Replace non-ASCII chars in metadata for multipart uploads with question mark
AWS does not handle non-ASCII encoded values for metadata values well since
they are transported in HTTP header values which, by specification, should
only contain ASCII characters. If you still supply AWS with, for example,
UTF-8 encoded strings it will reject the request due to mismatching
signatures. Our solution is to replace these characters with question
marks.
2016-07-06 16:25:06 +02:00
Marius 2f436ab7e3 Update mocked S3 API for tests 2016-05-24 17:32:21 +02:00
Marius 199487bdf6 Set correct ID in S3Store.GetInfo 2016-03-26 18:23:37 +01:00
Marius be8fd83d0d Document functions for handling StoreComposers 2016-03-11 20:17:43 +01:00
Marius 21ae1c45a7 Add initial draft of StoreComposr 2016-02-21 23:25:35 +01:00
Marius 11b92b91c2 Provide better URL to S3 consistency models 2016-02-11 12:23:46 +01:00
Marius bbf706622c Implement Concatentation extension for S3Store 2016-02-03 21:18:21 +01:00
Marius bfde73ff89 Test interface implementations for data stores 2016-01-19 22:37:05 +01:00
Marius 29047eddc2 Add support for meta data in S3Store 2016-01-19 21:39:24 +01:00
Marius 101648e382 Add documentation about S3Store 2016-01-17 15:30:19 +01:00
Marius 810fa89494 Add support for upload termination in S3Store 2016-01-16 16:12:37 +01:00
Marius cb0f8dac6a Add isAwsError helper method 2016-01-12 22:21:05 +01:00
Marius 2073521776 Add tests for S3Store 2016-01-05 18:21:53 +01:00
Marius 063d4a2442 Add first draft for s3Store 2015-12-08 22:26:37 +01:00