Commit Graph

1058 Commits

Author SHA1 Message Date
Derrick Hammer b3df326980
feat: add support for recursively pinning a manifest and all its children 2024-02-29 12:02:49 -05:00
Derrick Hammer 4126c06cd8
dep: update libs5 2024-02-29 11:33:40 -05:00
Derrick Hammer 99a6f3a2f6
fix: StorageLocationTypeFile/StorageLocationTypeFull expiry should be an hour 2024-02-29 10:16:16 -05:00
Derrick Hammer e32a9b2070
fix: only say we have the file if no error 2024-02-29 09:55:51 -05:00
Derrick Hammer 64f52a87bd
fix: return on any error 2024-02-29 09:55:07 -05:00
Derrick Hammer 754ab390f0
refactor: add build tag to s5 specific tables 2024-02-28 21:24:06 -05:00
Derrick Hammer bb167ea360
refactor: use a dynamic model registration process so we can use build tags to exclude/include 2024-02-28 21:20:05 -05:00
Derrick Hammer 4fe856fd03
dep: update libs5 2024-02-28 14:16:50 -05:00
Derrick Hammer 7abba4ac8e
refactor: move averageReadTime out of loop 2024-02-28 12:59:11 -05:00
Derrick Hammer 68d82390ed
refactor: add timing tracking for reading 2024-02-28 12:56:39 -05:00
Derrick Hammer dd9fdabf47
fix: move verification time logging outside loop and only record average 2024-02-28 12:41:03 -05:00
Derrick Hammer 0b000bfc89
fix: track after buffer write 2024-02-28 12:37:32 -05:00
Derrick Hammer c984d72cfd
refactor: add performance logging for bao 2024-02-28 12:33:13 -05:00
Derrick Hammer 0d0ec43125
refactor: add eta to logging 2024-02-28 12:20:16 -05:00
Derrick Hammer a49da3fdfe
fix: partStartTime needs to include reading 2024-02-28 12:16:47 -05:00
Derrick Hammer 5f0e2d2e15
refactor: add performance tracking and logging 2024-02-28 12:13:26 -05:00
Derrick Hammer b030de9714
feat: add generic s3 upload tracking 2024-02-28 11:36:53 -05:00
Derrick Hammer b2325eb9af
refactor: keep filename singular 2024-02-28 11:17:31 -05:00
Derrick Hammer 55203fa466
refactor: add more debug info 2024-02-28 11:08:05 -05:00
Derrick Hammer fba98da0e0
refactor: enable tracking the called line 2024-02-28 11:06:35 -05:00
Derrick Hammer 844d3a0a5f
refactor: add debug log line 2024-02-28 10:59:44 -05:00
Derrick Hammer e45ab26a09
refactor: add error message support to verify 2024-02-28 10:49:15 -05:00
Derrick Hammer 72fed662e8
fix: use io.ReadFull 2024-02-28 10:42:01 -05:00
Derrick Hammer b756ad73e9
fix: skip unexported fields 2024-02-28 09:50:42 -05:00
Derrick Hammer 6b7724eb51
fix: skip unexported fields 2024-02-28 09:48:20 -05:00
Derrick Hammer 0ae69e5ba1
fix: ensure m.root.Core.Clustered is not nil 2024-02-28 09:45:34 -05:00
Derrick Hammer 80d34000a0
refactor: store protocol config in root 2024-02-28 09:43:06 -05:00
Derrick Hammer bfb8559e32
fix: hack: make Protocol an interface map 2024-02-28 09:42:06 -05:00
Derrick Hammer 586a6dc205
fix: maybeConfigureCluster called in the wrong place 2024-02-28 09:36:01 -05:00
Derrick Hammer fcf43e1f79
refactor: add cluster configuration support to set db cache based on the cluster settings 2024-02-28 09:21:20 -05:00
Derrick Hammer d71849493b
chore: unneeded method 2024-02-28 09:16:35 -05:00
Derrick Hammer 9b82da72ca
refactor: move redis to its own file and add in defaults 2024-02-28 09:14:11 -05:00
Derrick Hammer 960c2b01d9
refactor: move all config defaults and add some validations. remove initCheckRequiredConfig. 2024-02-28 09:04:47 -05:00
Derrick Hammer cb558cdfc3
feat: add reflection-based system to manage defaults and validations per struct 2024-02-28 08:47:33 -05:00
Derrick Hammer 1a20a7d35f
refactor: define a cluster config with redis and etcd support 2024-02-28 08:03:36 -05:00
Derrick Hammer 7edab13afe
refactor: define Protocol under Config for completeness 2024-02-28 06:11:04 -05:00
Derrick Hammer d9d85f2804
fix: casing 2024-02-27 11:14:38 -05:00
Derrick Hammer 5878d1557f
refactor: switch using hkdf for child key 2024-02-27 11:10:46 -05:00
Derrick Hammer 097e29aa94
refactor: switch to using All api then filter to find valid locations 2024-02-27 10:58:15 -05:00
Derrick Hammer cdb23540ca
dep: update libs5 2024-02-27 10:51:09 -05:00
Derrick Hammer 9782907c90
fix: increase iterations to 600k 2024-02-27 09:54:11 -05:00
Derrick Hammer 1ca46dddd4
fix: use proto.identity 2024-02-27 09:15:18 -05:00
Derrick Hammer b5509f11d1
fix: use a deterministic child key 2024-02-27 09:11:17 -05:00
Derrick Hammer 944c5f01b9
refactor: add support for ExcludeNodes via excludeSelf in newStorageLocationProvider 2024-02-27 07:31:59 -05:00
Derrick Hammer f9e8d4e2fc
dep: update libs5 2024-02-27 07:28:43 -05:00
Derrick Hammer d002c56ffe
fix: set no limit on message size 2024-02-27 07:16:00 -05:00
Derrick Hammer c8a6570e8f
refactor: bao verifier needs to manage a data buffer to allow any number of bytes read while verifying in chunks 2024-02-27 06:37:49 -05:00
Derrick Hammer 9f5b676f47
refactor: use S3MultipartUpload if greater than S3_MULTIPART_MIN_PART_SIZE 2024-02-27 06:14:27 -05:00
Derrick Hammer 7411228106
feat: add S3MultipartUpload api 2024-02-27 06:12:41 -05:00
Derrick Hammer 01eda4aa23
fix: need to pass content length 2024-02-27 04:31:55 -05:00