Pēteris Caune
9660bc293c
Update hc.lib.s3 to retry failed requests *one time*
2024-07-17 17:26:49 +03:00
Pēteris Caune
ce5d9bcf56
Re-enable S3 retries
2024-07-17 17:04:33 +03:00
Pēteris Caune
1877a8324f
Disable S3 API request retries
...
urrlib3's default number of retries is 3.
If requests to the S3 API are timing out, the retries usually
don't help, but a 10-second timeout turns into 10*3=30 seconds
of python code being blocked.
2024-07-12 03:09:21 +03:00
Pēteris Caune
475bdff87d
Add handling for minio.InvalidResponseError exceptions
2024-06-18 23:50:11 +03:00
Pēteris Caune
274a59956a
Make statsd metrics collection optional
...
To enable, set STATSD_HOST env var (or set STATSD_HOST in
local_settings.py):
STATSD_HOST=localhost:1234
cc: #974
2024-03-18 12:55:36 +02:00
Pēteris Caune
7cb47188b9
Remove redundant cast
2024-01-22 16:09:20 +02:00
Pēteris Caune
42f88f4fb0
Add S3_SECURE setting
...
It controls whether to use secure (TLS) connection to S3 or not.
2024-01-10 12:19:45 +02:00
Pēteris Caune
ae94648efd
Fix minio-related mypy warnings
...
A few warnings remain, I think these should be fixed in minio-py:
https://github.com/minio/minio-py/pull/1389
2024-01-04 16:01:34 +02:00
Pēteris Caune
767c3ae702
Add a management command for pruning all checks
2023-12-21 14:55:05 +02:00
Pēteris Caune
b9df18728a
Improve error logging in hc.lib.s3._remove_objects
2023-11-24 10:11:31 +02:00
Pēteris Caune
ec9649a6a6
Fix logging calls to avoid string formatting in client code
2023-10-27 10:33:55 +03:00
Pēteris Caune
ab7449a279
Improve type hints in hc.lib.s3
2023-10-26 10:19:28 +03:00
Pēteris Caune
ee40cd8dab
Fix type warnings
2023-09-08 16:47:27 +03:00
Pēteris Caune
5746bbb015
Improve type hints in hc.api.models
2023-09-08 15:58:39 +03:00
Pēteris Caune
aaa172cd28
Improve type hints
2023-08-29 17:52:20 +03:00
Pēteris Caune
f6a04d2256
Fix lint issues
2023-07-12 10:07:14 +03:00
Pēteris Caune
475cd574ee
Improve logging in hc.lib.s3._remove_objects
2023-06-12 11:25:58 +03:00
Pēteris Caune
9ead449d90
Add error logging in hc.lib.s3.get_object
2023-04-14 15:14:23 +03:00
Pēteris Caune
dcd174f761
Add statsd metric collection in hc.lib.s3.get_object()
2023-04-14 13:23:16 +03:00
Pēteris Caune
f6aeda978d
Fix hc.lib.s3.get_object to handle more urllib3 exceptions
2023-03-29 14:00:10 +03:00
Pēteris Caune
3992c0927b
Add handling for ProtocolError exceptions in hc.lib.s3.get_object
2023-02-01 09:31:15 +02:00
Pēteris Caune
161430fb10
Sort imports and add "from __future__ import annotations"
2022-10-17 16:52:15 +03:00
Pēteris Caune
58da098d6b
Add timeout handling in hc.lib.s3._remove_objects()
2022-08-25 10:54:59 +03:00
Pēteris Caune
7695010398
Document hc.lib.curl.request
2022-08-17 13:07:20 +03:00
Pēteris Caune
3effd77e70
Add retry limit in hc.lib.s3.put_object
2022-07-01 23:47:47 +03:00
Pēteris Caune
527bc32d67
Fix hc.lib.s3.get_object to handle missing S3 credentials
2022-03-14 10:21:11 +02:00
Pēteris Caune
7ae0849652
Fix crash in _remove_objects when n is negative
2022-03-11 14:33:46 +02:00
Pēteris Caune
beda1f1a05
Remove dead code
2022-03-04 17:37:05 +02:00
Pēteris Caune
c1ac2ffc9c
Fix s3.py to reuse Minio client and retry InternalError errors
2022-03-04 17:33:13 +02:00
Pēteris Caune
1aad3e181b
Add "block" boolean argument to hc.lib.s3.remove_objects
2022-03-04 14:26:11 +02:00
Pēteris Caune
4422f2cf95
Add timeout for S3 operations
2022-03-04 13:09:38 +02:00
Pēteris Caune
5774757843
Add handling for missing S3 objects
2022-03-04 12:48:28 +02:00
Pēteris Caune
6b82f1d17f
Document the trickery in s3.py
2022-02-28 15:36:26 +02:00
Pēteris Caune
67edf5a442
Fix tests
2022-02-28 11:55:47 +02:00
Pēteris Caune
05bb80130b
Add support for storing ping bodies in S3-compatible object storage
...
This is an initial, minimal implementation. It is currently
missing: error handling, timeouts for S3 operations, documentation.
cc: #609
2022-02-28 11:51:04 +02:00
