Messed with PutS3Object, add signature on top on content

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Messed with PutS3Object, add signature on top on content

Etienne Jouvin
Hello all.

I am trying to use S3 storage (s3Ninja for "unit test") and I am really messed about how it works.
Simple process
image.png

I fetch a simple PDF content and try to post to s3.
But in the resulting content, I found the first line :
20000;chunk-signature=c0d176ad5f54df79ca61a1de74d71731f3bf2e92972d7c5ca25f79d83b99b63f

I am trying to google this, and I found that it may come from chunk upload.
But the file is small, less than 1 Mo.

It is also the case with simple text file.

I use Wireshark to track traffic, and I found that there is the signature added during the call.

Also, I always have an exception saying the signature is not valid :
com.amazonaws.SdkClientException: Unable to verify integrity of data upload. Client calculated content hash (contentMD5: RJHSWwvsGtV1rweS3gmThQ== in base 64) didn't match hash (etag: 9a86aae10bb9318d22c3ddea1f93fd07 in hex) calculated by Amazon S3.  You may need to delete the data stored in Amazon S3. (metadata.contentMD5: null, md5DigestStream: com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream@239e8a25, bucketName: renditions, key: 43cc8b40-0f36-4bb4-9ad6-ea079ecbd5a1.pdf)

Any idea or input for investigation ?

Regards

Etienne Jouvin

Reply | Threaded
Open this post in threaded view
|

Re: Messed with PutS3Object, add signature on top on content

Etienne Jouvin
Hello all.

Got it.
It was a matter of signature version.
Because I use s3Ninja, I needed to set with V2 :
image.png





Le ven. 26 juin 2020 à 15:43, Etienne Jouvin <[hidden email]> a écrit :
Hello all.

I am trying to use S3 storage (s3Ninja for "unit test") and I am really messed about how it works.
Simple process
image.png

I fetch a simple PDF content and try to post to s3.
But in the resulting content, I found the first line :
20000;chunk-signature=c0d176ad5f54df79ca61a1de74d71731f3bf2e92972d7c5ca25f79d83b99b63f

I am trying to google this, and I found that it may come from chunk upload.
But the file is small, less than 1 Mo.

It is also the case with simple text file.

I use Wireshark to track traffic, and I found that there is the signature added during the call.

Also, I always have an exception saying the signature is not valid :
com.amazonaws.SdkClientException: Unable to verify integrity of data upload. Client calculated content hash (contentMD5: RJHSWwvsGtV1rweS3gmThQ== in base 64) didn't match hash (etag: 9a86aae10bb9318d22c3ddea1f93fd07 in hex) calculated by Amazon S3.  You may need to delete the data stored in Amazon S3. (metadata.contentMD5: null, md5DigestStream: com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream@239e8a25, bucketName: renditions, key: 43cc8b40-0f36-4bb4-9ad6-ea079ecbd5a1.pdf)

Any idea or input for investigation ?

Regards

Etienne Jouvin

Reply | Threaded
Open this post in threaded view
|

Re: Messed with PutS3Object, add signature on top on content

Pierre Villard
Glad you figured it out and thanks for sharing your findings with the community!

Le ven. 26 juin 2020 à 16:49, Etienne Jouvin <[hidden email]> a écrit :
Hello all.

Got it.
It was a matter of signature version.
Because I use s3Ninja, I needed to set with V2 :
image.png





Le ven. 26 juin 2020 à 15:43, Etienne Jouvin <[hidden email]> a écrit :
Hello all.

I am trying to use S3 storage (s3Ninja for "unit test") and I am really messed about how it works.
Simple process
image.png

I fetch a simple PDF content and try to post to s3.
But in the resulting content, I found the first line :
20000;chunk-signature=c0d176ad5f54df79ca61a1de74d71731f3bf2e92972d7c5ca25f79d83b99b63f

I am trying to google this, and I found that it may come from chunk upload.
But the file is small, less than 1 Mo.

It is also the case with simple text file.

I use Wireshark to track traffic, and I found that there is the signature added during the call.

Also, I always have an exception saying the signature is not valid :
com.amazonaws.SdkClientException: Unable to verify integrity of data upload. Client calculated content hash (contentMD5: RJHSWwvsGtV1rweS3gmThQ== in base 64) didn't match hash (etag: 9a86aae10bb9318d22c3ddea1f93fd07 in hex) calculated by Amazon S3.  You may need to delete the data stored in Amazon S3. (metadata.contentMD5: null, md5DigestStream: com.amazonaws.services.s3.internal.MD5DigestCalculatingInputStream@239e8a25, bucketName: renditions, key: 43cc8b40-0f36-4bb4-9ad6-ea079ecbd5a1.pdf)

Any idea or input for investigation ?

Regards

Etienne Jouvin