Skip to main content

I have a need to use keboola to check for the existence of a file in s3. Is there a good way to accomplish this?

Where am I?

In Keboola you can ask and answer questions and share your experience with others!

Martin Fiser
Martin Fiser Keboola TEAM

yes, or a python code that utilized Boto3 library to check that out.

Leo Chan
Leo Chan Keboola TEAM

Yes this will work. Unless it is a custom component, I would recommend the generic extractor approach since it allows you to encrypt your s3 tokens.

Milan Veverka
Milan Veverka Keboola TEAM

Cool use case! I would think generic extractor to call the ListObject API , do you agree @Martin Fiser , @David Esner or @Leo Chan ?

David Esner
David Esner Keboola TEAM

I think this would be a bit complicated to setup but doable. Authentication header and signature must be constructed via HMAC hash: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html It would need to use the unsigned payload option. The extractor supports sha256 https://developers.keboola.com/extend/generic-extractor/functions/#hash_hmac so it should be doable.

Martin Fiser
Martin Fiser Keboola TEAM

Or test that out in the python code (using BOTO which does plenty of stuff for you) and then bundle that as a component - yes, there is an overhead but not really that big and you would have it simpler to work on that actual code - plus it can enrich the community :)

Related Posts