If you want to get a JSON file from an S3 Bucket and load it into a Python Dictionary then you can use the example codes below.
There are 4 scenarios for the examples scripts below.
Basic JSON file from S3 to Python Dictionary
With Try/Except block
With datetime, date, and time conversions
Running the code in a Lambda Function
AWS boto3 provides 2 ways to access S3 files, the
boto3.resource('s3'). For each of the example scenarios above, a code will be provided for the two methods.
Related: Writing a Dictionary to JSON file in S3 using boto3 and Python
Since both methods will function the same, you can choose whichever method you like.
Continue reading How to read a JSON file in S3 and store it in a Dictionary using boto3 and Python
Each resource in AWS has an Amazon Resource Name (ARN). An ARN is a unique identifier of your resource. Its value has no duplicate in other accounts and only exists in your account.
It’s used especially in IAM policies where you set which resources you will allow access to.
You can actually predict the ARN of an S3 Bucket since it has a standard format of
But if you are like me who is afraid of making a mistake typing the S3 bucket ARN, I prefer going to the AWS Console, searching for the S3 Bucket ARN, and copy-pasting it.
Follow the instructions below to get the S3 Bucket ARN.
Continue reading How to get the ARN of an S3 Bucket
There are many ways to download files from an S3 Bucket, but if you are downloading an entire S3 Bucket then I would recommend using AWS CLI and running the command
aws s3 sync s3://SOURCE_BUCKET LOCAL_DESTINATION.
In the examples below, I’m going to download the contents of my S3 Bucket named
My S3 Bucket in the AWS management console
Example 1: Download S3 Bucket to Current Local Folder
If you want to download the whole S3 Bucket in the same folder that you are in, then you should use the command
aws s3 sync s3://SOURCE_BUCKET ..
In our example S3 Bucket above, the AWS CLI will be like this.
Continue reading How to download all files in an S3 Bucket using AWS CLI