boto3 Archives - Radish Logic

How to load a JSON file from S3 to a Python dictionary using boto3

If you want to get a JSON file from an S3 Bucket and load it into a Python Dictionary then you can use the example codes below.

There are 4 scenarios for the examples scripts below.

Basic JSON file from S3 to Python Dictionary
With Try/Except block
With datetime, date, and time conversions
Running the code in a Lambda Function

AWS boto3 provides 2 ways to access S3 files, the boto3.client('s3') and boto3.resource('s3'). For each of the example scenarios above, a code will be provided for the two methods.

Since both method will function the same, you can choose whichever method you like.

How to read a file in S3 and store it in a String using Python and boto3

If you want to get a file from an S3 Bucket and then put it in a Python string, try the examples below.

boto3, the AWS SDK for Python, offers two distinct methods for accessing files or objects in Amazon S3: client method and the resource method.

Option 1 uses the boto3.client('s3') method, while options 2 and 3 use the boto3.resource('s3') method.

All 3 options do the exact same thing so get the one that you feel comfortable with or the one that will fit your use case.

How to write a Dictionary to JSON file in S3 Bucket using boto3 and Python

If you want to write a python dictionary to a JSON file in S3 then you can use the code examples below.

There are two code examples doing the same thing below because boto3 provides a client method and a resource method to edit and access AWS S3.

Writing Python Dictionary to an S3 Object using boto3 Client

import boto3
import json
from datetime import date

data_dict = {
    'Name': 'Daikon Retek',
    'Birthdate': date(2000, 4, 7),
    'Subjects': ['Math', 'Science', 'History']
}

# Convert Dictionary to JSON String
data_string = json.dumps(data_dict, indent=2, default=str)


# Upload JSON String to an S3 Object
client = boto3.client('s3')

client.put_object(
    Bucket='radishlogic-bucket', 
    Key='s3_folder/client_data.json',
    Body=data_string
)

boto3: Convert AMI Creation Date from string to Python datetime

When retrieving the AMI Creation Date from boto3 it returns a string data type. Visually, this is okay but it is challenging to do operations and comparisons to the AMI Creation Date in this format.

To solve the issue we need to convert the AMI Creation Date from type string to datetime before we could do some operations.

The AMI Creation Date string looks like 2019-09-18T07:34:34.000Z. To convert this we need to use the strptime function from the datetime.datetime library.