How to write a Dictionary to JSON file in S3 Bucket using boto3 and Python

If you want to write a python dictionary to a JSON file in S3 then you can use the code examples below.

There are two code examples doing the same thing below because boto3 provides a client method and a resource method to edit and access AWS S3.

Writing Python Dictionary to an S3 Object using boto3 Client

import boto3
import json
from datetime import date

data_dict = {
    'Name': 'Daikon Retek',
    'Birthdate': date(2000, 4, 7),
    'Subjects': ['Math', 'Science', 'History']
}

# Convert Dictionary to JSON String
data_string = json.dumps(data_dict, indent=2, default=str)


# Upload JSON String to an S3 Object
client = boto3.client('s3')

client.put_object(
    Bucket='radishlogic-bucket', 
    Key='s3_folder/client_data.json',
    Body=data_string
)

Writing Python Dictionary to an S3 Object using boto3 Resource

import boto3
import json
from datetime import date

data_dict = {
    'Name': 'Daikon Retek',
    'Birthdate': date(2000, 4, 7),
    'Subjects': ['Math', 'Science', 'History']
}

# Convert Dictionary to JSON String
data_string = json.dumps(data_dict, indent=2, default=str)


# Upload JSON String to an S3 Object
s3_resource = boto3.resource('s3')

s3_bucket = s3_resource.Bucket(name='radishlogic-bucket')

s3_bucket.put_object(
    Key='s3_folder/resource_data.json',
    Body=data_string
)

You can select either of the examples to use when writing a python dictionary to a JSON object in an S3 Bucket.

Converting a Dictionary to JSON String

A python dictionary is a set of key-value pairs. It is the hash map or hash table of python.

If we simply print a dictionary, then we will get a single line of key-value pairs with single quotes that represent a string. Like the python example below.

import json
from datetime import date

data_dict = {
    'Name': 'Daikon Retek',
    'Birthdate': date(2000, 4, 7),
    'Subjects': ['Math', 'Science', 'History']
}

print(data_dict)

Output

{'Name': 'Daikon Retek', 'Birthdate': datetime.date(2000, 4, 7), 'Subjects': ['Math', 'Science', 'History']}

Notice that the print() only has single quotes (') for the strings. For some objects, it will show the object type like the datetime.date(2000, 4, 7).

JSON format requires double quotes (") to represent strings.

To convert a dictionary to a JSON formatted string we need to import the json package, then use json.dumps() method.

In the example below, we input the dictionary data_dict to json.dumps() with additional parameters of indent=2 and default=str.

import json
from datetime import date

data_dict = {
    'Name': 'Daikon Retek',
    'Birthdate': date(2000, 4, 7),
    'Subjects': ['Math', 'Science', 'History']
}

# Convert Dictionary to JSON String
data_string = json.dumps(data_dict, indent=2, default=str)

print(data_string)

Output

{
  "Name": "Daikon Retek",
  "Birthdate": "2000-04-07",
  "Subjects": [
    "Math",
    "Science",
    "History"
  ]
}

If we remove the indent=2 in json.dumps(), then it will remove the white spaces in the string and result in the following single-line JSON format.

{"Name": "Daikon Retek", "Birthdate": "2000-04-07", "Subjects": ["Math", "Science", "History"]}

Then if we remove the default=str then it will throw a TypeError: Object of type date is not JSON serializable since the date does not have a function that could automatically convert it to a string (or serialization).

The default parameter is the function that json.dumps() will use to convert any non-serializable objects in the dictionary to a JSON formatted string.

Once we get the JSON string (data_string in the example code), we will now be able to write the string to an S3 object.

Lambda Function writing Python Dictionary to a JSON S3 Object

If you are planning to write the dictionary to an S3 object from a Lambda Function using Python then the codes will help you.

These are the same codes as above but they are formatted for use inside a Lambda function.

Lambda Function using boto3 S3 Client

import boto3
import json
from datetime import date

def lambda_handler(event, context):
    data_dict = {
        'Name': 'Daikon Retek',
        'Birthdate': date(2000, 4, 7),
        'Subjects': ['Math', 'Science', 'History']
    }
    
    # Convert Dictionary to JSON String
    data_string = json.dumps(data_dict, indent=2, default=str)
    
    
    # Upload JSON String to an S3 Object
    client = boto3.client('s3')
    
    client.put_object(
        Bucket='radishlogic-bucket', 
        Key='s3_folder/client_data.json',
        Body=data_string
    )

Lambda Function using boto3 S3 Resource

import boto3
import json
from datetime import date

def lambda_handler(event, context):
    data_dict = {
        'Name': 'Daikon Retek',
        'Birthdate': date(2000, 4, 7),
        'Subjects': ['Math', 'Science', 'History']
    }
    
    # Convert Dictionary to JSON String
    data_string = json.dumps(data_dict, indent=2, default=str)
    
    
    # Upload JSON String to an S3 Object
    s3_resource = boto3.resource('s3')
    
    s3_bucket = s3_resource.Bucket(name='radishlogic-bucket')
    
    s3_bucket.put_object(
        Key='s3_folder/resource_data.json',
        Body=data_string
    )

I hope this helps you write a Python Dictionary to a JSON file in an S3 Bucket in your project.

Let me know your experience, questions, or suggestions in the comments below.