Radish Logic

How to write a Dictionary to JSON file in S3 Bucket using boto3 and Python

If you want to write a python dictionary to a JSON file in S3 then you can use the code examples below.

There are two code examples doing the same thing below because boto3 provides a client method and a resource method to edit and access AWS S3.

Writing Python Dictionary to an S3 Object using boto3 Client

import boto3
import json
from datetime import date

data_dict = {
    'Name': 'Daikon Retek',
    'Birthdate': date(2000, 4, 7),
    'Subjects': ['Math', 'Science', 'History']
}

# Convert Dictionary to JSON String
data_string = json.dumps(data_dict, indent=2, default=str)


# Upload JSON String to an S3 Object
client = boto3.client('s3')

client.put_object(
    Bucket='radishlogic-bucket', 
    Key='s3_folder/client_data.json',
    Body=data_string
)

How to upload a file to S3 Bucket using boto3 and Python

There are 3 ways to upload or copy a file from your local computer to an Amazon Web Services (AWS) S3 Bucket using boto3. All of these will be discussed in this post including multipart uploads.

The codes below will work if you are Windows, Mac, and Linux. Will also work if you working in a Lambda function using Python.

Each method will have an example using boto3 S3 client and S3 resource so you can use whatever method you are comfortable with.

upload_file method
upload_fileobj method (supports multipart upload)
put_object method

How to write Python string to a file in S3 Bucket using boto3

To write a file from a Python string directly to an S3 bucket we need to use the boto3 package.

There are 2 ways to write a file in S3 using boto3. The first is via the boto3 client, and the second is via the boto3 resource. Both of these methods will be shown below.

S3 objects and keys

If you are new to AWS S3, you might be confused with some of the terms. So we’ll define some of them here. If you already know what objects and keys are then you can skip this section.

S3 objects are the same as files. When we run the method put_object what it means is that we are putting a file into S3.

S3 keys are the same as the filename with its full path. So if we want to create an object in S3 with the name of filename.txt within the foobar folder then the key is foobar/filename.txt.

Now that we have clarified some of the AWS S3 terms, follow the details below to start writing Python strings directly to objects in S3.

How to connect Logitech M350 Pebble mouse to Windows 11

The Logitech M350 Pebble is so far the best mouse that I have used, and I have used a lot of mouse in my 30 years of existence.

I like it because it is wireless, lightweight, quiet, responds well and does not have ribs in the scroll bar. I can use the mouse the whole day without my hands or fingers getting numb.

Also, it has been a year since I bought the mouse and I have not even changed the battery yet.

When I first bought the Logitech M350 Pebble I got confused because it says that it has Bluetooth connection. But when I saw the wireless USB dongle, I thought I got tricked. As it turns out this mouse has both – a (1) wireless USB dongle mode and a (2) Bluetooth mode.

How to reverse the F-Keys of Logitech K380 Keyboard

I have been using my Logitech K380 Keyboard for about a week now and I’m very happy with it.

I like it because it is quiet, has some weight, and can connect and switch to 3 devices with a press of the button.

When I received it when I press the F-keys, its default behavior is to use alternate functions like control the volume and media.

If I wanted to use F1 or F2 I have to click on the fn key then press the F-key. As someone who uses a lot of the F-keys due to my work, this is annoying.

Luckily, I was able to solve this issue by reversing the behavior of the F-keys. Now when I press the F11 to full screen my chrome browser it will work without pressing the fn key.

I think some people call this process Fn Lock

Follow the instructions below to reverse the behavior of the F-keys.

EC2 with IAM Role: CloudFormation Sample Template

Creating an EC2 Instance with an IAM Role is easy when you do it via the AWS Console but doing this with CloudFormation is not as direct. You will need an Instance Profile to connect an EC2 with an IAM Role.

TL;DR: See the CloudFormation Template below.

How to get the ARN of an S3 Bucket

Each resource in AWS has an Amazon Resource Name (ARN). An ARN is a unique identifier of your resource. Its value has no duplicate in other accounts and only exists in your account.

It’s used especially in IAM policies where you set which resources you will allow access to.

You can actually predict the ARN of an S3 Bucket since it has a standard format of arn:aws:s3:::S3_BUCKET_NAME.

But if you are like me who is afraid of making a mistake typing the S3 bucket ARN, I prefer going to the AWS Console, searching for the S3 Bucket ARN, and copy-pasting it.

Follow the instructions below to get the S3 Bucket ARN.

How to install WSL on multiple Windows User Accounts

If you are using multiple user accounts in your Windows computer and have installed Windows Subsystem for Linux (WSL) in one of your users, when you try to run WSL in another account you will encounter the WSL has no installed distributions error.

If you try going to another Windows profile, you will end up with the same error.

Minimum IAM Permission to create S3 presigned URLs

If you wanted to publicly share a file or an object inside a private S3 bucket you will need to create an S3 presigned URL. This will create a temporary link to the S3 file which you can share and access publicly.

As best practice, we must apply the least privileged permission to the IAM user or IAM role that will create the S3 presigned URL. Which brings us to the question, what is the minimum IAM permission to create an S3 presigned URL?

How to download all files in an S3 Bucket using AWS CLI

There are many ways to download files from an S3 Bucket, but if you are downloading an entire S3 Bucket then I would recommend using AWS CLI and running the command aws s3 sync s3://SOURCE_BUCKET LOCAL_DESTINATION.

In the examples below, I’m going to download the contents of my S3 Bucket named radishlogic-bucket.

My S3 Bucket in the AWS Console — My S3 Bucket in the AWS management console

Example 1: Download S3 Bucket to Current Local Folder

If you want to download the whole S3 Bucket in the same folder that you are in, then you should use the command aws s3 sync s3://SOURCE_BUCKET ..

In our example S3 Bucket above, the AWS CLI will be like this.