Tag: Python 3

How to read a JSON file in S3 and store it in a Dictionary using boto3 and Python

If you want to get a JSON file from an S3 Bucket and load it into a Python Dictionary then you can use the example codes below.

There are 4 scenarios for the examples scripts below.

  1. Basic JSON file from S3 to Python Dictionary
  2. With Try/Except block
  3. With datetime, date, and time conversions
  4. Running the code in a Lambda Function

AWS boto3 provides 2 ways to access S3 files, the boto3.client('s3') and boto3.resource('s3'). For each of the example scenarios above, a code will be provided for the two methods.

Related: Writing a Dictionary to JSON file in S3 using boto3 and Python

Since both methods will function the same, you can choose whichever method you like.

Continue reading How to read a JSON file in S3 and store it in a Dictionary using boto3 and Python

How to download files from S3 Bucket using boto3 and Python

If you want to download a file from an AWS S3 Bucket using Python, then you can use the sample codes below.

The codes below use AWS SDK for Python named boto3.

boto3 provides three methods to download a file.

  1. download_file()
  2. download_fileobj() – with multipart upload
  3. get_object()

Then for each method, you can use the client class or the resource class of boto3.

Both of the classes will be used for each of the methods above.

Note: All examples will work with any Python3 environment running in Windows, MacOS or Linux operating systems.

Continue reading How to download files from S3 Bucket using boto3 and Python

Convert boto3 AMI Creation Date from string to Python datetime

When retrieving the AMI Creation Date from boto3 it returns a string data type. Visually, this is okay but it is challenging to do operations and comparisons to the AMI Creation Date in this format.

To solve the issue we need to convert the AMI Creation Date from type string to datetime before we could do some operations.

The AMI Creation Date string looks like 2019-09-18T07:34:34.000Z. To convert this we need to use the strptime function from the datetime.datetime library.

Continue reading Convert boto3 AMI Creation Date from string to Python datetime

Installing MySQLdb for Python 3 in Windows

My favorite Python connector for MySQL or MariaDB is MySQLdb, the problem with this connector is that it is complicated to install on Windows!

I am creating this article for those who want to install MySQLdb for Python 3 for Windows. Especially me, since each time I am doing a Python project that needs to connect to MariaDB or MySQL I always look on how to install MySQLdb.

If you are interested why I prefer MySQLdb compared to other MySQL connectors you may want to read the comparison of MySQL-connector and MySQLdb from Charles Nagy.

Problems with installing MySQLdb on Windows

You can actually install MySQLdb using pip. See pypi documentation here.

pip install MySQL-python

Unfortunately, the pypi documentation is already out of date with the latest release was on Jan 3, 2014. Continue reading Installing MySQLdb for Python 3 in Windows