Tag: Python

How to download files from S3 Bucket using boto3 and Python

If you want to download a file from an AWS S3 Bucket using Python, then you can use the sample codes below.

The codes below use AWS SDK for Python named boto3.

boto3 provides three methods to download a file.

  1. download_file()
  2. download_fileobj() – with multipart upload
  3. get_object()

Then for each method, you can use the client class or the resource class of boto3.

Both of the classes will be used for each of the methods above.

Note: All examples will work with any Python3 environment running in Windows, MacOS or Linux operating systems.

Continue reading How to download files from S3 Bucket using boto3 and Python

How to read a JSON file in S3 and store it in a Dictionary using boto3 and Python

If you want to get a JSON file from an S3 Bucket and load it into a Python Dictionary then you can use the example codes below.

There are 4 scenarios for the examples scripts below.

  1. Basic JSON file from S3 to Python Dictionary
  2. With Try/Except block
  3. With datetime, date, and time conversions
  4. Running the code in a Lambda Function

AWS boto3 provides 2 ways to access S3 files, the boto3.client('s3') and boto3.resource('s3'). For each of the example scenarios above, a code will be provided for the two methods.

Related: Writing a Dictionary to JSON file in S3 using boto3 and Python

Since both methods will function the same, you can choose whichever method you like.

Continue reading How to read a JSON file in S3 and store it in a Dictionary using boto3 and Python

How to read a file in S3 and store it in a String using Python and boto3

If you want to get a file from an S3 Bucket and then put it in a Python string, try the examples below.

boto3, the AWS SDK for Python, offers two distinct methods for accessing files or objects in Amazon S3: client method and the resource method.

Option 1 uses the boto3.client('s3') method, while options 2 and 3 use the boto3.resource('s3') method.

All 3 options do the exact same thing so get the one that you feel comfortable with or the one that will fit your use case.

Continue reading How to read a file in S3 and store it in a String using Python and boto3

boto3: Convert AMI Creation Date from string to Python datetime

When retrieving the AMI Creation Date from boto3 it returns a string data type. Visually, this is okay but it is challenging to do operations and comparisons to the AMI Creation Date in this format.

To solve the issue we need to convert the AMI Creation Date from type string to datetime before we could do some operations.

The AMI Creation Date string looks like 2019-09-18T07:34:34.000Z. To convert this we need to use the strptime function from the datetime.datetime library.

Continue reading boto3: Convert AMI Creation Date from string to Python datetime

Installing MySQLdb for Python 3 in Windows

My favorite Python connector for MySQL or MariaDB is MySQLdb, the problem with this connector is that it is complicated to install on Windows!

I am creating this article for those who want to install MySQLdb for Python 3 for Windows. Especially me, since each time I am doing a Python project that needs to connect to MariaDB or MySQL I always look on how to install MySQLdb.

If you are interested why I prefer MySQLdb compared to other MySQL connectors you may want to read the comparison of MySQL-connector and MySQLdb from Charles Nagy.

Problems with installing MySQLdb on Windows

You can actually install MySQLdb using pip. See pypi documentation here.

pip install MySQL-python

Unfortunately, the pypi documentation is already out of date with the latest release was on Jan 3, 2014. Continue reading Installing MySQLdb for Python 3 in Windows

AWS Lambda Console: Accessing Environment Variables via Python

Editing configuration values inside the code is a high risk for error since there is a high chance that not only the values that you are changing you will change, you might even delete a letter or edit a line. In order to avoid this risk, you want your code to be able to accept configuration values at run time. This is extremely useful when developing codes on different environments like development, testing and production.

With AWS Lambda you can reuse your code on different environments using the Environment Variables.

Below is the way to use Environment Variables on AWS Lambda Console using Python 3.6.

Note: This is the same way to use Environment Variables on Python 2.7 and Python 3.7.

Environment Variables Setup

The Environment Variables section can be found under the Function Code section.

Environment Variables are Key/Value Pairs. The Key is what you will use on your Lambda Code, to access its Value. Continue reading AWS Lambda Console: Accessing Environment Variables via Python