Image Upload and Retrieval from S3 Using AWS API Gateway and Lambda

The new upcoming hero in the computer world is AWS. You can see this everywhere, and every time and you will be in need of this more than anything else in the future.

I tried few things out to check how this new trend works and found out few things that I would like to share with all of you.

In this blog, I will be basically talking about Amazon API Gateway, Lambda, and S3.

I will be talking about more on these services later on, from another blog. So stay tuned!!! 🙂

So basically what I tried out was uploading an image to an S3 bucket through a POST REST call done through the API Gateway and vice versa, retrieving an uploaded image from the S3 bucket from a GET REST call through the API Gateway.

The following diagram shows how my uploading is done using the 3 AWS services, API Gateway, Lambda, and S3.

Figure 1: Service Integration

First Lets focus on uploading an image to S3.

Uploading an Image to S3

For uploading, you will have to configure the API Gateway service by creating a new API.

Creating a new API with API Gateway

Figure 2: Create API in API Gateway

As fig 2 shows, you can create an API by clicking on the Create API button.

You will see a new tab opening as fig 3 shows. Fill the name and add a description and Create API.

Figure 3: Create New API

You will be directed to a window as fig 4 when Clicked Create API .

Figure 4: Resources Window

To upload an image, we need to do a POST REST Call. So, let’s create a new resource with a POST Method.

Go to Actions and select Create Resource to create a new REST Call with a new resource.

Figure 5: Create Resource

I will be using upload-to-s3 as my resource path in this example as fig 6 shows.

Figure 6: Create new resource

When you press the Create Resource button, you can see a new resource added to the resource path as shown in fig 7.

Figure 7: New Resource Added

Now we need to create a new method under the resource we created. As shown in Fig 8, Go to Actions and select Create Method .

Figure 8: Create Method

Select POST from the drop-down list and press the correct button to your right as in fig 9.

Figure 9: Select POST

You will see a window as shown in fig 10 below. Make sure you tick the Use Lambda Proxy Integration and use correct Lambda region and Lambda Function.

Figure 10: Configure POST Method

Now you got to do some more work meanwhile here. That is creating a new Lambda function for our POST Method. Let’s see how we are going to do that.

Creating a new Lambda Function

Login to AWS Lambda Console and you will see a button as Create Functionas shown in fig 11.

Figure 11: Create Function

You will see a window as shown in fig 12 which will let you configure your Lambda function. If you configure your Lambda function correctly, you can create the function by pressing Create Function .

Now, this is what you have to use the Lambda Function in fig 10.

Figure 12: Create a Lambda function

Now there is some more work here, that is creating a new AWS IAM role. Let’s see how we are going to do that.

Creating a new AWS IAM Role

Go to AWS Management Console and you can find IAM under Security, Identity & Compliance section. Under Roles, you can create a new role (shown in fig 13) which gives permission to access above 3 AWS services.

Figure 13: Create new IAM Role

Choose the Lambda as the AWS Service and Click Next Permissions as shown in fig 14.

Figure 14: Select Trust

Choose the AmazonS3FullAccess and CloudWatchLogsFullAccess policy and press Next Review as shown in fig 15.

Figure 15: IAM Policy

Next, add a Role name and create the Role as shown in fig 16.

Figure 16: Summary of the new role

Now, this is the role that you have to use under Existing role as shown in fig 12.

Deploying the API in API Gateway

Now we have to deploy our API, to do that Go to Actions and select Deploy API as shown in fig 17. You will have to create a new stage for the deployment as shown in fig 18. This will make you deploy your API to do REST Calls from your Application.

Figure 17: Deploy API
Figure 18: Create new Deployment Stage

When you navigate to Stages you can see a stage as named as prod and inside that, you can find the method call and related invoke URL as shown in fig 19.

Figure 19: Invoke URL for POST REST Call

Then, go to the lambda we have created before and add API Gateway as a trigger as shown in fig 20.

Figure 20: Adding API gateway Trigger

Then configure it providing the API Gateway name we created above as shown in fig 21.

Figure 21: Configuring API Gateway

Now you have to create a new S3 bucket to upload your images. Go to S3 Console and Click Create Bucket as shown in fig 22.

Creating an S3 Bucket

Figure 22: Create new S3 bucket

Then configure the bucket proving necessary information as shown in fig 23.

Figure 23: Create new S3 bucket

I know it’s quite hectic to go through all these configurations. But trust me, it’s worth it at the end.

Now as the configuration work is done, we can move on to get your hands dirty with Lambda.

Lambda Script for Uploading an Image to S3

In my API Call, I send a query param as username and a request body with an image as a Base64 encoded String.

In lambda what I do is, I extract this query param and the request body.

I use the query param to name the image when uploading to S3.

Since the request body is a Base64 string, we have to decode it back and upload to S3.

Following is my script for uploading the image to S3 bucket.

const AWS = require('aws-sdk');
//*/ get reference to S3 client 
var s3 = new AWS.S3();

exports.handler = (event, context, callback) => {

     let encodedImage =JSON.parse(event.body).user_avatar;
     let decodedImage = Buffer.from(encodedImage, 'base64');
     var filePath = "avatars/" + event.queryStringParameters.username + ".jpg"
     var params = {
       "Body": decodedImage,
       "Bucket": "find-my-mate-hasangi",
       "Key": filePath  
    };
    s3.upload(params, function(err, data){
       if(err) {
           callback(err, null);
       } else {
           let response = {
        "statusCode": 200,
        "headers": {
            "my_header": "my_value"
        },
        "body": JSON.stringify(data),
        "isBase64Encoded": false
    };
           callback(null, response);
    }
    });
    
};

It is essential to export the aws-sdk and get a reference to the S3 client.

params are how you define the S3 bucket.

Body is what you are going to upload to S3 bucket.

Bucket is the name of the S3 bucket you need to upload the image to.

Key is the location in the bucket with the file name you want to upload to.

You need to include the following code to prevent 502 Malformed Lambda proxy response error if you are using Lambda proxy integration in your API

let response = {
        "statusCode": 200,
        "headers": {
            "my_header": "my_value"
        },
        "body": JSON.stringify(data),
        "isBase64Encoded": false
    };

Make sure that you save the Lambda :).

Now the work is done to upload an image to S3.

Let’s focus on retrieving the uploaded image

Retrieving the Uploaded Image from S3

For this first, we have to create a new Get Method in API Gateway.

As we did before for the POST Call, we can create a new resource and create a new method directing to a new lambda function with the same IAM Role as shown in fig 24.

Figure 24: Create new Lambda function for GET API Call

API REST Call will look like below in fig 25.

Figure 25: GET Method Configurations

Newly created Lambda function should be configured as we did before by adding a trigger to the API Gateway.

In the GET REST Call, I send a query param which specifies the key of the S3 bucket.

Following is my Lambda Script for Retrieving the image from S3.

const AWS = require('aws-sdk');
//*/ get reference to S3 client 
var s3 = new AWS.S3();

exports.handler = (event, context, callback) => {
    var params = {
  "Bucket": "find-my-mate-hasangi",
  "Key": event.queryStringParameters.key  
    };
    s3.getObject(params, function(err, data){
       if(err) {
           callback(err, null);
       } else {
           let response = {
        "statusCode": 200,
        "headers": {
            "my_header": "my_value"
        },
        "body": JSON.stringify(data),
        "isBase64Encoded": false
    };
           callback(null, response);
    }
    });
    
};

I invoke the GET Call as a JSON request from the backend of my Application and what I get in the response body is a JSONArray which represents the image.

Tadah!!

It’s done. Hope you had fun with AWS Services!!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s