Get Objects from AWS S3 Bucket with GoLang

Hello Gophers,

In this blog, we are going to see how to list down all the objects from an AWS S3 Bucket with AWS GO SDK v2. The AWS SDK for Go provides APIs and utilities that developers can use to build Go applications that use AWS services.

Video tutorial for this blog:

Get your AWS Access Keys

1. Open the IAM console.
2. On the navigation menu, choose Users.
3. Choose your IAM user name (not the check box).
4. Open the Security credentials tab, and then choose to Create access key.
5. To see the new access key, choose Show. Your credentials resemble the following: Access key ID: AKIAIOSFODNN7EXAMPLE Secret access key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
6. To download the key pair, choose the Download .csv file. Store the keys in a secure location.

Config Setup

1. Your AWS config and credentials will be stored under ~/.aws/config and ~/.aws/credentials. Those are the default values for your AWS SDK Environment. If you want you can edit them.

2. Run the below command in the terminal to let the Go Program know that it should pick up the AWS config from the ~/.aws directory

$ export AWS_SDK_LOAD_CONFIG=true

Setup

• First create a bucket in the AWS S3 Console as in the below pictorial representation. Here I am creating a bucket with the name infytech2

• Make sure you allow public access while creating the bucket

• You can go with all the default options selected and click on Create Bucket Button.

Upload an object

• Now upload any of the files or folders in your system into the bucket by clicking on the Upload Button as shown below.

Execution

• To get started initialize your local project by running the following Go command.

$ go mod init aws_s3

• The following commands show how to retrieve the standard set of SDK modules to use in your application.

$ go get github.com/aws/aws-sdk-go-v2
$ go get github.com/aws/aws-sdk-go-v2/config

This will retrieve the core SDK module, and the config module which is used for loading the AWS shared configuration.

• Next you can install one or more AWS service API clients required by your application. All API clients are located under github.com/aws/aws-sdk-go-v2/service . In this example, we retrieve the Amazon S3 API client.

$ go get github.com/aws/aws-sdk-go-v2/service/s3

• Now in main.go file setup the import statements

package main

import (
    "context"
    "log"
    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/s3"
)

• Now in your main function

• Load the AWS Config

cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithSharedConfigProfile("VasanthK"))

• Initialise a s3 Client

client := s3.NewFromConfig(cfg)

• Call ListObjects with the help of the client and bucket name specified which returns the list of objects from a particular bucket

    output, err := client.ListObjectsV2(context.TODO(), &s3.ListObjectsV2Input{
        Bucket: aws.String("infytech1"),
    })

• Finally print out the objects from the bucket into the console with Object name and its Size

    for _, object := range output.Contents {
        log.Printf("key=%s size=%d", aws.ToString(object.Key), object.Size)
    }

Output

All Objects in infytech2 bucket
key=thumbnail.png size=149021

Entire Code File

package main

import (
    "context"
    "log"

    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/s3"
)

func main() {

    cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithSharedConfigProfile("VasanthK"))

    if err != nil {
        log.Fatal(err)
    }

    client := s3.NewFromConfig(cfg)

    output, err := client.ListObjectsV2(context.TODO(), &s3.ListObjectsV2Input{
        Bucket: aws.String("infytech2"),
    })

    if err != nil {
        log.Fatal(err)
    }
    log.Println("All Objects in infytech2 bucket")

    for _, object := range output.Contents {
        log.Printf("key=%s size=%d", aws.ToString(object.Key), object.Size)
    }
}

Finally thanks for reading Subscribe %[youtube.com/c/INFYTECH]