GistTree.Com
Entertainment at it's peak. The news is by your side.

Accessing IBM Cloud Object Storage Using Go Cloud

0

In this put up I will blow their own horns you the finest approach to access IBM Cloud Object Storage the usage of Go Cloud.

Background

When photographing the usage of RAW files, you will desire a number of storage means to save all these images. Many photographers are the usage of a NAS at residence or some more or much less on-line storage SaaS esteem Google Power or Dropbox Expert. Nonetheless as a developer with files on clouds and moreover working for a firm working its dangle cloud, that is impartial too uncomplicated. Therefore, I’m developing my dangle storage system per cloud skills.

What I’m currently doing is uploading the total files at this time to an IBM Cloud Object Storage. Why am I the usage of IBM Cloud for this? Attributable to in my comparison of diversified Cloud Storage’s it had the most fine label-earnings ratio for me (but compare for your self). Concurrently I’m working on a administration platform for synchronization and managing all these images. I’m planning to document from my efforts with some uncomplicated examples about how I’m doing things.

That is the indispensable put up about this mission displaying how I work in conjunction with my IBM Cloud Object Storage bucket the usage of Go Cloud.

What’s Go Cloud?

Google introduced on Google Cloud Subsequent 2018 or no longer it is ‘Portable Cloud Programming’ toolkit Go Cloud Construction Kit (CDK) (aka Go Cloud). It is cause supporting groups in multi-cloud and hybrid-cloud architectures.

Most continuously or no longer it is a situation of libraries abstracting the platform dependent ingredients and offering a generic and originate cloud API. This model when I carry out my application with this library, it requires finest a number of adjustments to alternate the cloud supplier. So I form no longer occupy a dealer lock-in and may per chance presumably perhaps take the cloud supplier with the most fine offering.

Why is it working with IBM COS?

IBM Cloud Object Storage has a S3-effectively matched API, that’s why it’ll utilize the S3-implementation of Go Cloud. The particular element you may per chance presumably perhaps impartial must strategies is to connect to a explicit endpoint.

Checklist all objects on storage bucket

In this put up I will blow their own horns you the finest approach to access IBM Cloud Object Storage the usage of Go Cloud. Let’s originate!

Necessities

  • IBM Cloud Story: Within the event you will want one assemble a free memoir right here
  • Place in Go instrument chain: I’m currently the usage of model 1.15.1

Setting up IBM Cloud Object Storage Bucket

  1. Login to your IBM Cloud Story and search within the catalog for Object Storage. Produce an instance and likewise you’re going to be forwarded to the Getting started page of your Cloud Storage instance.
  2. Subsequent now we must assemble a Storage Bucket. Click on on Produce bucket and utilize the predefined Instant Start up bucket. Subsequent you may per chance presumably perhaps impartial must enter a explicit bucket name. On the next page you may per chance presumably perhaps impartial occupy a probability to add some files. I counsel uploading two or three tiny files, so you’re going to be ready to gaze something when executing this system later.
  3. Exit the wizard the usage of the Request bucket configuration button. On this goal or no longer it is indispensable to display some particulars:
    • Bucket name
    • Remark
    • Public endpoint (or no longer it is extra down on the page, e.g. s3.european-de.cloud-object-storage.appdomain.cloud)

View Configuration properties

  1. What are we lacking now? Upright, the credentials. IBM has auto-generated a carrier memoir for you, but we need HMAC credentials to access the bucket with Go Cloud. Navigate to the Carrier Credentials tab on the left and click on Unique Credential. Favor Role Reader, originate the Developed choices and suggested the Include HMAC Credentials probability. Form this by clicking on Add.

The created credentials will appear within the list and when opened it reveals a JSON yarn. Extract the next three properties:

  • apikey
  • cos_hmac_keys.access_key_id
  • cos_hmac_keys.secret_access_key

HMAC Service credentials

Setting up this system

Let’s originate the usage of Go Cloud. Produce a modern Go mission named amble-cloud-ibm-cos and add Go Cloud as dependency:

mkdir ~/amble/src/github.com/schoenenberg/amble-cloud-ibm-cos
# $_ is a bash shortcut for the usage of the final argument of the previous command
cd $_
# Initialize Go modules
amble mod init
# Add Go Cloud as dependency
amble catch -u gocloud.dev
# For the s3 constructor we also need the AWS SDK
amble catch -u github.com/aws/aws-sdk-amble/aws

Subsequent we assemble a equipment bucketop the put we assemble all these operations on a bucket. We separate them in a single other equipment in remark to reuse them in a while. Let’s originate with the ListObjects goal, which shall return all objects within the bucket:

// file: ./pkg/bucketop/list.amble

equipment bucketop

import (
	"context"
	"io"

	"gocloud.dev/blob"
)

// ListObjects lists all objects saved within the offered bucket
func ListObjects(
	ctx context.Context,
	bkt *blob.Bucket,
) (*[]blob.ListObject, error) {
	// Init consequence array of Objects
	objects := carry out([]blob.ListObject, 0)
	// Produce a list quiz on the bucket
	iter := bkt.List(nil)
	// Iterate by way of every element of the iterator
	for {
		obj, err := iter.Subsequent(ctx)
		if err == io.EOF {
			damage
		} else if err != nil {
			return nil, err
		}
		objects = append(objects, *obj)
	}

	return &objects, nil
}

The chilly element about this goal is we lovely provide a *blob.Bucket as an argument and this already summary the underlying Object Storage skills, so we are in a position to utilize this goal also with e.g. Google Cloud Storage and Azure Blob Storage.

Within the extinguish, we are developing a straightforward application to list the total files in a bucket the usage of our ListObjects goal.

// file: ./bin/list_objects/indispensable.amble

equipment indispensable

import (
	"context"
	"fmt"
	"log"

	"github.com/aws/aws-sdk-amble/aws"
	"github.com/aws/aws-sdk-amble/aws/credentials"
	"github.com/aws/aws-sdk-amble/aws/session"
	"gocloud.dev/blob/s3blob"

	"github.com/schoenenberg/amble-cloud-ibm-cos/pkg/bucketop"
)

// Enter your credentials right here!
const (
	endpoint      = ""
	goal        = ""
	bucketName    = ""
	apiKey        = ""
	secret        = ""
	keyId         = ""
)

func indispensable() {
	ctx := context.Background()

	// Produce a session with s3 constructor
	sess, err := session.NewSession(&aws.Config{
		Problem:      aws.String(goal),
		Endpoint:    aws.String(endpoint),
		Credentials: credentials.NewStaticCredentials(
			keyId,
			secret,
			apiKey,
		),
	})
	if err != nil {
		log.Fatalln(err)
	}

	// Start the bucket
	bucket, err := s3blob.OpenBucket(
		ctx,
		sess,
		bucketName,
		nil,
	)
	if err != nil {
		log.Fatalln(err)
	}
	defer bucket.Discontinuance()

	// Spend our ListObjects goal
	objs, err := bucketop.ListObjects(ctx, bucket)
	if err != nil {
		log.Fatalln(err)
	}

	// And print all objects with its measurement
	for _, obj := differ *objs {
		fmt.Printf("%s - Size %.2f MBn",
			obj.Key,
			amble with the waft64(obj.Size)/1000000.0,
		)
	}
}

What we are in total doing is growing a session, opening the bucket and executing our ListObjects goal. Afterwards, we print the consequence. To form this case flee amble flee ./bin/list_objects for your command line:

amble flee ./bin/list_objects
# example.txt - Size 0.08 MB
# hello_world.txt - Size 0.22 MB

That is it! Within the event you don’t wish to reproduction&paste the code from this put up you’re going to be ready to also clone or fork my repo from Github.

Summary

We developed a tiny application the usage of Google’s Go Cloud Construction Kit to access an IBM Cloud Storage bucket and listing the total files in it.

References

Disclaimer: The postings on this goal are my dangle and do not basically symbolize my employer IBM’s positions, strategies or opinions.

Read More

Leave A Reply

Your email address will not be published.