GistTree.Com
Entertainment at it's peak. The news is by your side.

A GitHub-Action to create a Google cloud dataproc cluster for test use-cases

0

A Github Stir to achieve a dataproc cluster in your test expend-cases.

Utilization

# Setup gcloud CLI
- makes expend of: GoogleCloudPlatform/github-actions/setup-gcloud@master
  with:
    version: '314.0.0'
    service_account_key: ${{ secrets and ways.service_account_key }}
    service_account_email: ${{ secrets and ways.service_account_email }}
    project_id: ${{ secrets and ways.project_id }}
    export_default_credentials: marvelous
- flee: gcloud files

- name: Acquire Google Cloud Dataproc cluster
  makes expend of: vishnudxb/gcloud-dataproc-introduction@main
  with:
    cluster: "your dataproc cluster name"
    utter: "Your dataproc luster utter"
    project: "Your gcloud project id"
    initialization-actions: "gs:///"
    master-boot-disk-dimension: "500GB"
    master-boot-disk-form: "pd-traditional"
    master-machine-form: "n1-traditional-2"
    max-idle: "1h"
    num-masters: 1
    worker-boot-disk-dimension: "500GB"
    worker-boot-disk-form: "pd-traditional"
    worker-machine-form: "n1-traditional-2"
    zone: "us-east1-b"
    picture-version: "preview-ubuntu18"
    metadata: "PIP_PACKAGES=$(sed -e '/^#/d' -e 's/#.*$//' -e 's/ *$//g' requirements.txt | tr 'n' ' ')"
    scopes: "https://www.googleapis.com/auth/cloud-platform"
    num-workers: 2
    properties: "spark:spark.scheduler.mode=FIFO"
    labels: "provider=internet-app,env=dev"

The inputs description:

inputs:
  cluster:
    description: 'Enter your gcloud dataproc cluster name'
    required: marvelous
  utter:
    description: 'Enter your gcloud dataproc cluster utter'
    required: unfounded
    default: us-east1
  project:
    description: 'Enter your gcloud project id'
    required: marvelous
  initialization_actions:
    description: 'Executables or scripts that could flee on all nodes in your cluster right this moment after the cluster is determined up'
    required: marvelous
  master-boot-disk-dimension:
    description: 'Dataproc Master Boot disk dimension'
    required: unfounded
    default: 500GB
  master-boot-disk-form:
    description: 'Dataproc Master Boot disk form'
    required: unfounded
    default: pd-traditional
  master-machine-form:
    description: 'Dataproc Master Machine form'
    required: unfounded
    default: n1-traditional-2
  max-idle:
    description: 'The length earlier than cluster is auto-deleted after final job completes, similar to "2h" or "1d".'
    required: unfounded
    default: '1h'
  num-masters:
    description: 'The choice of master nodes in the cluster.'
    required: unfounded
    default: 1
  worker-boot-disk-dimension:
    description: 'Dataproc Worker Boot disk dimension'
    required: unfounded
    default: 500GB
  worker-boot-disk-form:
    description: 'Dataproc Worker Boot disk form'
    required: unfounded
    default: pd-traditional
  worker-machine-form:
    description: 'Dataproc Worker Machine form'
    required: unfounded
    default: n1-traditional-2
  zone:
    description: 'Dataproc Cluster Zone'
    required: unfounded
    default: us-east1-b
  picture-version:
    description: 'The picture version to make expend of for the cluster.'
    required: unfounded
    default: preview-ubuntu18
  metadata:
    description: 'Metadata to be made available to the visitor working system running on the cases'
    required: marvelous
  scopes:
    description: 'Specifies scopes for the node cases.'
    required: unfounded
    default: https://www.googleapis.com/auth/cloud-platform
  num-workers:
    description: 'The choice of worker nodes in the cluster.'
    required: unfounded
    default: 2
  properties:
    description: 'Specifies configuration properties for set in programs, similar to Hadoop and Spark.'
    required: marvelous
  labels:
    description: 'Checklist of ticket KEY=VALUE pairs so as to add.'
    required: unfounded

After your trying out that it is seemingly you’ll well delete the dataproc cluster you created by running the below motion.

- name: Delete Google Cloud Dataproc cluster
  makes expend of: vishnudxb/gcloud-dataproc-deletion@main
  with:
    cluster: "your dataproc cluster name"
    utter: "Your dataproc luster utter"
    project: "Your gcloud project id"

Read More

Leave A Reply

Your email address will not be published.