Automating Cloud Operations with Python: AWS, Azure, and GCP

Alright, folks! We’ve come a long way in mastering Python for DevOps, and this blog marks the finale of this series—well, for now! But don’t worry, the journey doesn’t stop here. As we dive deeper into core DevOps topics, we’ll revisit Python whenever we work on real-world projects.

This blog is all about understanding how Python plays a crucial role in Cloud Automation across AWS, Azure, and GCP. No hands-on this time—especially if you're new to the cloud, no stress! We’ve got dedicated cloud blogs coming up where we’ll break things down step by step.

For now, just focus on grasping how Python simplifies cloud operations, and trust me, when we implement these concepts later, everything will fall into place seamlessly!

Introduction to Cloud SDKs

What Are Cloud SDKs?

Software Development Kits (SDKs) provide tools and libraries to interact with cloud services programmatically.

Why Use Python for Cloud Automation?

Cross-Platform Support: Write once, deploy anywhere.
Rich Ecosystem: SDKs like boto3 (AWS), azure-identity (Azure), and google-cloud (GCP).
Scalability: Automate tasks across hundreds of servers or services.

Setting Up Python for Cloud Automation

Installing Required SDKs

Before automating cloud tasks, install the required Python libraries:

bashCopyEditpip install boto3  # AWS SDK
pip install azure-mgmt-compute azure-identity  # Azure SDK
pip install google-cloud-storage google-auth  # GCP SDK

You will also need:
🔹 AWS credentials (~/.aws/credentials or IAM role)
🔹 Azure authentication (az login)
🔹 GCP authentication (gcloud auth application-default login)

Automating AWS Operations with Python (Boto3)

What is Boto3?

Boto3 is the AWS SDK for Python, used for managing AWS services like EC2, S3, and Lambda.

Example 1: Creating an EC2 Instance Using Python

Use Case: Automating EC2 Instance Provisioning

import boto3

ec2 = boto3.resource("ec2")

instance = ec2.create_instances(
    ImageId="ami-12345678",  # Replace with a valid AMI ID
    MinCount=1,
    MaxCount=1,
    InstanceType="t2.micro",
    KeyName="my-key",
)

print(f"Instance created with ID: {instance[0].id}")

Automates the creation of EC2 instances for Dev/Test environments.

Example 2: Uploading Files to S3 for Backups

Use Case: Automating Backup to S3

import boto3

s3 = boto3.client("s3")

def upload_to_s3(file_name, bucket):
    s3.upload_file(file_name, bucket, file_name)
    print(f"Uploaded {file_name} to {bucket}")

upload_to_s3("backup.tar.gz", "my-backup-bucket")

Automates scheduled backups of critical files to S3.

Automating Azure Operations with Python

What is Azure SDK?

Azure SDK for Python allows the automation of cloud operations like managing VMs, storage, and networking.

Example 1: Creating an Azure Virtual Machine

Use Case: Automating VM Deployment in Azure

from azure.identity import DefaultAzureCredential
from azure.mgmt.compute import ComputeManagementClient

subscription_id = "your-subscription-id"
credential = DefaultAzureCredential()
compute_client = ComputeManagementClient(credential, subscription_id)

vm_params = {
    "location": "eastus",
    "hardware_profile": {"vm_size": "Standard_B1ls"},
    "storage_profile": {
        "image_reference": {"publisher": "Canonical", "offer": "UbuntuServer", "sku": "18.04-LTS", "version": "latest"},
    },
    "os_profile": {
        "computer_name": "devops-vm",
        "admin_username": "azureuser",
        "admin_password": "YourSecurePassword123!",
    },
    "network_profile": {
        "network_interfaces": [{"id": "/subscriptions/.../networkInterfaces/myNic"}]
    },
}

vm = compute_client.virtual_machines.begin_create_or_update("myResourceGroup", "devops-vm", vm_params)
print(f"VM {vm.result().name} created successfully!")

Automates VM provisioning in Azure for CI/CD pipelines.

Example 2: Automating Azure Blob Storage Uploads

Use Case: Storing Logs in Azure Blob Storage

from azure.storage.blob import BlobServiceClient

connection_string = "your-azure-storage-connection-string"
blob_service_client = BlobServiceClient.from_connection_string(connection_string)

container_name = "logs"
blob_name = "app-log.txt"

blob_client = blob_service_client.get_blob_client(container=container_name, blob=blob_name)

with open("app-log.txt", "rb") as data:
    blob_client.upload_blob(data)
print("Log file uploaded to Azure Blob Storage.")

Automates log storage in Azure Blob for better observability.

Automating GCP Operations with Python

What is Google Cloud SDK for Python?

It enables the automation of GCP services like Compute Engine, Cloud Storage, and BigQuery.

Example 1: Creating a Virtual Machine in GCP

Use Case: Automating GCP Compute Engine VM Creation

from google.cloud import compute_v1

def create_instance(project_id, zone, instance_name):
    instance_client = compute_v1.InstancesClient()

    config = compute_v1.Instance(
        name=instance_name,
        machine_type=f"zones/{zone}/machineTypes/e2-micro",
        disks=[
            compute_v1.AttachedDisk(
                boot=True,
                initialize_params=compute_v1.AttachedDiskInitializeParams(
                    source_image="projects/debian-cloud/global/images/family/debian-10",
                ),
            )
        ],
        network_interfaces=[compute_v1.NetworkInterface(network="global/networks/default")],
    )

    operation = instance_client.insert(project=project_id, zone=zone, instance_resource=config)
    operation.result()

    print(f"Created instance {instance_name} in GCP.")

create_instance("my-gcp-project", "us-central1-a", "devops-instance")

Automates VM creation in GCP for testing environments.

Example 2: Uploading Files to Google Cloud Storage

Use Case: Storing DevOps Reports in GCP

from google.cloud import storage

def upload_to_gcp(bucket_name, file_path):
    client = storage.Client()
    bucket = client.bucket(bucket_name)
    blob = bucket.blob(file_path)

    blob.upload_from_filename(file_path)
    print(f"Uploaded {file_path} to {bucket_name}.")

upload_to_gcp("my-devops-bucket", "deployment-report.txt")

Automates the storage of deployment reports in Google Cloud Storage.

Best Practices for Multi-Cloud Automation

Use Environment Variables for Credentials – Avoid hardcoding secrets.
Implement IAM Permissions Properly – Ensure least privilege access.
Use Terraform/Ansible with Python – Combine infrastructure as code with automation scripts.
Monitor and Log Cloud Operations – Store logs in centralized storage for debugging.
Write Modular and Reusable Scripts – Avoid redundancy by creating generic functions for cloud tasks.

When to Use Which Service?

Task	AWS	Azure	GCP
Compute	EC2	Virtual Machines	Compute Engine
Storage	S3	Blob Storage	Cloud Storage
Serverless	Lambda	Functions	Cloud Functions

Hurray! Give yourself a round of applause! 👏👏 Yes, do it—you absolutely deserve it! Why? Because you’ve made it through an incredible journey mastering Python for DevOps!

Let’s take a moment to recap this wild ride:
We kicked things off by understanding the Role of Programming in DevOps and why Python is the ultimate partner-in-crime for automation. From there, we set up our Python environment, got hands-on with Fundamentals, dived deep into Data Types, Control Flow, Functions, Modules & Packages, Libraries, tackled File Handling, and finally explored Automating Cloud Operations. Phew! That’s a solid DevOps foundation right there.

So, tell me—how are you feeling? Good? Great? Or completely overwhelmed? Either way, I hope you had as much fun learning as I had writing this series!

Now that we’ve wrapped up the coding part, it’s time to level up with something equally exciting—Version Control Systems! In the next blog, we’ll dive into tracking, storing, and managing your code efficiently—because, let’s be real, great code deserves great management!

See you soon, fellow DevOps Engineers, with a brand-new topic and even more fun!

Until next time, keep coding, automating, and advancing in DevOps! 😁

Peace out ✌️