Getting Started with Google Cloud Platform: A Beginner’s Guide to Cloud Excellence

📘 Chapter 5: Data Analytics and Machine Learning Tools in GCP

🔍 Overview

Google Cloud Platform (GCP) offers powerful, scalable tools for data analytics and machine learning (ML). Whether you're building dashboards, crunching massive datasets, or training neural networks, GCP provides services that are serverless, cost-effective, and tightly integrated with Google’s ecosystem.

In this chapter, we’ll cover:

• Core services for data analytics: BigQuery, Cloud Storage, Dataflow, Pub/Sub
• ML tools: Vertex AI, AutoML, TPUs, and AI APIs
• Real-world examples and use cases
• Code snippets and visual summaries

📊 1. BigQuery: Serverless Data Warehouse

🔹 What is BigQuery?

BigQuery is a fully managed, serverless, highly scalable data warehouse designed for fast SQL analytics on large datasets.

🔹 Features

• ANSI SQL support
• Real-time streaming inserts
• Automatic scaling and partitioning
• Integration with Looker Studio, Sheets, Cloud ML

🔹 Example: Create a Dataset

sql

-- Create dataset

CREATE SCHEMA my_dataset;

🔹 Example: Run SQL Query

sql

SELECT name, COUNT(*) as total_sales

FROM `project.dataset.sales`

GROUP BY name

ORDER BY total_sales DESC;

🗃️ 2. Cloud Storage: Data Lake Foundation

Used for storing structured, semi-structured, and unstructured data. Serves as the data lake layer in modern architectures.

🔹 Use Case:

• Store raw CSV, JSON, Parquet files
• Stream logs or ETL inputs for processing

🔹 Example: Upload File

bash

gsutil cp localfile.csv gs://my-bucket/dataset/

🔄 3. Dataflow: Real-time and Batch Processing (Apache Beam)

Dataflow is a serverless streaming and batch data processing service built on Apache Beam.

🔹 Features

• Unified batch and stream pipelines
• Auto-scaling and load balancing
• Support for Python and Java SDKs

🔹 Example Use Case:

• Cleanse and transform streaming sensor data into BigQuery

🔹 Example Python Snippet:

python

with beam.Pipeline() as pipeline:

    rows = (

        pipeline

        | 'Read CSV' >> beam.io.ReadFromText('gs://my-bucket/data.csv')

        | 'Parse' >> beam.Map(lambda row: row.split(','))

        | 'Write to BQ' >> beam.io.WriteToBigQuery(

            'my_project:my_dataset.my_table',

            schema='field1:STRING, field2:INTEGER',

            write_disposition=beam.io.BigQueryDisposition.WRITE_APPEND)

    )

📬 4. Pub/Sub: Messaging for Analytics Pipelines

Pub/Sub is a messaging middleware used to ingest events/data into analytics systems like Dataflow, BigQuery, and Dataproc.

🔹 Use Case:

• Ingest IoT data
• Capture app logs
• Real-time dashboard updates

🔹 Example:

bash

# Create topic

gcloud pubsub topics create sales-stream

# Publish a message

gcloud pubsub topics publish sales-stream --message="sale:1234"

🧠 5. Vertex AI: Unified ML Platform

Vertex AI is the all-in-one solution for training, tuning, deploying, and monitoring ML models on GCP.

🔹 Features:

• Jupyter-based Workbench
• Support for AutoML + custom models
• Model registry + explainable AI
• Scalable GPU/TPU training

🔹 Example: Train a Model with AutoML

bash

gcloud beta ai custom-jobs create \

  --region=us-central1 \

  --display-name="my_model_training" \

  --worker-pool-spec=machine-type=n1-standard-4,replica-count=1,container-image-uri=gcr.io/cloud-aiplatform/training/tf-cpu.2-3:latest

🔹 Vertex AI Workbench

Interactive JupyterLab instances with BigQuery, GitHub, TensorFlow, and other integrations pre-installed.

Common libraries:

python

import pandas as pd

from google.cloud import bigquery

🧠 6. BigQuery ML: Machine Learning with SQL

Use SQL to train ML models directly inside BigQuery—no code required.

🔹 Example: Train Linear Regression Model

sql

CREATE OR REPLACE MODEL `my_dataset.sales_model`

OPTIONS(model_type='linear_reg') AS

SELECT

  feature_1,

  feature_2,

  label

FROM

  `my_dataset.sales_data`;

🔹 Predict:

sql

SELECT

  feature_1,

  predicted_label

FROM

  ML.PREDICT(MODEL `my_dataset.sales_model`,

             (SELECT feature_1, feature_2 FROM `my_dataset.new_data`));

🧪 7. AI APIs (Pre-trained ML Models)

GCP offers ready-to-use APIs for computer vision, natural language, translation, and speech.

🔹 Example: Python Call to Vision API

python

from google.cloud import vision

client = vision.ImageAnnotatorClient()

image = vision.Image()

image.source.image_uri = 'gs://my-bucket/cat.jpg'

response = client.label_detection(image=image)

for label in response.label_annotations:

    print(label.description)

📈 8. Looker Studio + BigQuery for Dashboards

• Connect Looker Studio to BigQuery datasets
• Build interactive charts with real-time analytics
• Share insights with teams and clients securely

📋 Summary Table – GCP Analytics & ML Stack