End-To-End Data Engineering Project. Made to learn some common data engineering practices.
A reusable solution to kill billing on any projects associated with a given budget and budget alert. If your budget is exceeded, the resulting budget alert will cause all projects associated with that alert to be detached from the billing account. This immediately prevents all paid resources in the project from being consumed.
Build a data pipeline on Google Cloud using an event-driven architecture, leveraging GCS, Cloud Run functions, and BigQuery. Explore both VM and Composer options for Airflow management, and utilize Logging & Monitoring for pipeline health. Discover how SQL-based BigQuery ML can be used for initial ML implementation in specific scenarios.
A Google Cloud ELT pipeline project with dbt, Cloud Run, Composer, BigQuery, and Dataplex. Includes event-driven automation, data quality checks, and complete setup guidance.
Real-time streaming fraud detection architecture on Google Cloud using Pub/Sub, Cloud Run Functions, BigQuery, and Terraform with interactive simulation dashboards.
Serverless loader: copy remote files directly into Google Cloud Storage by a fully automated workflow using Terraform and Cloud Run Functions.
A serverless, event-driven data engineering pipeline on Google Cloud (Medallion Architecture) for real-time Transport for NSW transit analytics.
PySpark-based ETL pipeline leveraging Dataproc, Cloud Storage, Cloud Run Functions and BigQuery, to automate Spotify "New Releases" data processing and visualization in Looker Studio.
A real-time analytics dashboard implemented using a distributed, event-driven architecture on Google Cloud Platform (Cloud Run, Pub/Sub, Cloud Functions, BigQuery) and MongoDB Atlas
Los Mirlos Storytelling, a magical serverless backend built with Google Cloud Functions and powered by Gemini (Google's LLM)
Develop and bundle your Cloud Run functions with ease
Cloud Run functionsとActiveReports for .NETでつくる帳票生成API
A fully automated ETL pipeline that fetches and stores real-time traffic and weather data in BigQuery, with a live Looker dashboard for visualization.
Open-source starter kit for building an Agentic Platform where users can create, deploy, and chat with custom AI agents powered by their own tools.
Go microservices and serverless functions for GCP. Deploys to Cloud Run and Cloud Functions. Includes Makefile for automation. IAM setup and curl test included.
Official LINE account for notifying garbage collection days
Serverless ETL pipeline on GCP using Cloud Run, Pub/Sub, BigQuery & Looker for real-time NYC Taxi ML insights.
This repo hosts a data-engineering pipeline that utilizes google cloud-storage, google pub/sub, google cloud function, airflow, ngrok, open-ai to facilitate a trigger based ELT pipeline. It also manages some vectorized data and geometric data following standard data retrieval and storage practices.
Cloud Run service that exposes an HTTP endpoint to accept JSON payloads representing sales transaction. The service validates, transforms, and loads the data into a BigQuery table.
To associate your repository with the cloud-run-functions topic, visit your repo's landing page and select "manage topics."