Booking options
Price:
BuyContact Us£52.99
Course Images
Snowflake - Build and Architect Data Pipelines Using AWS
By Packt
🔥 Limited Time Offer 🔥
Get a 10% discount on your first order when you use this promo code at checkout: MAY24BAN3X
- 30 Day Money Back Guarantee
- Completion Certificate
- 24/7 Technical Support
Highlights
On-Demand course
8 hours 39 minutes
All levels
Description
The course helps you learn Snowflake from scratch and explore a few of its important features. You will build automated pipelines with Snowflake and use the AWS cloud with Snowflake as a data warehouse. You will also explore Snowpark to be worked on the data pipelines.Snowflake is the next big thing, and it is becoming a full-blown data ecosystem. With the level of scalability and efficiency in handling massive volumes of data and also with several new concepts in it, this is the right time to wrap your head around Snowflake and have it in your toolkit. This course not only covers the core features of Snowflake but also teaches you how to deploy Python/PySpark jobs in AWS Glue and Airflow that communicate with Snowflake, which is one of the most important aspects of building pipelines. In this course, you will look at Snowflake, and then the most crucial aspects of Snowflake in an efficient manner. You will be writing Python/Spark Jobs in AWS Glue Jobs for data transformation and seeing real-time streaming using Kafka and Snowflake. You will be interacting with external functions and use cases, and see the security features in Snowflake. Finally, you will look at Snowpark and explore how it can be used for data pipelines and data science. By the end of this course, you will have learned about Snowflake and Snowpark, and learned how to build and architect data pipelines using AWS. You need to have an active AWS account in order to perform the sections related to Python and PySpark. For the rest of the course, a free trial Snowflake account should suffice.
What You Will Learn
Learn about Snowflake and its basics before getting started with labs
Check the crucial aspects of Snowflake in a very practical manner
Write Python/Spark jobs in AWS Glue Jobs for data transformation
Execute real-time streaming using Kafka and Snowflake
Interact with external functions and use cases
Learn and explore the Snowpark library
Audience
This course is ideal for software engineers, aspiring data engineers or data analysts, and data scientists who want to excel in their careers in the IT domain. Apart from them, this course is also good for programmers and database administrators with experience in writing SQL queries.
Prior programming experience in SQL or at least some prior knowledge in writing queries and Python is a must. You should have a basic experience or understanding of cloud services such as AWS along with an active AWS account.
Approach
This course is comprehensive, easy to understand, and designed for intermediate to advance with a basic understanding of programming and SQL knowledge.
You can read the reference links and the official documentation of Snowflake as much as possible, which are attached to the resource links. There are lab exercises and practical content demonstrated after providing theoretical information; therefore, a well-balanced course.
Key Features
Learn from an easy-to-understand and step-by-step course, divided into 85+ videos along with detailed resource files * Integrate real-time streaming data and data orchestration with Airflow and Snowflake * Highly practical explanations and lab exercises to help you grasp the most out of the course
Github Repo
https://github.com/PacktPublishing/Snowflake---Build-and-Architect-Data-Pipelines-using-AWS
About the Author
Siddharth Raghunath
Siddharth Raghunath is a business-oriented engineering manager with a vast experience in the field of software development, distributed processing, and cloud data engineering. He has worked on different cloud platforms such as AWS and GCP as well as on-premise Hadoop clusters. He conducts seminars on distributed processing using Spark, real-time streaming and analytics, and best practices for ETL and data governance. He is passionate about coding and building optimal data pipelines for robust data processing and streaming solutions.
Course Outline
1. Introduction to the Course
1. Course Roadmap This video provides the course's roadmap. |
2. Prerequisites and How to Succeed in This Course This video talks about the prerequisites and how to succeed in this course. |
2. Introduction to Snowflake and AWS
1. What Is Data-Warehouse? This video talks about the Data-Warehouse. |
2. Two Aspects of a Data Ecosystem This video demonstrates the two aspects of a data ecosystem. |
3. Lab - Set Up Snowflake Trial Account This is a lab video on setting up a Snowflake trial account. |
4. Snowflake Architecture This video explains the Snowflake architecture. |
5. Snowflake Object Hierarchy This video explains about the Snowflake object hierarchy. |
6. Snowflake - Virtual Warehouses This video explains about virtual warehouses in Snowflake. |
7. Snowflake - Different Billing Components This video demonstrates about the different billing components in Snowflake. |
8. Snowflake - Track Your Consumption This video explains how to track your consumption in Snowflake. |
9. Snowflake- Resource Monitors This video explains about resource monitors in Snowflake. |
3. Snowflake - Tables
1. Introduction - Different Tables in Snowflake This video provides an introduction to different tables in Snowflake. |
2. Lab - Create Tables in Snowflake This is a lab video on how to create tables in Snowflake. |
3. Snowflake - Views, Materialized Views and Secure Views This video explains views, materialized views, and secure views in Snowflake. |
4. Lab - Create Views in Snowflake This is a lab video on how to create views in Snowflake. |
5. Lab - Create Secure Views in Snowflake This is a lab video that explains how to create secure views in Snowflake. |
6. More about Views in Snowflake This video explores more about views in Snowflake. |
4. Snowflake - Partitioning, Clustering, and Performance Optimization
1. Section Overview This video provides an overview of the section. |
2. Introduction to Partitions and clustering keys This video demonstrates an introduction to partitions and clustering keys. |
3. Lab - Micro-Partitions and Clustering Keys This video explains micro-partitions and clustering keys. |
4. Benefits of Micro-Partitions and Clustering This video explains the benefits of micro-partitions and clustering. |
5. Understanding Clustering Depth and Cluster Overlap This video helps you in understanding clustering depth and cluster overlap. |
6. Lab - Selecting Your Clustering Keys This is a lab video explains on selecting your clustering keys. |
7. Lab - Check Query Profile and History This is a lab video on checking the query profile and history. |
8. Lab - Query Processing and Caching This is a lab video that explains query processing and caching. |
9. Search Optimization Feature This video explains the search optimization feature. |
5. Snowflake - Data Loading/Ingestion and Extraction
1. Section Overview This video provides an overview to the section. |
2. Data Ingestion - Real-World Use Cases This video shows data ingestion with the help of real world use cases. |
3. Lab - Create an Integration Object to Connect Snowflake with AWS S3 This is a lab video that helps create an integration object to connect Snowflake with AWS S3. |
4. Lab - Ingest CSV from S3 to Snowflake This is a lab video to ingest CSV from S3 to Snowflake. |
5. Lab - Ingest JSON from S3 to Snowflake This is a lab video to ingest JSON from S3 to Snowflake. |
6. Introduction to Continuous Data Ingestion in Snowflake This video provides an introduction to continuous data ingestion in Snowflake. |
7. Lab - Create and Implement Snow Pipe This is a lab video that demonstrates the creation and implementation of Snow pipe. |
8. Snow pipe - Billing Estimation and Key Considerations for Data Ingestion This video explains the billing estimation and key considerations for data ingestion in Snow pipe. |
9. Lab - Extracting/Unload Data from Snowflake to S3 This is a lab video that demonstrates extracting/unloading data from Snowflake to S3. |
6. Snowflake - Tasks and Query Scheduling
1. Section Overview This video provides an overview of the section. |
2. Introduction to Tasks This video provides an introduction to tasks. |
3. Lab - Create Standalone and Dependent Tree of Tasks This is a lab video that helps create standalone and dependent tree of tasks. |
4. Lab - Billing and Query History for Tasks This s a lab video that demonstrates the billing and query history for tasks. |
7. Snowflake - Streams and Change Data Capture
1. Section Overview This video provides an overview of the section. |
2. Introduction to Streams This video provides an introduction to streams. |
3. Lab - Implement Standard Streams This is a lab video that helps implement standard streams. |
4. Lab - Implement Append-Only Streams This is a lab video to implement append-only streams. |
5. Lab - Streams in a Transaction This is a lab video that explains the streams in a transaction. |
6. Streams - Data Retention and Staleness This video explains about the data retention and staleness in streams. |
7. Lab - Change Tracking Using "Changes" This is a lab video to help you change tracking using "Changes". |
8. Project Overview This video demonstrates the project overview. |
9. Lab - Create Streams - Project Solution This is a lab video that demonstrates how to create streams - project solution. |
10. Lab - Create Streams - Continuation This lab video is a continuation to the previous video on creating streams. |
11. Lab - End-to-End Pipeline in Action This is a lab video that talks about the end-to-end pipeline in action. |
8. Snowflake - User-Defined Functions
1. Introduction to User-Defined Functions and UDF Types This video provides an Introduction to user-defined functions and UDF types. |
2. Lab - Write and Implement a Scalar UDF This is a lab video to write and implement a Scalar UDF. |
3. Lab - Write Tabular UDF in SQL This is a lab video to write Tabular UDF in SQL. |
4. Lab - Implement JavaScript UDFs This is a lab video helps implement JavaScript UDFs. |
5. What Is Pushdown in UDF? This video explains Pushdown in UDF. |
6. Lab - How Can Pushdown Expose the Underlying Data? This is a lab video that explains how pushdown can expose the underlying data. |
7. Lab - Write Secure UDFs This is a lab video to write secure UDFs. |
9. Snowflake - External Functions
1. Section Overview This video provides an overview of the section. |
2. Introduction to External Functions This video provides an introduction to external functions. |
3. Lab - Write Deploy AWS Lambda Function This is a lab video that demonstrates how to write deploy AWS Lambda function. |
4. Create IAM Role This video demonstrates how to create IAM role. |
5. Lab - Create API Gateway This is a lab video to help you create API Gateway. |
6. Lab - Secure and Deploy API Gateway This is a lab video that demonstrates securing and deploying API gateway. |
7. Lab - Create External Function in Snowflake This is a lab video to create an external function in Snowflake. |
10. Snowflake with Python, Spark, and Airflow on AWS
1. Section Overview This video provides an overview of the section. |
2. Lab - Connect Python with Snowflake in Your Local Machine This is a lab video that helps you connect Python with Snowflake in your local machine. |
3. Introduction to AWS Glue This video provides an introduction to AWS Glue. |
4. Lab - Deploy and Execute Python Script to AWS Glue This is a lab video that helps deploy and execute Python script to AWS Glue. |
5. Lab - Parameterize Your Python Script on AWS Glue This is a lab video that helps parameterize your Python script on AWS Glue. |
6. Lab - Python Pandas with Snowflake on AWS Glue This is a lab video on Python Pandas with Snowflake on AWS Glue. |
7. What Is Pushdown in Spark 3.1? This video explains about Pushdown in Spark 3.1. |
8. Lab - Deploy a PySpark Script Using AWS Glue This is a lab video to deploy a PySpark script using AWS Glue. |
9. Lab - Set Up Managed Airflow Cluster on AWS This is a lab video that helps set up a managed airflow cluster on AWS. |
10. Lab - Configure Snowflake Connectivity in Airflow This is a lab video that explains how to configure Snowflake connectivity in airflow. |
11. Lab - Deploy a PySpark Transformation job in AWS Glue This is a lab video that helps deploy a PySpark transformation job in AWS Glue. |
12. Lab - Set Up Airflow DAG This is a lab video to set up airflow DAG. |
11. Real-Time Streaming with Kafka and Snowflake
1. Section Overview This video provides an overview of the section. |
2. Lab - Download the Necessary JAR Files This is a lab video to download the necessary JAR files. |
3. Lab - Set Up Kafka in your local system This is a lab video to set up Kafka in your local system. |
4. Lab - Set Up Kafka Snowflake Connector This is a lab video to set up the Kafka Snowflake Connector. |
5. Lab - Set Up Encryption Keys for Kafka-Snowflake Connectivity This is a lab video that helps you set up encryption keys for Kafka-Snowflake connectivity. |
6. Lab - Streaming Data in Action This is a lab video on streaming data in action. |
12. Snowflake - Data Protection and Governance
1. Section Overview This video provides an overview of the section. |
2. What Is Time Travel and Failsafe in Snowflake? This video explains about Time Travel and Failsafe in Snowflake. |
3. Lab - Time Travel and Data Recovery This is a lab video that talks about Time Travel and data recovery. |
4. Lab - Column Level Dynamic Data Masking This is a lab video that demonstrates about column level dynamic data masking. |
5. What Is Row Level Security? This video explains about row level security. |
6. Lab - Create and Implement Row Level Access Policy This is a lab video to create and implement row level access policy. |
13. Wrap Up and More Learning
1. More Updates and What's Next This video talks about what's next after this course and more updates. |
Course Content
- Snowflake - Build and Architect Data Pipelines Using AWS
About The Provider
Packt
Birmingham
Founded in 2004 in Birmingham, UK, Packt’s mission is to help the world put software to work in new ways, through the delivery of effective learning and i...
Reviews
There are no reviews for Snowflake - Build and Architect Data Pipelines Using AWS yet. Why not be the first one?