Description

Scala is doubtless one of the most in-demand skills for data scientists and data engineers. This competitive course will teach you the essential concepts and methodologies of Scala with a lot of practical implementations.

The course Scala from Beginner to Pro is refreshingly different. The well-thought-out quizzes and mini projects that cover all the important aspects will make your Scala learning journey that much easier. This course includes an overview of Hadoop and Spark with a hands-on project with Scala Spark. Right through the course, every theoretical explanation is followed by practical implementation. This course is designed to reflect the most in-demand Scala skills that you will start using right away at the workplace. The course is simple to follow and understand. Expressive, exhaustive, and practical with live coding sessions and replete with quizzes. Rich with state-of-the-art and up-to-date knowledge of this field. The 6 mini-projects and one Scala Spark project included in this course are a vital component of this course. These projects present you with a hands-on opportunity to experiment for yourself with trial and error. You get a chance to learn from the mistakes you make. Importantly, it's easy to understand the potential gaps that might exist between theory and practice. Code bundles are available here: https://github.com/PacktPublishing/Scala-Spark-Master-Big-Data-with-Scala-and-Spark

What You Will Learn

Explore variables in data types in Scala
Explore functions and their usage in Scala
Learn data structures; namely, lists, ListBuffer, maps, sets, and stack
Understand the flow controls in Scala and different ways to control the flow
Explore the difference between Spark RDDs and Spark DFs
ETL pipeline from AWS S3 to AWS RDS using Spark

Audience

This course is for absolute beginners who want to learn Spark and Scala along with its implementation in realistic projects; people who want to develop smart solutions; data scientists; and machine learning experts.

Approach

This course follows a hand-on learning approach with exercises at the end of each section that will enable you to grasp the concepts by thinking about the right solution on your own and implementing it. It provides an overall interactive learning experience.

Key Features

Relate the practical concepts of Scala and Spark with real-world problems * Implement any project from scratch that requires Scala and Spark knowledge * Know the theoretical and practical aspects of Scala and Spark

Github Repo

https://github.com/PacktPublishing/Scala-Spark-Master-Big-Data-with-Scala-and-Spark/tree/main/03-Codes/Codes/06-Project

About the Author

AI Sciences

AI Sciences are experts, PhDs, and artificial intelligence practitioners, including computer science, machine learning, and Statistics. Some work in big companies such as Amazon, Google, Facebook, Microsoft, KPMG, BCG, and IBM. AI sciences produce a series of courses dedicated to beginners and newcomers on techniques and methods of machine learning, statistics, artificial intelligence, and data science. They aim to help those who wish to understand techniques more easily and start with less theory and less extended reading. Today, they publish more comprehensive courses on specific topics for wider audiences. Their courses have successfully helped more than 100,000 students master AI and data science.

Course Outline

1. Introduction

1. Why Scala

This video provides an overview of Scala and its scope.

2. Scala Applications

Explore the applications of Scala in this session.

3. About the Instructor

Let's get introduced to the instructor.

4. Introduction to Course

This video provides an overview of the entire course.

5. Projects Overview

This video provides an overview of all the projects.

2. Overview

1. What is Scala

Learn all about Scala in this video session.

2. Scala Setup (Local Machine)

Learn the steps to install and set up Scala in your local machine.

3. Hello World

Let's look at a sample Hello World code.

4. Scala Setup (Online)

Learn how to set up and run Scala code online.

5. Variables in Scala

In this session, you will learn all about variables in Scala.

6. Arithmetic Operations on Variables-1

In this lesson, we will explore arithmetic operations on variables.

7. Arithmetic Operations on Variables-2

Let's look at a few basic operations on strings.

8. Quiz (Arithmetic Operations)

Let's solve a short quiz for quick revision.

9. Solution (Arithmetic Operations)

In this lesson, we're going to discuss the solution to the problem we saw in our previous session.

10. Quiz (Strings)

Let's solve a short quiz for quick revision.

11. Solution (Strings)

In this lesson, we're going to discuss the solution to the problem we saw in our previous session.

12. Type Casting

This session covers the concept of type casting in details.

13. Taking Input from User

In this lesson, we will learn how to take input from the user.

14. Quiz (User Input and Type Casting)

Let's look at a quiz question in this video.

15. Solution (User Input and Type Casting)

Let's discuss the solution to the quiz question from our previous session.

3. Flow Control

1. Overview of Control Statements

This video provides an overview of the entire section.

2. If else statements

In this lesson, you will learn about If-else statements.

3. Conditions in If

Let's dive deeper into If-else statements and look at a few examples in this session.

4. Quiz (if statement)

It's knowledge check time! Let's look at the short quiz.

5. Solution (if statement)

Let's discuss the solution to the quiz question we saw in our previous lesson.

6. Nested If Else

In this lesson, we will learn about nested if else statements.

7. Quiz (Nested If Else)

It's knowledge check time! Let's look at a short quiz on nested if else.

8. Solution (nested if else)

In this video session, we will discuss the solution for the quiz we saw in our previous session.

9. Logical Operators

In this video session, we will learn about the different types of logical operators.

10. Quiz (Logical Operators)

Let's look at a problem statement in this short quiz.

11. Solution (Logical Operators)

Let's discuss the solution to the quiz we saw in the previous lesson.

12. If Else If

In this lesson, we will learn about another variation of the if-else statements, which is if-else-if statements in Scala.

13. Quiz (If Else If)

It's time for a short quiz on if-else-if statements.

14. Solution (If Else If)

Let's discuss the solution to the quiz in this session.

15. Overview of Loops

This video provides an overview of loops. Learn what loops are and the types.

16. Overview of While Loop

Learn what are while loops in this session.

17. While Loop

Let's explore while loops in detail in this video.

18. Quiz (While Loop)

It's time for a short quiz on while loops.

19. Solution 1 (While Loop)

Let's discuss the solution to the quiz on while loop (part 1).

20. Solution 2 (While Loop)

In this session, we will continue discussing the solution to the quiz (part 2).

21. Do While Loop

In this lesson, you will learn about do while loops.

22. For Loop

In this lesson, you will learn about for loops.

23. Quiz (For Loop)

It's time for a quiz on for loop.

24. Solution (For Loop)

Let's discuss the solution to the quiz on for loop in this session.

25. Quiz(For Loop)

Here is another problem question on for loop.

26. Solution(For Loop)

Let's look at the solution of the problem statement we discussed in the previous session.

27. Break

In this lesson, we will explore the Break statement.

28. Break Fix

In this video we will discuss Break fix.

29. Project Overview

This video provides an overview of the mini project that we will be building in the next few sessions.

30. Project Solution Design

In this session, we will design a solution for our game mini project.

31. Project Solution Code 1

Let's start coding the first part of the project.

32. Project Solution Code 2

Let's start coding for the second part of the mini-project.

33. Project Solution Code 3

Let's continue coding for our project. We will go ahead and code the third part in this session.

34. Project Solution Code 4

Let's complete the coding for the game-project and wrap up this section.

4. Functions

1. Overview of Functions

This video introduces you to functions.

2. Writing Addition Function

In this session, you will learn how to write the addition function.

3. Quiz (Basic Functions)

It's time for a quiz on basic functions.

4. Solution (Basic Functions)

Let's discuss the solution of the quiz on basic functions in this session.

5. Functions Common Issues

Learn the basic errors and common issues that you might encounter when using functions.

6. Named Arguments

This session covers functions with named arguments.

7. Quiz (String Concatenation Function)

It's time for a quiz on string concatenation functions.

8. Solution (String Concatenation Function)

Let's discuss the solution to the quiz on string concatenation.

9. Quiz (Dividing Code in Functions)

Let's look at another quiz on dividing code in functions.

10. Solution (Dividing Code in Functions)

Let's discuss the solution to the quiz on dividing code in functions.

11. Default Arguments

In this session, you will learn about default arguments in Scala.

12. Quiz (Default Arguments)

It's time for a quiz on default arguments.

13. Solution (Default Arguments)

Let's discuss the solution to the quiz on default arguments.

14. Anonymous Functions

In this lesson, you will learn all about the anonymous functions of Scala.

15. Quiz (Anonymous Functions)

It's time for a short quiz on anonymous functions.

16. Solution (Anonymous Functions)

Let's discuss the solution to the short quiz on anonymous functions in this session.

17. Scopes

In this lesson, you will learn all about Scopes.

18. Project Overview

In this video, you will get an overview of the mini-project. For this project, we will be writing a program for an ATM machine.

19. Checking Credentials

Let's start working on the mini-project. In this session, we will code for checking the credentials.

20. Prompting the Menu

In this session, we will work on prompting the menu for the mini-project.

21. Basic Functions

In this session, we will work on the global variables and the basic functions required for the project.

22. Breaking Code in More Functions

In this lesson, we will incorporate the Break statement.

23. Final Run

Let's wrap up our project in this session.

5. Classes

1. Introduction to Classes

This video introduces you to classes and various concepts related to classes.

2. Creating Class

In this lesson, you will learn how to create a class in Scala.

3. Class Constructor

In this lesson, you will learn about constructors in Scala.

4. Functions and Classes

Let's look at functions and classes in this session.

5. Project Overview

This session provides an overview of the next mini-project. Let's start working on our project and write a program for basic operations.

6. Basic Structure

In this session, we will work on creating the basic structure for our project.

7. Final Run

Let's wrap up our project in this session.

6. Data Structures

1. Introduction of Data Structures

This video introduces you to data structures in Scala.

2. Lists introduction

In this session, we will explore list data structures.

3. Lists Create and Delete Elements

Learn how to create, append and delete elements in a list in this session. Lists are immutable in Scala.

4. Lists Take

In this lesson, you will learn how to take from lists in Scala.

5. ListBuffer Introduction

In this lesson, we will discuss ListBuffer. A ListBuffer is a collection that contains mutable data.

6. Add Data in ListBuffer

In this session, you will learn how to add data in ListBuffer.

7. Remove Data from ListBuffer

In this lesson, you will learn how to remove data in ListBuffer.

8. Take Data from ListBuffer

Learn how to take elements from the ListBuffer in this session.

9. Project Overview

It's time to apply whatever we have learned so far and build a mini-project.

10. Project Architecture Discussion

Before we move on to the coding part, let's first discuss the project architecture in this session.

11. Project Architecture Implementation

We have already discussed the project architecture in our previous session. We will now go ahead and implement the project architecture.

12. User Input for Objects

In this session, you will learn how to get user input for objects.

13. Implementing the Control Flow

In this lesson, we will implement the control flow in the project.

14. Creating Required Functions Inside Class

In this session, we will create the required function inside the class and wrap up the project.

15. Overview of Maps

In this session, you will learn about maps in detail.

16. Creating Maps

In this lesson, you will learn how to create maps in Scala.

17. Check Key in Map

In this lesson, we will discuss how to check whether a key is present inside a map or not.

18. Update Value in Map

Learn how to update the key value pair in this lesson.

19. Add and Remove Items from Maps

Learn how to add and remove/delete items from maps.

20. Iterating in Maps

In this session, you will learn how to iterate in a map.

21. Project Overview

It's time for a mini-project. We are going to write a word count program.

22. Project Architecture

Let's discuss the project architecture in the session before we start the implementation process.

23. Project Structure Code

In this session, we will start creating the basic project structure.

24. Using Maps for Word Count

In this lesson, we will work on the code for getting the word count using maps.

25. Final Run

In this lesson, we will do a test run before wrapping up the project.

26. Sets Overview

Learn all about sets in this session.

27. Add and Remove Item from the Set

In this session, we will learn how to add items and remove items from sets.

28. Set Operations

In this session, we will discuss set operations, Union(), Intersection(), and Difference().

29. Overview of Stack

Learn about Stack's data structure in this pleasant.

30. Push and Pop in Stack

Let's discuss about two important functions in Stack, which are Push() and Pop().

31. Stack Attributes

In this lesson, we will explore some more functions related to Stack.

32. Project Overview

In this lesson, you will get an overview of the next mini-project we will be working on.

33. Project Architecture

Let's discuss the basic structure of the project.

34. Extra Closing Bracket Use Case

In this lesson, we will start working on the code.

35. Extra Starting Bracket Use Case

Let's wrap up the mini-project in this session.

7. Project

1. Project Introduction

In this session, you will get introduced to Scala Spark and get an overview of the entire project work.

2. Why Spark

In this session, let's explore a little bit more about Spark and why we use Spark.

3. Hadoop Ecosystem

In this lesson, we will explore the Hadoop ecosystem.

4. Spark Architecture

Learn about the Spark architecture in details in this session.

5. Spark Ecosystem

In this lesson, we will learn about the Spark core ecosystem.

6. DataBricks Account

In this lesson, let's explore Databricks and how to run our code in it.

7. Setting Up DataBricks Cluster

In this video lesson, you will learn how to set up Databricks cluster.

8. Spark Local Setup

Learn how to set up Spark in your local system in this session.

9. Spark Hadoop Setup

In this session, we will continue setting up Spark.

10. Spark RDDs

In this lesson, we will explore Spark RDDs in detail.

11. Spark RDDs (textFile, collect)

In this lesson, we will work with Spark RDDS.

12. Spark Local Run

In this lesson, we will replicate the same process we did in Databricks in our local machines.

13. Understanding Map

In this lesson, you will learn about Map functions in details.

14. Understanding Flat Map

In this session, we will discuss flat maps.

15. Understanding Reduce by Key

In this session, you will learn about reduce by key.

16. Word Count Example

In this lesson, we will look at an example on word count.

17. Spark DFs

In this lesson, we will learn about Spark Dataframes.

18. Spark DF Read Data

In this lesson, we will read the data in a dataframe.

19. Spark Print Schema, Select

In this video session we will learn about Spark Print Schema.

20. Spark Group By

Learn the grouping mechanism in Spark in this session.

21. Spark DF Write

In this session, you will learn how to write in a Spark Dataframe.

22. Creating S3 Bucket

In this session, you will learn how to create S3 buckets in AWS.

23. Creating Database in RDS

In this lesson, you will learn how to create a database in RDS.

24. Performing ETL

In this session, you will write a Scala Spark job inside AWS glue that will migrate the data from S3 to RDS.

Course Images

Scala & Spark-Master Big Data with Scala and Spark

By Packt

Booking options

Highlights