32 Data Engineering courses delivered Live Online

Duration 4 Days 24 CPD hours This course is intended for The primary audience for this course is data professionals, data architects, and business intelligence professionals who want to learn about data engineering and building analytical solutions using data platform technologies that exist on Microsoft Azure. The secondary audience for this course includes data analysts and data scientists who work with analytical solutions built on Microsoft Azure. In this course, the student will learn how to implement and manage data engineering workloads on Microsoft Azure, using Azure services such as Azure Synapse Analytics, Azure Data Lake Storage Gen2, Azure Stream Analytics, Azure Databricks, and others. The course focuses on common data engineering tasks such as orchestrating data transfer and transformation pipelines, working with data files in a data lake, creating and loading relational data warehouses, capturing and aggregating streams of real-time data, and tracking data assets and lineage. Prerequisites Successful students start this course with knowledge of cloud computing and core data concepts and professional experience with data solutions. AZ-900T00 Microsoft Azure Fundamentals DP-900T00 Microsoft Azure Data Fundamentals 1 - INTRODUCTION TO DATA ENGINEERING ON AZURE * What is data engineering * Important data engineering concepts * Data engineering in Microsoft Azure 2 - INTRODUCTION TO AZURE DATA LAKE STORAGE GEN2 * Understand Azure Data Lake Storage Gen2 * Enable Azure Data Lake Storage Gen2 in Azure Storage * Compare Azure Data Lake Store to Azure Blob storage * Understand the stages for processing big data * Use Azure Data Lake Storage Gen2 in data analytics workloads 3 - INTRODUCTION TO AZURE SYNAPSE ANALYTICS * What is Azure Synapse Analytics * How Azure Synapse Analytics works * When to use Azure Synapse Analytics 4 - USE AZURE SYNAPSE SERVERLESS SQL POOL TO QUERY FILES IN A DATA LAKE * Understand Azure Synapse serverless SQL pool capabilities and use cases * Query files using a serverless SQL pool * Create external database objects 5 - USE AZURE SYNAPSE SERVERLESS SQL POOLS TO TRANSFORM DATA IN A DATA LAKE * Transform data files with the CREATE EXTERNAL TABLE AS SELECT statement * Encapsulate data transformations in a stored procedure * Include a data transformation stored procedure in a pipeline 6 - CREATE A LAKE DATABASE IN AZURE SYNAPSE ANALYTICS * Understand lake database concepts * Explore database templates * Create a lake database * Use a lake database 7 - ANALYZE DATA WITH APACHE SPARK IN AZURE SYNAPSE ANALYTICS * Get to know Apache Spark * Use Spark in Azure Synapse Analytics * Analyze data with Spark * Visualize data with Spark 8 - TRANSFORM DATA WITH SPARK IN AZURE SYNAPSE ANALYTICS * Modify and save dataframes * Partition data files * Transform data with SQL 9 - USE DELTA LAKE IN AZURE SYNAPSE ANALYTICS * Understand Delta Lake * Create Delta Lake tables * Create catalog tables * Use Delta Lake with streaming data * Use Delta Lake in a SQL pool 10 - ANALYZE DATA IN A RELATIONAL DATA WAREHOUSE * Design a data warehouse schema * Create data warehouse tables * Load data warehouse tables * Query a data warehouse 11 - LOAD DATA INTO A RELATIONAL DATA WAREHOUSE * Load staging tables * Load dimension tables * Load time dimension tables * Load slowly changing dimensions * Load fact tables * Perform post load optimization 12 - BUILD A DATA PIPELINE IN AZURE SYNAPSE ANALYTICS * Understand pipelines in Azure Synapse Analytics * Create a pipeline in Azure Synapse Studio * Define data flows * Run a pipeline 13 - USE SPARK NOTEBOOKS IN AN AZURE SYNAPSE PIPELINE * Understand Synapse Notebooks and Pipelines * Use a Synapse notebook activity in a pipeline * Use parameters in a notebook 14 - PLAN HYBRID TRANSACTIONAL AND ANALYTICAL PROCESSING USING AZURE SYNAPSE ANALYTICS * Understand hybrid transactional and analytical processing patterns * Describe Azure Synapse Link 15 - IMPLEMENT AZURE SYNAPSE LINK WITH AZURE COSMOS DB * Enable Cosmos DB account to use Azure Synapse Link * Create an analytical store enabled container * Create a linked service for Cosmos DB * Query Cosmos DB data with Spark * Query Cosmos DB with Synapse SQL 16 - IMPLEMENT AZURE SYNAPSE LINK FOR SQL * What is Azure Synapse Link for SQL? * Configure Azure Synapse Link for Azure SQL Database * Configure Azure Synapse Link for SQL Server 2022 17 - GET STARTED WITH AZURE STREAM ANALYTICS * Understand data streams * Understand event processing * Understand window functions 18 - INGEST STREAMING DATA USING AZURE STREAM ANALYTICS AND AZURE SYNAPSE ANALYTICS * Stream ingestion scenarios * Configure inputs and outputs * Define a query to select, filter, and aggregate data * Run a job to ingest data 19 - VISUALIZE REAL-TIME DATA WITH AZURE STREAM ANALYTICS AND POWER BI * Use a Power BI output in Azure Stream Analytics * Create a query for real-time visualization * Create real-time data visualizations in Power BI 20 - INTRODUCTION TO MICROSOFT PURVIEW * What is Microsoft Purview? * How Microsoft Purview works * When to use Microsoft Purview 21 - INTEGRATE MICROSOFT PURVIEW AND AZURE SYNAPSE ANALYTICS * Catalog Azure Synapse Analytics data assets in Microsoft Purview * Connect Microsoft Purview to an Azure Synapse Analytics workspace * Search a Purview catalog in Synapse Studio * Track data lineage in pipelines 22 - EXPLORE AZURE DATABRICKS * Get started with Azure Databricks * Identify Azure Databricks workloads * Understand key concepts 23 - USE APACHE SPARK IN AZURE DATABRICKS * Get to know Spark * Create a Spark cluster * Use Spark in notebooks * Use Spark to work with data files * Visualize data 24 - RUN AZURE DATABRICKS NOTEBOOKS WITH AZURE DATA FACTORY * Understand Azure Databricks notebooks and pipelines * Create a linked service for Azure Databricks * Use a Notebook activity in a pipeline * Use parameters in a notebook ADDITIONAL COURSE DETAILS: Nexus Humans DP-203T00 Data Engineering on Microsoft Azure training program is a workshop that presents an invigorating mix of sessions, lessons, and masterclasses meticulously crafted to propel your learning expedition forward. This immersive bootcamp-style experience boasts interactive lectures, hands-on labs, and collaborative hackathons, all strategically designed to fortify fundamental concepts. Guided by seasoned coaches, each session offers priceless insights and practical skills crucial for honing your expertise. Whether you're stepping into the realm of professional skills or a seasoned professional, this comprehensive course ensures you're equipped with the knowledge and prowess necessary for success. While we feel this is the best course for the DP-203T00 Data Engineering on Microsoft Azure course and one of our Top 10 we encourage you to read the course outline to make sure it is the right content for you. Additionally, private sessions, closed classes or dedicated events are available both live online and at our training centres in Dublin and London, as well as at your offices anywhere in the UK, Ireland or across EMEA.

Duration 5 Days 30 CPD hours Overview Mining data Manipulating data Visualizing and reporting data Applying basic statistical methods Analyzing complex datasets while adhering to governance and quality standards throughout the entire data life cycle CompTIA Data+ is an early-career data analytics certification for professionals tasked with developing and promoting data-driven business decision-making. CompTIA Data+ gives you the confidence to bring data analysis to life. As the importance for data analytics grows, more job roles are required to set context and better communicate vital business intelligence. Collecting, analyzing, and reporting on data can drive priorities and lead business decision-making. 1 - IDENTIFYING BASIC CONCEPTS OF DATA SCHEMAS * Identify Relational and Non-Relational Databases * Understand the Way We Use Tables, Primary Keys, and Normalization 2 - UNDERSTANDING DIFFERENT DATA SYSTEMS * Describe Types of Data Processing and Storage Systems * Explain How Data Changes 3 - UNDERSTANDING TYPES AND CHARACTERISTICS OF DATA * Understand Types of Data * Break Down the Field Data Types 4 - COMPARING AND CONTRASTING DIFFERENT DATA STRUCTURES, FORMATS, AND MARKUP LANGUAGES * Differentiate between Structured Data and Unstructured Data * Recognize Different File Formats * Understand the Different Code Languages Used for Data 5 - EXPLAINING DATA INTEGRATION AND COLLECTION METHODS * Understand the Processes of Extracting, Transforming, and Loading Data * Explain API/Web Scraping and Other Collection Methods * Collect and Use Public and Publicly-Available Data * Use and Collect Survey Data 6 - IDENTIFYING COMMON REASONS FOR CLEANSING AND PROFILING DATA * Learn to Profile Data * Address Redundant, Duplicated, and Unnecessary Data * Work with Missing Value * Address Invalid Data * Convert Data to Meet Specifications 7 - EXECUTING DIFFERENT DATA MANIPULATION TECHNIQUES * Manipulate Field Data and Create Variables * Transpose and Append Data * Query Data 8 - EXPLAINING COMMON TECHNIQUES FOR DATA MANIPULATION AND OPTIMIZATION * Use Functions to Manipulate Data * Use Common Techniques for Query Optimization 9 - APPLYING DESCRIPTIVE STATISTICAL METHODS * Use Measures of Central Tendency * Use Measures of Dispersion * Use Frequency and Percentages 10 - DESCRIBING KEY ANALYSIS TECHNIQUES * Get Started with Analysis * Recognize Types of Analysis 11 - UNDERSTANDING THE USE OF DIFFERENT STATISTICAL METHODS * Understand the Importance of Statistical Tests * Break Down the Hypothesis Test * Understand Tests and Methods to Determine Relationships Between Variables 12 - USING THE APPROPRIATE TYPE OF VISUALIZATION * Use Basic Visuals * Build Advanced Visuals * Build Maps with Geographical Data * Use Visuals to Tell a Story 13 - EXPRESSING BUSINESS REQUIREMENTS IN A REPORT FORMAT * Consider Audience Needs When Developing a Report * Describe Data Source Considerations For Reporting * Describe Considerations for Delivering Reports and Dashboards * Develop Reports or Dashboards * Understand Ways to Sort and Filter Data 14 - DESIGNING COMPONENTS FOR REPORTS AND DASHBOARDS * Design Elements for Reports and Dashboards * Utilize Standard Elements * Creating a Narrative and Other Written Elements * Understand Deployment Considerations 15 - UNDERSTAND DEPLOYMENT CONSIDERATIONS * Understand How Updates and Timing Affect Reporting * Differentiate Between Types of Reports 16 - SUMMARIZING THE IMPORTANCE OF DATA GOVERNANCE * Define Data Governance * Understand Access Requirements and Policies * Understand Security Requirements * Understand Entity Relationship Requirements 17 - APPLYING QUALITY CONTROL TO DATA * Describe Characteristics, Rules, and Metrics of Data Quality * Identify Reasons to Quality Check Data and Methods of Data Validation 18 - EXPLAINING MASTER DATA MANAGEMENT CONCEPTS * Explain the Basics of Master Data Management * Describe Master Data Management Processes ADDITIONAL COURSE DETAILS: Nexus Humans CompTIA Data Plus (DA0-001) training program is a workshop that presents an invigorating mix of sessions, lessons, and masterclasses meticulously crafted to propel your learning expedition forward. This immersive bootcamp-style experience boasts interactive lectures, hands-on labs, and collaborative hackathons, all strategically designed to fortify fundamental concepts. Guided by seasoned coaches, each session offers priceless insights and practical skills crucial for honing your expertise. Whether you're stepping into the realm of professional skills or a seasoned professional, this comprehensive course ensures you're equipped with the knowledge and prowess necessary for success. While we feel this is the best course for the CompTIA Data Plus (DA0-001) course and one of our Top 10 we encourage you to read the course outline to make sure it is the right content for you. Additionally, private sessions, closed classes or dedicated events are available both live online and at our training centres in Dublin and London, as well as at your offices anywhere in the UK, Ireland or across EMEA.

Duration 4 Days 24 CPD hours This course is intended for This class is intended for experienced developers who are responsible for managing big data transformations including: Extracting, loading, transforming, cleaning, and validating data. Designing pipelines and architectures for data processing. Creating and maintaining machine learning and statistical models. Querying datasets, visualizing query results and creating reports Overview Design and build data processing systems on Google Cloud Platform. Leverage unstructured data using Spark and ML APIs on Cloud Dataproc. Process batch and streaming data by implementing autoscaling data pipelines on Cloud Dataflow. Derive business insights from extremely large datasets using Google BigQuery. Train, evaluate and predict using machine learning models using TensorFlow and Cloud ML. Enable instant insights from streaming data Get hands-on experience with designing and building data processing systems on Google Cloud. This course uses lectures, demos, and hand-on labs to show you how to design data processing systems, build end-to-end data pipelines, analyze data, and implement machine learning. This course covers structured, unstructured, and streaming data. INTRODUCTION TO DATA ENGINEERING * Explore the role of a data engineer. * Analyze data engineering challenges. * Intro to BigQuery. * Data Lakes and Data Warehouses. * Demo: Federated Queries with BigQuery. * Transactional Databases vs Data Warehouses. * Website Demo: Finding PII in your dataset with DLP API. * Partner effectively with other data teams. * Manage data access and governance. * Build production-ready pipelines. * Review GCP customer case study. * Lab: Analyzing Data with BigQuery. BUILDING A DATA LAKE * Introduction to Data Lakes. * Data Storage and ETL options on GCP. * Building a Data Lake using Cloud Storage. * Optional Demo: Optimizing cost with Google Cloud Storage classes and Cloud Functions. * Securing Cloud Storage. * Storing All Sorts of Data Types. * Video Demo: Running federated queries on Parquet and ORC files in BigQuery. * Cloud SQL as a relational Data Lake. * Lab: Loading Taxi Data into Cloud SQL. BUILDING A DATA WAREHOUSE * The modern data warehouse. * Intro to BigQuery. * Demo: Query TB+ of data in seconds. * Getting Started. * Loading Data. * Video Demo: Querying Cloud SQL from BigQuery. * Lab: Loading Data into BigQuery. * Exploring Schemas. * Demo: Exploring BigQuery Public Datasets with SQL using INFORMATION_SCHEMA. * Schema Design. * Nested and Repeated Fields. * Demo: Nested and repeated fields in BigQuery. * Lab: Working with JSON and Array data in BigQuery. * Optimizing with Partitioning and Clustering. * Demo: Partitioned and Clustered Tables in BigQuery. * Preview: Transforming Batch and Streaming Data. INTRODUCTION TO BUILDING BATCH DATA PIPELINES * EL, ELT, ETL. * Quality considerations. * How to carry out operations in BigQuery. * Demo: ELT to improve data quality in BigQuery. * Shortcomings. * ETL to solve data quality issues. EXECUTING SPARK ON CLOUD DATAPROC * The Hadoop ecosystem. * Running Hadoop on Cloud Dataproc. * GCS instead of HDFS. * Optimizing Dataproc. * Lab: Running Apache Spark jobs on Cloud Dataproc. SERVERLESS DATA PROCESSING WITH CLOUD DATAFLOW * Cloud Dataflow. * Why customers value Dataflow. * Dataflow Pipelines. * Lab: A Simple Dataflow Pipeline (Python/Java). * Lab: MapReduce in Dataflow (Python/Java). * Lab: Side Inputs (Python/Java). * Dataflow Templates. * Dataflow SQL. MANAGE DATA PIPELINES WITH CLOUD DATA FUSION AND CLOUD COMPOSER * Building Batch Data Pipelines visually with Cloud Data Fusion. * Components. * UI Overview. * Building a Pipeline. * Exploring Data using Wrangler. * Lab: Building and executing a pipeline graph in Cloud Data Fusion. * Orchestrating work between GCP services with Cloud Composer. * Apache Airflow Environment. * DAGs and Operators. * Workflow Scheduling. * Optional Long Demo: Event-triggered Loading of data with Cloud Composer, Cloud Functions, Cloud Storage, and BigQuery. * Monitoring and Logging. * Lab: An Introduction to Cloud Composer. INTRODUCTION TO PROCESSING STREAMING DATA * Processing Streaming Data. SERVERLESS MESSAGING WITH CLOUD PUB/SUB * Cloud Pub/Sub. * Lab: Publish Streaming Data into Pub/Sub. CLOUD DATAFLOW STREAMING FEATURES * Cloud Dataflow Streaming Features. * Lab: Streaming Data Pipelines. HIGH-THROUGHPUT BIGQUERY AND BIGTABLE STREAMING FEATURES * BigQuery Streaming Features. * Lab: Streaming Analytics and Dashboards. * Cloud Bigtable. * Lab: Streaming Data Pipelines into Bigtable. ADVANCED BIGQUERY FUNCTIONALITY AND PERFORMANCE * Analytic Window Functions. * Using With Clauses. * GIS Functions. * Demo: Mapping Fastest Growing Zip Codes with BigQuery GeoViz. * Performance Considerations. * Lab: Optimizing your BigQuery Queries for Performance. * Optional Lab: Creating Date-Partitioned Tables in BigQuery. INTRODUCTION TO ANALYTICS AND AI * What is AI?. * From Ad-hoc Data Analysis to Data Driven Decisions. * Options for ML models on GCP. PREBUILT ML MODEL APIS FOR UNSTRUCTURED DATA * Unstructured Data is Hard. * ML APIs for Enriching Data. * Lab: Using the Natural Language API to Classify Unstructured Text. BIG DATA ANALYTICS WITH CLOUD AI PLATFORM NOTEBOOKS * What's a Notebook. * BigQuery Magic and Ties to Pandas. * Lab: BigQuery in Jupyter Labs on AI Platform. PRODUCTION ML PIPELINES WITH KUBEFLOW * Ways to do ML on GCP. * Kubeflow. * AI Hub. * Lab: Running AI models on Kubeflow. CUSTOM MODEL BUILDING WITH SQL IN BIGQUERY ML * BigQuery ML for Quick Model Building. * Demo: Train a model with BigQuery ML to predict NYC taxi fares. * Supported Models. * Lab Option 1: Predict Bike Trip Duration with a Regression Model in BQML. * Lab Option 2: Movie Recommendations in BigQuery ML. CUSTOM MODEL BUILDING WITH CLOUD AUTOML * Why Auto ML? * Auto ML Vision. * Auto ML NLP. * Auto ML Tables.

Duration 1 Days 6 CPD hours This course is intended for The primary audience for this course is data professionals who are familiar with data modeling, extraction, and analytics. It is designed for professionals who are interested in gaining knowledge about Lakehouse architecture, the Microsoft Fabric platform, and how to enable end-to-end analytics using these technologies. Job role: Data Analyst, Data Engineer, Data Scientist Overview Describe end-to-end analytics in Microsoft Fabric Describe core features and capabilities of lakehouses in Microsoft Fabric Create a lakehouse Ingest data into files and tables in a lakehouse Query lakehouse tables with SQL Configure Spark in a Microsoft Fabric workspace Identify suitable scenarios for Spark notebooks and Spark jobs Use Spark dataframes to analyze and transform data Use Spark SQL to query data in tables and views Visualize data in a Spark notebook Understand Delta Lake and delta tables in Microsoft Fabric Create and manage delta tables using Spark Use Spark to query and transform data in delta tables Use delta tables with Spark structured streaming Describe Dataflow (Gen2) capabilities in Microsoft Fabric Create Dataflow (Gen2) solutions to ingest and transform data Include a Dataflow (Gen2) in a pipeline This course is designed to build your foundational skills in data engineering on Microsoft Fabric, focusing on the Lakehouse concept. This course will explore the powerful capabilities of Apache Spark for distributed data processing and the essential techniques for efficient data management, versioning, and reliability by working with Delta Lake tables. This course will also explore data ingestion and orchestration using Dataflows Gen2 and Data Factory pipelines. This course includes a combination of lectures and hands-on exercises that will prepare you to work with lakehouses in Microsoft Fabric. INTRODUCTION TO END-TO-END ANALYTICS USING MICROSOFT FABRIC * Explore end-to-end analytics with Microsoft Fabric * Data teams and Microsoft Fabric * Enable and use Microsoft Fabric * Knowledge Check GET STARTED WITH LAKEHOUSES IN MICROSOFT FABRIC * Explore the Microsoft Fabric Lakehouse * Work with Microsoft Fabric Lakehouses * Exercise - Create and ingest data with a Microsoft Fabric Lakehouse USE APACHE SPARK IN MICROSOFT FABRIC * Prepare to use Apache Spark * Run Spark code * Work with data in a Spark dataframe * Work with data using Spark SQL * Visualize data in a Spark notebook * Exercise - Analyze data with Apache Spark WORK WITH DELTA LAKE TABLES IN MICROSOFT FABRIC * Understand Delta Lake * Create delta tables * Work with delta tables in Spark * Use delta tables with streaming data * Exercise - Use delta tables in Apache Spark INGEST DATA WITH DATAFLOWS GEN2 IN MICROSOFT FABRIC * Understand Dataflows (Gen2) in Microsoft Fabric * Explore Dataflows (Gen2) in Microsoft Fabric * Integrate Dataflows (Gen2) and Pipelines in Microsoft Fabric * Exercise - Create and use a Dataflow (Gen2) in Microsoft Fabric

Study online for the Master’s Advanced Motorsport Engineering and boost your motorsport career. With the ability to fit your studies around your existing career and family, the MSc is flexible and affordable. Start on any date and study when and where suits you!

Duration 4 Days 24 CPD hours This course is intended for This course is designed for data scientists with existing knowledge of Python and machine learning frameworks like Scikit-Learn, PyTorch, and Tensorflow, who want to build and operate machine learning solutions in the cloud. Overview Learn how to operate machine learning solutions at cloud scale using Azure Machine Learning. This course teaches you to leverage your existing knowledge of Python and machine learning to manage data ingestion and preparation, model training and deployment, and machine learning solution monitoring in Microsoft Azure. Learn how to operate machine learning solutions at cloud scale using Azure Machine Learning. This course teaches you to leverage your existing knowledge of Python and machine learning to manage data ingestion and preparation, model training and deployment, and machine learning solution monitoring with Azure Machine Learning and MLflow. Prerequisites Creating cloud resources in Microsoft Azure. Using Python to explore and visualize data. Training and validating machine learning models using common frameworks like Scikit-Learn, PyTorch, and TensorFlow. Working with containers AI-900T00: Microsoft Azure AI Fundamentals is recommended, or the equivalent experience. 1 - DESIGN A DATA INGESTION STRATEGY FOR MACHINE LEARNING PROJECTS * Identify your data source and format * Choose how to serve data to machine learning workflows * Design a data ingestion solution 2 - DESIGN A MACHINE LEARNING MODEL TRAINING SOLUTION * Identify machine learning tasks * Choose a service to train a machine learning model * Decide between compute options 3 - DESIGN A MODEL DEPLOYMENT SOLUTION * Understand how model will be consumed * Decide on real-time or batch deployment 4 - DESIGN A MACHINE LEARNING OPERATIONS SOLUTION * Explore an MLOps architecture * Design for monitoring * Design for retraining 5 - EXPLORE AZURE MACHINE LEARNING WORKSPACE RESOURCES AND ASSETS * Create an Azure Machine Learning workspace * Identify Azure Machine Learning resources * Identify Azure Machine Learning assets * Train models in the workspace 6 - EXPLORE DEVELOPER TOOLS FOR WORKSPACE INTERACTION * Explore the studio * Explore the Python SDK * Explore the CLI 7 - MAKE DATA AVAILABLE IN AZURE MACHINE LEARNING * Understand URIs * Create a datastore * Create a data asset 8 - WORK WITH COMPUTE TARGETS IN AZURE MACHINE LEARNING * Choose the appropriate compute target * Create and use a compute instance * Create and use a compute cluster 9 - WORK WITH ENVIRONMENTS IN AZURE MACHINE LEARNING * Understand environments * Explore and use curated environments * Create and use custom environments 10 - FIND THE BEST CLASSIFICATION MODEL WITH AUTOMATED MACHINE LEARNING * Preprocess data and configure featurization * Run an Automated Machine Learning experiment * Evaluate and compare models 11 - TRACK MODEL TRAINING IN JUPYTER NOTEBOOKS WITH MLFLOW * Configure MLflow for model tracking in notebooks * Train and track models in notebooks 12 - RUN A TRAINING SCRIPT AS A COMMAND JOB IN AZURE MACHINE LEARNING * Convert a notebook to a script * Run a script as a command job * Use parameters in a command job 13 - TRACK MODEL TRAINING WITH MLFLOW IN JOBS * Track metrics with MLflow * View metrics and evaluate models 14 - PERFORM HYPERPARAMETER TUNING WITH AZURE MACHINE LEARNING * Define a search space * Configure a sampling method * Configure early termination * Use a sweep job for hyperparameter tuning 15 - RUN PIPELINES IN AZURE MACHINE LEARNING * Create components * Create a pipeline * Run a pipeline job 16 - REGISTER AN MLFLOW MODEL IN AZURE MACHINE LEARNING * Log models with MLflow * Understand the MLflow model format * Register an MLflow model 17 - CREATE AND EXPLORE THE RESPONSIBLE AI DASHBOARD FOR A MODEL IN AZURE MACHINE LEARNING * Understand Responsible AI * Create the Responsible AI dashboard * Evaluate the Responsible AI dashboard 18 - DEPLOY A MODEL TO A MANAGED ONLINE ENDPOINT * Explore managed online endpoints * Deploy your MLflow model to a managed online endpoint * Deploy a model to a managed online endpoint * Test managed online endpoints 19 - DEPLOY A MODEL TO A BATCH ENDPOINT * Understand and create batch endpoints * Deploy your MLflow model to a batch endpoint * Deploy a custom model to a batch endpoint * Invoke and troubleshoot batch endpoints

Duration 1 Days 6 CPD hours This course is intended for This course is intended for the following participants:Cloud professionals interested in taking the Data Engineer certification exam.Data engineering professionals interested in taking the Data Engineer certification exam. Overview This course teaches participants the following skills: Position the Professional Data Engineer Certification Provide information, tips, and advice on taking the exam Review the sample case studies Review each section of the exam covering highest-level concepts sufficient to build confidence in what is known by the candidate and indicate skill gaps/areas of study if not known by the candidate Connect candidates to appropriate target learning This course will help prospective candidates plan their preparation for the Professional Data Engineer exam. The session will cover the structure and format of the examination, as well as its relationship to other Google Cloud certifications. Through lectures, quizzes, and discussions, candidates will familiarize themselves with the domain covered by the examination, to help them devise a preparation strategy. Rehearse useful skills including exam question reasoning and case comprehension. Tips and review of topics from the Data Engineering curriculum. UNDERSTANDING THE PROFESSIONAL DATA ENGINEER CERTIFICATION * Position the Professional Data Engineer certification among the offerings * Distinguish between Associate and Professional * Provide guidance between Professional Data Engineer and Associate Cloud Engineer * Describe how the exam is administered and the exam rules * Provide general advice about taking the exam SAMPLE CASE STUDIES FOR THE PROFESSIONAL DATA ENGINEER EXAM * Flowlogistic * MJTelco * DESIGNING AND BUILDING (REVIEW AND PREPARATION TIPS) * Designing data processing systems * Designing flexible data representations * Designing data pipelines * Designing data processing infrastructure * Build and maintain data structures and databases * Building and maintaining flexible data representations * Building and maintaining pipelines * Building and maintaining processing infrastructure * ANALYZING AND MODELING (REVIEW AND PREPARATION TIPS) * Analyze data and enable machine learning * Analyzing data * Machine learning * Machine learning model deployment * Model business processes for analysis and optimization * Mapping business requirements to data representations * Optimizing data representations, data infrastructure performance and cost * RELIABILITY, POLICY, AND SECURITY (REVIEW AND PREPARATION TIPS) * Design for reliability * Performing quality control * Assessing, troubleshooting, and improving data representation and data processing infrastructure * Recovering data * Visualize data and advocate policy * Building (or selecting) data visualization and reporting tools * Advocating policies and publishing data and reports * Design for security and compliance * Designing secure data infrastructure and processes * Designing for legal compliance * RESOURCES AND NEXT STEPS * Resources for learning more about designing data processing systems, data structures, and databases * Resources for learning more about data analysis, machine learning, business process analysis, and optimization * Resources for learning more about data visualization and policy Resources for learning more about reliability design * Resources for learning more about business process analysis and optimization * Resources for learning more about reliability, policies, security, and compliance ADDITIONAL COURSE DETAILS: Nexus Humans Preparing for the Professional Data Engineer Examination training program is a workshop that presents an invigorating mix of sessions, lessons, and masterclasses meticulously crafted to propel your learning expedition forward. This immersive bootcamp-style experience boasts interactive lectures, hands-on labs, and collaborative hackathons, all strategically designed to fortify fundamental concepts. Guided by seasoned coaches, each session offers priceless insights and practical skills crucial for honing your expertise. Whether you're stepping into the realm of professional skills or a seasoned professional, this comprehensive course ensures you're equipped with the knowledge and prowess necessary for success. While we feel this is the best course for the Preparing for the Professional Data Engineer Examination course and one of our Top 10 we encourage you to read the course outline to make sure it is the right content for you. Additionally, private sessions, closed classes or dedicated events are available both live online and at our training centres in Dublin and London, as well as at your offices anywhere in the UK, Ireland or across EMEA.

Duration 1 Days 6 CPD hours This course is intended for This course is intended for: A technical audience at an intermediate level Overview Using Amazon SageMaker, this course teaches you how to: Prepare a dataset for training. Train and evaluate a machine learning model. Automatically tune a machine learning model. Prepare a machine learning model for production. Think critically about machine learning model results In this course, learn how to solve a real-world use case with machine learning and produce actionable results using Amazon SageMaker. This course teaches you how to use Amazon SageMaker to cover the different stages of the typical data science process, from analyzing and visualizing a data set, to preparing the data and feature engineering, down to the practical aspects of model building, training, tuning and deployment. DAY 1 * Business problem: Churn prediction Load and display the dataset Assess features and determine which Amazon SageMaker algorithm to use Use Amazon Sagemaker to train, evaluate, and automatically tune the model Deploy the model Assess relative cost of errors ADDITIONAL COURSE DETAILS: Nexus Humans Practical Data Science with Amazon SageMaker training program is a workshop that presents an invigorating mix of sessions, lessons, and masterclasses meticulously crafted to propel your learning expedition forward. This immersive bootcamp-style experience boasts interactive lectures, hands-on labs, and collaborative hackathons, all strategically designed to fortify fundamental concepts. Guided by seasoned coaches, each session offers priceless insights and practical skills crucial for honing your expertise. Whether you're stepping into the realm of professional skills or a seasoned professional, this comprehensive course ensures you're equipped with the knowledge and prowess necessary for success. While we feel this is the best course for the Practical Data Science with Amazon SageMaker course and one of our Top 10 we encourage you to read the course outline to make sure it is the right content for you. Additionally, private sessions, closed classes or dedicated events are available both live online and at our training centres in Dublin and London, as well as at your offices anywhere in the UK, Ireland or across EMEA.

Duration 5 Days 30 CPD hours This course is intended for This intermediate and beyond level course is geared for experienced technical professionals in various roles, such as developers, data analysts, data engineers, software engineers, and machine learning engineers who want to leverage Scala and Spark to tackle complex data challenges and develop scalable, high-performance applications across diverse domains. Practical programming experience is required to participate in the hands-on labs. Overview Working in a hands-on learning environment led by our expert instructor you'll: Develop a basic understanding of Scala and Apache Spark fundamentals, enabling you to confidently create scalable and high-performance applications. Learn how to process large datasets efficiently, helping you handle complex data challenges and make data-driven decisions. Gain hands-on experience with real-time data streaming, allowing you to manage and analyze data as it flows into your applications. Acquire practical knowledge of machine learning algorithms using Spark MLlib, empowering you to create intelligent applications and uncover hidden insights. Master graph processing with GraphX, enabling you to analyze and visualize complex relationships in your data. Discover generative AI technologies using GPT with Spark and Scala, opening up new possibilities for automating content generation and enhancing data analysis. Embark on a journey to master the world of big data with our immersive course on Scala and Spark! Mastering Scala with Apache Spark for the Modern Data Enterprise is a five day hands on course designed to provide you with the essential skills and tools to tackle complex data projects using Scala programming language and Apache Spark, a high-performance data processing engine. Mastering these technologies will enable you to perform a wide range of tasks, from data wrangling and analytics to machine learning and artificial intelligence, across various industries and applications.Guided by our expert instructor, you?ll explore the fundamentals of Scala programming and Apache Spark while gaining valuable hands-on experience with Spark programming, RDDs, DataFrames, Spark SQL, and data sources. You?ll also explore Spark Streaming, performance optimization techniques, and the integration of popular external libraries, tools, and cloud platforms like AWS, Azure, and GCP. Machine learning enthusiasts will delve into Spark MLlib, covering basics of machine learning algorithms, data preparation, feature extraction, and various techniques such as regression, classification, clustering, and recommendation systems. INTRODUCTION TO SCALA * Brief history and motivation * Differences between Scala and Java * Basic Scala syntax and constructs * Scala's functional programming features INTRODUCTION TO APACHE SPARK * Overview and history * Spark components and architecture * Spark ecosystem * Comparing Spark with other big data frameworks BASICS OF SPARK PROGRAMMING SPARKCONTEXT AND SPARKSESSION * Resilient Distributed Datasets (RDDs) * Transformations and Actions * Working with DataFrames SPARK SQL AND DATA SOURCES * Spark SQL library and its advantages * Structured and semi-structured data sources * Reading and writing data in various formats (CSV, JSON, Parquet, Avro, etc.) * Data manipulation using SQL queries BASIC RDD OPERATIONS * Creating and manipulating RDDs * Common transformations and actions on RDDs * Working with key-value data BASIC DATAFRAME AND DATASET OPERATIONS * Creating and manipulating DataFrames and Datasets * Column operations and functions * Filtering, sorting, and aggregating data INTRODUCTION TO SPARK STREAMING * Overview of Spark Streaming * Discretized Stream (DStream) operations * Windowed operations and stateful processing PERFORMANCE OPTIMIZATION BASICS * Best practices for efficient Spark code * Broadcast variables and accumulators * Monitoring Spark applications INTEGRATING EXTERNAL LIBRARIES AND TOOLS, SPARK STREAMING * Using popular external libraries, such as Hadoop and HBase * Integrating with cloud platforms: AWS, Azure, GCP * Connecting to data storage systems: HDFS, S3, Cassandra, etc. INTRODUCTION TO MACHINE LEARNING BASICS * Overview of machine learning * Supervised and unsupervised learning * Common algorithms and use cases INTRODUCTION TO SPARK MLLIB * Overview of Spark MLlib * MLlib's algorithms and utilities * Data preparation and feature extraction LINEAR REGRESSION AND CLASSIFICATION * Linear regression algorithm * Logistic regression for classification * Model evaluation and performance metrics CLUSTERING ALGORITHMS * Overview of clustering algorithms * K-means clustering * Model evaluation and performance metrics COLLABORATIVE FILTERING AND RECOMMENDATION SYSTEMS * Overview of recommendation systems * Collaborative filtering techniques * Implementing recommendations with Spark MLlib INTRODUCTION TO GRAPH PROCESSING * Overview of graph processing * Use cases and applications of graph processing * Graph representations and operations * Introduction to Spark GraphX * Overview of GraphX * Creating and transforming graphs * Graph algorithms in GraphX BIG DATA INNOVATION! USING GPT AND GENERATIVE AI TECHNOLOGIES WITH SPARK AND SCALA * Overview of generative AI technologies * Integrating GPT with Spark and Scala * Practical applications and use cases Bonus Topics / Time Permitting INTRODUCTION TO SPARK NLP * Overview of Spark NLP Preprocessing text data * Text classification and sentiment analysis PUTTING IT ALL TOGETHER * Work on a capstone project that integrates multiple aspects of the course, including data processing, machine learning, graph processing, and generative AI technologies.