Start Monitoring Snowflake Data and Set Up Customizable Data Alerts
Background
What you’ll learn
- Connect Monte Carlo to Snowflake.
- Monitor Data Anomalies in Monte Carlo.
- Configure Customizable Data Notifications.
- Custom SQL Data Monitors.
- Field Health Monitors.
- JSON Schema Monitors.
- Volume and Freshness Monitors.
- Infrastructure as Code.
- Github Actions for CI/CD.
- Useful Snowflake Monitors for All Organizations.
Course Content
- Basics –> 10 lectures • 1hr 54min.
- Setting Up Realistic Data and Monitors –> 4 lectures • 47min.
- Essential Snowflake Monitors –> 7 lectures • 52min.
- Advanced Features and Best Practices –> 6 lectures • 1hr.
Requirements
- SQL.
- git.
- Ability to work in the command line.
Background
Monte Carlo is an incredibly powerful Data Observability tool for your data stack that monitors and alerts for data issues across your data warehouses, data lakes, ETL, and business intelligence. The platform uses machine learning to infer and learn your data, proactively identify data issues, assess their impact, and notify those who need to know. Beyond their ML monitoring, Monte Carlo offers robust customizable data monitoring and alerting so you can set up all types of data alerts for your organization – whether they are for data issues that need to be addressed, or simply for alerts that a specific metric has reached a certain threshold.
What you’ll learn
By the end of the class, you’ll have completely connected your own Monte Carlo and Snowflake accounts following best practices, and you’ll be able to monitor and set up alerts on your Snowflake objects. Some of these skills you’ll gain include:
- Configuring customizable monitors based on any data in your warehouse
- Ensuring your data pipelines remain fresh with monitoring and alerting
- Setting up useful Snowflake monitors to enhance the monitoring of your Snowflake
- Configuring notifications through Slack or email
- Managing all of the monitoring infrastructure through code
- Setting up github actions to preview the infrastructure changes before deployment, and automatically deploy on merge requests
Why you should use Monte Carlo to monitor your Snowflake account
Snowflake is an incredible data warehouse critical to thousands of organizations. But effectively monitoring Snowflake at scale has been difficult, until we start taking advantage of Monte Carlo.
Some of the benefits of using Monte Carlo with Snowflake include:
- Automatically monitoring all relevant data with Monte Carlo’s ML monitoring
- Detect data quality issues in your pipeline before they reach downstream reports
- Identify and manage data anomalies by assigning an owner to each data incident that needs investigation
Essentially, Monte Carlo allows you to use Infrastructure as Code to have full control over monitoring your Snowflake account. By establishing a process for identifying and triaging data incidents, you can efficiently manage your Snowflake account knowing that you have the monitors and alerts in place to let you know about any issues.
About Monte Carlo
Monte Carlo lets you monitor other platforms as well including other Data Warehouses, Data Lakes, and even BI Tools like Tableau. They’re working to reduce data downtime for organizations and are continually pushing to help data teams measure the health of their applications.