Snowflake builds a best-in-class data stack with Fivetran

Key results:

• Deployed over 900 Fivetran connectors moving over 400M monthly active rows.
• Built a 360-degree view of customer behavior, enabling the marketing team to drive massive efficiencies in marketing spend and targeting.
• Established a comprehensive data consumption layer, which enables hundreds of internal customers and citizen analysts to access critical data.

Background:

• Company Size: 4,000+ employees
• ELT: Fivetran, Fivetran Transformations for dbt Core™
• Destination: Snowflake Data Cloud
• Fivetran Connectors: Connections Service, Marketo, Jira, Salesforce, Salesforce Sandbox, Github, Servicenow, Webhooks, Greenhouse, Hubspot, Snowplow, Google Analytics, Bing Ads, Google Ads, Linkedin Ads, Workday, Youtube Analytics, Qualtrics, Google Sheets, Facebook Pages, Facebook Ads, Instagram Business, Email, Google Drive, Stripe, Survey Monkey
• BI: Tableau, ThoughtSpot
• Other data technologies: dbt Core™, Alation, ThoughtSpot, Airflow
• Fivetran Plan: Enterprise
  ‍
  

Founded in 2012, Snowflake has become ubiquitous with the growing modern data stack ecosystem, delivering access to the data cloud for over 6,000 customers worldwide. Snowflake’s own growth has been driven by rapid iteration and best-in-class data practices, including the development of a powerful internal data stack that gives every employee at the organization access to data.

At the core of this meteoric rise is Snowflake’s marketing intelligence function, a team with a bold vision: to predict real-time ROI to dynamically optimize all Snowflake marketing programs, disrupting legacy B2B marketing analytics practices, and create huge efficiencies. 

“For most businesses, 40% of marketing budgets are wasted due to ineffectiveness of campaigns. Measurement and visibility are critical to optimizing marketing.” – Lourenço Mello, Product Marketing Lead - Solutions, Snowflake

Snowflake’s marketing intelligence team saw an opportunity to centralize its data within the organization's Snowflake instance, ‘Snowhouse,’ to power segmentation models, recommendation engines, and ultimately build a 360-degree view of customers.

Breaking down data silos

Fivetran is a critical part of the modern data stack for Snowflake, which has deployed over 900 Fivetran connectors moving over 400 million monthly active rows, both through marketing analytics connectors such as Google, Bing and Facebook Ads, and core SaaS tools like Marketo, Salesforce and Jira.

“Fivetran makes things easy. With a few clicks, we can authenticate against any of the platforms and then seamlessly and almost instantly the data appears within Snowflake.” — Carl-Johan Wehtje, BI & Analytics, Snowflake

Snowflake’s market analytics ecosystem now includes over a thousand tables for marketing, and is fully powered by the Data Cloud.

‍

Enabling analytics with Fivetran dbt Core™-compatible data models

‍

Snowflake used to keep its data modeling and transformation logic within a separate BI tool — but this approach had a few downsides. Every time the business needed to run models out of the tool, or conduct ad-hoc analytics, analysts needed to recreate their models from scratch. This approach was time-consuming and prone to error.

The implementation of dbt Core™ enabled a much more flexible experience for end users within the business. The team saw better overall performance as most of the compute-intensive calculations were conducted earlier in the process.

“We use Google, Facebook, and LinkedIn to run digital ads, and we ingest this data with Fivetran,” says Carl-Johan Wehtje. “Fivetran’s data models automatically clean up our raw data sources and output them into one model. We immediately get a clean and normalized reporting set, enabling us to focus on more complex calculations and joining work. It means we can expose this data to our end users faster, so that they can start drawing insights from it.”

Snowflake also uses Airflow to manage dbt™ jobs and define however often they want to update their models and tables, giving analysts the ability to balance data refresh times with performance and cost considerations.

‍

Establishing a data consumption layer

The ‘consumption layer’ is where Snowflake’s marketing analytics team enables collaboration and invites its business partners to start their interactions with the data. 

With a strong ingestion and transformation framework in place, the marketing analytics team can focus on driving value for the organization with advanced data science and machine learning models, attribution scoring, forecasting, and segmentation. Results from these predictive models are sent to consumption platforms, including Snowsight, Tableau, ThoughtSpot, and other tools:

• Snowsight is used for quick and simple data analysis. It’s a great starting point when users are looking to explore data and uncover simple trends and patterns in large datasets. “It's a great starting point when developing more advanced reporting because it's directly in SQL,” says Wehtje. “I can very quickly try things out, iterate, and evolve on it.”
• Tableau is the tool of choice for business-critical dashboards and reporting out at a more regular cadence. Analysts can share metrics to a wider business audience who may not necessarily have the same frame of reference. “These dashboards make it a lot easier for business leaders to see the performance on a quarter-over-quarter or on a year-over-year basis,” says Wehtje. “Dashboards are a great way of framing a clear story – which is even more important when you're sharing with wider audiences, where not everyone may have the full context or knowledge of what the key metrics are.”
• ThoughtSpot provides a comprehensive self-service reporting environment that allows for more individualized reporting. ThoughtSpot provides an intuitive way for citizen analysts to drill down and explore multiple layers into the data. And Wehtje isn’t worried about creating multiple sources of truth. “Because we’ve built clear models, we essentially use the same model regardless of whether it's the ad hoc analytics, Tableau dashboards, or ThoughtSpot dashboards. Everyone is still looking at the same data and getting the same results and outputs.”
  
  

The last component of Snowflake’s data stack is its data lineage and cataloging solution. As data volumes increase, so does the need for clear data lineage, cataloging, and governance — understanding where data has originated and how fresh it is. “As a data team, we need transparency on where data is being pulled from,” says Wehtje. “While I consider documentation hugely important, it can also be extremely time consuming.” Snowflake has addressed these needs with Alation, a tool which takes the metadata from Snowflake to automatically outline data lineage and the content of different tables.

“Looking ahead, this is really just the beginning of what we're doing,” says Wehtje. “We are working on many other things, such as creating even better ways for our internal customers to interact with the data. We’re very excited to share those with the business soon.”

Download this IDC report to learn how Fivetran drives millions in financial impact and enables new business initiatives for enterprises.