We have deprecated the Metadata API and moved metadata sharing to the Fivetran Platform connector. Learn more in our docs center.

While data teams grow and scale infrastructure to respond to increasing data requests from across the organization, it can be challenging to understand, control and secure what data is entering the system and how that data is being handled. Data governance programs meant to remedy these challenges have failed to deliver on their promises of controlled data usage because of the lack of access to information about masses of data being loaded into an organization’s infrastructure. 

To address this shortcoming, we’ve introduced the Fivetran Metadata API to ensure that enterprise data teams have full visibility over the data entering their systems and access to high quality and trusted data. With this added visibility, customers can also integrate with governance and observability tools, such Atlan, data.world, Collibra and Alation, to give data teams more control over who has access to what data. 

[CTA_MODULE]

What is the Fivetran Metadata API?

The Fivetran Metadata API enables governed data movement by providing enhanced visibility into where the data came from, who accessed it and what changes have occurred in the pipeline. The API enables customers to integrate Fivetran into their existing governance tools and processes. With this, customers can retrieve Fivetran’s metadata about data moving through the pipeline and visualize where the data comes from in lineage graphs in the discovery or catalog tool of choice. 

With the Fivetran Metadata API:

• Data analysts are provided with a deep understanding of where data is coming from and are able to run impact analyses on it.
  
• Data stewards know end-users have access to data that has been handled securely and is compliant with governance requirements.
  
• Security and legal teams can complete security audits and ensure the data moved is in compliance with organizational policies.
  
• Data architects and engineers can better understand upstream schema changes and ensure downstream processes are updated.
  

Partnering with leading data catalog providers 

The Metadata API can also share information about the data being loaded by Fivetran into a data catalog. This could be an in-house data catalog or a data catalog vendor. 

Data catalog vendors

Using a pre-built integration, Fivetran automatically sends information to data catalog providers, such as Atlan, data.world, Collibra and Alation, so that enterprise teams can see all the information about the data Fivetran has loaded, perform source trace analysis, explore lineage and better understand what data has been loaded. This allows the appropriate policies to be applied to keep that data safe.

“With the new Metadata API, customers can now bring metadata from Fivetran into Atlan for a truly automated data governance experience,” says Prukalpa Sankar, Co-Founder at Atlan. “This end-to-end lineage covers everything from upstream sources to the warehouse and BI layer, helping customers stay informed across complex data stacks with powerful root cause and impact analysis.”

“data.world's interface with Fivetran enables data producers and data consumers to understand and trust the data synchronization and mapping along all databases, data sources and applications that matter to enterprises,” Jon Loyens, CPO & Co-Founder at data.world, adds. “Together with Fivetran, we provide granular data visibility,  including metadata of source-destination column pairs, simplifying data discovery, data governance and actionable insights.”

In-house catalog

Fivetran can issue a JSON file to an in-house catalog that can be picked up, unnested and consumed by the data catalog to provide auditability and lineage.

“Fivetran’s Metadata API solves a major gap in extracting data from operational systems into modern analytical systems by delivering the much-needed context,” Sanjeev Mohan, Principal at SanjMo, a data and analytics expert and former Research Vice President at Gartner, tells us. “The availability of the Metadata API will accelerate development of reliable and secure data-intensive applications by exposing lineage, impact analysis, and security and privacy aspects.”

As organizations face new data privacy regulations and rely more on data analytics to help optimize their operations to drive decision making, our new offering will help enterprises address compliance requirements by easily integrating into their existing privacy and security strategies while bolstering the value of investments you’ve made in data catalogs and data quality solutions. 

[CTA_MODULE]

Read more in our documentation or schedule a demo by reaching out to our sales team.