Fivetran improves data transformation with integrated scheduling

Fivetran, a data integration platform that helps enterprises “extract, load, and transform” (ELT) data from multiple SaaS and on-prem sources, today announced two key improvements to Fivetran Transformation for dbt Core – integrated scheduling and data lineage graphs.

Transformations are critical in ELT as they turn raw data into clean datasets for use in downstream analytics workflows — from basic reporting to data science. Over a year ago, to simplify this transformation process and allow faster access to analytics-ready data, Fivetran integrated with Dbt Labs’ open-source transformation framework dbt Core and announced the beta launch of Fivetran Transformations for dbt Core. The offering, it said, will allow customers to take advantage of an automated cloud data integration experience in a single environment, complete with cleaning, testing, transformation, modeling, and documentation of datasets.

Product improvements

Now, with the launch of new features, Fivetran is improving this service and deepening its integration with dbt Core. Integrated scheduling, according to the company, will allow its users to schedule their dbt Core models to run automatically following the completion of a Fivetran connector sync. This will reduce data latency and speed up the end-to-end ELT pipeline while also helping customers save money on unnecessary compute costs by only running transformations on new or updated data.

Data lineage graphs, on the other hand, will visualize dbt Core data models in Fivetran, allowing users to better track and manage their end-to-end data pipelines in support of data governance. This particular feature will eliminate the need for data analysts to comb through SQL code to determine relationships between models, and provide a visual representation that data engineers can use to map data movement throughout the transformation process.

Moreover, these graphs could also be shared with data analysts and other business users across the organization for a more collaborative experience, the company notes.

“Fivetran’s ability to orchestrate dbt Core models brings the E, L, and T together and eliminates previous gaps in the process,” a Fivetran spokesperson said. “Our users are clear about their need for the freshest data while controlling their transformation costs. With Fivetran Transformations, our complete ELT data pipelines empower our customers to make revenue-impacting, data-driven decisions.”

The new capabilities come just a few months after Fivetran’s mega financing round of $565 million at a $5.6 billion valuation. The Oakland, California-based company had also acquired data replication startup HVR to provide “modern analytics for the world’s most business-critical data without compromising security, performance, or ease of use.” The terms of the deal were not disclosed.