Snowflake previews data marketplace, governance tools, management features

Snowflake today announced general availability of a marketplace that will allow organizations to sell data based on usage. This was one of several announcements regarding new data monetization, governance, and performance features for the cloud platform.

Announced during the online Snowflake Summit, the new capabilities will make the Snowflake Data Marketplace more accessible to a wider range of organizations that don’t typically monetize their data today, Snowflake senior VP Christian Kleinerman said.

Data marketplace

There are now more than 160 providers of data services participating in that marketplace. The bulk of those organizations sell data commercially as a core part of their business model. However, roughly 20% to 30% of organizations might be able to monetize their data if it was simple enough for them to sell, Kleinerman added. The biggest obstacle those organizations face is they don’t have the time or resources required to set up and maintain a data marketplace of their own, he noted, adding that they can only sell data if it doesn’t distract from their mission. “It’s not their primary business,” Kleinerman said.

Snowflake today also announced forthcoming editions to its platform, including a Snowpark applications development environment available via private preview. That development platform currently supports Java and Scala programming languages, with support for additional languages planned for a future release.

The company also previewed Java user-defined-functions (UDFs) that will enable IT teams to deploy custom code and business logic they have developed on the Snowflake platform, as well as a Snowflake SQL application programming interface (API) through which applications will be able to directly invoke Snowflake.

Snowflake also discussed new governance tools for managing unstructured data stored on a Snowflake platform that is based on a relational database engine. There is also a classification tool that automatically detects personally identifiable information (PII) in a table and uses the tagging framework to annotate the data in a way that makes it possible for IT teams to enforce compliance policies.

The company committed to adding support for Anonymized views that will better protect the privacy and identity of a dataset by automatically redacting sensitive data.

Daily operations

In terms of daily operations on the platform, Snowflake previewed a usage dashboard to enable customers to more easily track consumption of resources and revealed it has been employing a new compression algorithm to reduce the size of datasets anywhere from 9% to 30%, depending on the type of data, Kleinerman said. Snowflake also previewed updates that can result in an up to sixfold improvement to query throughput on a single compute cluster and an up to eightfold improvement in average query duration.

Finally, Snowflake said its platform can now be used as a data source for Amazon SageMaker Data Wrangler, a tool the cloud service provider makes available to collect data for training AI models. The company said it is rolling out a Powered by Snowflake branding initiative for partners that build applications on top of its cloud platform.

Snowflake as a cloud service running on the Amazon Web Services cloud has gained a significant amount of traction, despite the fact that AWS offers rival services. In addition, competitors are making a case for data lakes based on object-storage systems that eliminate the need for a separate platform to query a database. The core capabilities that differentiate Snowflake are how tightly coupled datasets are made available and the overall governance capabilities this capability enables Snowflake to provide, said Kleinerman.

It’s not clear how the battle between data platforms in the cloud will play out. The line between what constitutes a data warehouse versus a data lake is getting blurry, but it’s clear that as more data gets stored in the cloud it will become more challenging to bring order to the potential chaos.