After just a pair of short months in preview, Stability AI is today unleashing its next-generation Stable Diffusion 3 generative AI model. Alongside the update, Stability AI is also providing an early preview of a new chatbot technology dubbed "Stable Assistant."

Stable Diffusion 3 was first announced as a preview back in February. As of today, Stable Diffusion 3 will now be available for use via an API on the Stability AI developer platform. As an API, Stable Diffusion can be integrated into services and applications that make use of the text-to-image generation capabilities the model provides. In addition to the base model, there is now also a Stable Diffusion 3 Turbo model as well.

With Stable Diffusion 3, Stability AI has implemented a series of novel machine learning and AI techniques to help improve image generation quality as well as typography. A key focus for the company as part of the API release has been to make it production-ready.

"We have implemented a number of safeguards that help prevent misuse of SD3 and we continue to refine these safeguards based on user feedback,"  Christian Laforte, CTO and interim co-CEO at Stability AI told VentureBeat in an exclusive interview.

Open model is coming, but not yet

While Stable Diffusion 3 is now available via API, there isn't an open model that is generally available, at least not yet, but it is coming.

"We will continuously work to improve the model in advance of its open release," Laforte said. "In keeping with our commitment to open generative AI, we aim to make the model weights available for self-hosting with a Stability AI Membership soon."

The Stability AI membership is the strategy the company first announced in December to help it build out a new revenue model.

Fireworks will help to power and enable the Stable Diffusion 3 API

Of particular note with the Stable Diffusion 3 API is that it will benefit from a partnership Stability AI has with API platform provider Fireworks AI.

Enabling full inferencing performance of an API for a gen AI application can be a complicated operation, especially when done at scale. That's the challenge that Fireworks AI will help to solve.

"Fireworks AI are industry-leading [machine learning] ML compiler experts, which is a critical component of optimizing the inference speed of our models," Laforte said. "By partnering with them to power our Stable Diffusion 3 API we can provide the fastest and most reliable enterprise-grade API platform in the market."

Latent Adversarial Diffusion Distillation (LADD) powers Turbo model

The concept of a diffusion model has always been at the heart of Stable Diffusion, that's where its name comes from after all. With Stable Diffusion 3, several novel innovations go beyond the diffusion approach that powered the first iteration of Stable Diffusion.

One of the innovations is the Multimodal Diffusion Transformer (MMDiT) architecture which for the first time brings a transformer to Stable Diffusion. The benefit is that it can significantly improve text understanding as well as dramatically improve typography spelling.

The other novel innovation to note is specific to the Stable Diffusion Turbo (SD3-Turbo) model which is intended to be a faster version of Stable Diffusion 3. SD3-Turbo uses a new method called Latent Adversarial Diffusion Distillation (LADD) as outlined in the Stable Diffusion 3 Turbo research paper. 

"Essentially, SD3-Turbo runs much faster than SD3, up to 10X faster and produces images that are, on average, almost as good as SD3," Laforte said.

Coming up next? Stable Assistant 

As if a new Stable Diffusion model wasn't enough, Stability AI is also now providing an early beta preview of its next big innovation dubbed - Stable Assistant.

The basic idea behind Stable Assistant is not that dissimilar to how OpenAI's ChatGPT Plus chatbot is now integrated with OpenAI's DALL-E 3 to do both text and image generation.

Laforte said that Stable Assistant is a friendly chatbot powered by Stability AI’s text and image generation technology, featuring Stable Diffusion 3 and Stable LM 2 12B, which was just released earlier this month. He added that it will enable users to generate images from conversations, offer knowledgeable responses, help with writing projects and enhance content with matching images.

"Stable Assistant aims to be Stability AI’s multimodal chatbot where all our models and API services will be available to use, without technical expertise," Laforte said. "Language and image generation have already been integrated and we aim to continue developing Stable Assistant capabilities by adding image editing very soon, and including models from our other available modalities: video, 3D, audio, and code."