It all started with this tweet.
Dust purpose is to accelerate the adoption of generative models, in particular large language models (LLMs), by helping developers design, evaluate, share and deploy LLM apps (chain of prompted calls to models and intermediary processing steps).
This purpose is motivated by the belief that current LLM capabilities (let alone future ones) are heavily under utilized and have yet to fully permeate non ML-specialized tech and associated products.
Dust provides a platform to design, evaluate and deploy LLM apps, enabling prompt engineers (and in the future ~anybody) to expose specific LLM capabilities within their organization without having to setup an internal specialized infrastructure for it.
Prompt design is an highly iterative process that requires interactivity and granular versioning to correctly assess and keep track of the performance of each successive prompt iteration.
For this reason, LLM app design is a challenging exercise when working with generic tools (raw code or notebooks):
These difficulties are exacerbated when working on LLM apps that consists of many interdependent model calls and intermediary processing steps which, we believe, LLMwill be the standard for most LLM apps in the future.
Dust trades total flexibility (as provided by raw code or notebooks) for specialized built-in features to assist the design of LLM apps. Dust views an LLM app as the minimal envelope around prompted model calls consisting of prompt generation, calls to remote model APIs (OpenAI, Cohere, HuggingFace, custom) and necessary intermediary processing steps defined as JavaScript lambdas (see example LLM app specification below). In particular Dust is explicitly not meant to provide an end to end no code solution and is not meant to be used to implement business logic. Dust is a platform that helps you go from tinkering with generative models to serving production traffic in a fraction of the ffort it currently takes.
Dust views an LLM app as a chain of blocks whose inputs and outputs have their types constrained to JSON objects. A block can either be a purely functional processing step as defined by a JavaScript lambda or a prompted model call. The constraint of splitting an LLM app in blocks (vs say free-form code against a fixed interface) enable the automated efficient parallelization of LLM app execution, and, thanks to the guarantee (JSON) of being able to serialize block input/output, the opportunity to provide powerful introspection tools (both not generally possible with a fully generic lambda interface).
In particular, these constraints enable: