LMQL

Free

LMQL is a query language for large language models.

FreeFree tier

Inputs: textOutputs: text

Type

Open Source

Visit Website GitHub (40 stars)

About LMQL

LMQL is a programming language specifically designed for interacting with large language models (LLMs). It enables robust and modular prompting by incorporating types, templates, constraints, and an optimizing runtime. With LMQL, users can write expressive, multi-part prompts using Python control flow and string interpolation, enforce hard constraints on model output (such as length, stop tokens, and data types), and easily switch between multiple backends like llama.cpp, OpenAI, and Hugging Face Transformers. The language also supports advanced features like nested queries for modular instructions, tool augmentation, chatbot development, and meta prompting. LMQL is developed by the SRI Lab at ETH Zurich and is available as open source.

Key Features

Constrained LLMs with types, regex, and hard constraints via 'where' clauses

Multi-part prompts using Python control flow and string interpolation

Nested queries for modularized local instructions and re-use of prompt components

Automatic backend portability across llama.cpp, OpenAI, and Hugging Face Transformers

Support for tool augmentation, chatbots, meta prompting, and distribution measurement

Pros & Cons

Pros

Robust and modular prompting with type safety and constraints
Guaranteed output format through hard constraints
Deep Python integration for expressive programmatic prompts
Portable across multiple LLM backends with a single line of code change
Open source and free to use

Cons

Requires learning a new programming language/syntax specific to LMQL
May have a steeper learning curve compared to simpler prompting frameworks

Best For

Building complex multi-step LLM workflows with structured outputsDeveloping chatbots with constrained and reliable responsesCreating tool-augmented LLM applicationsPortable LLM code that runs across different backends without modificationInteractive prompting with precise control over output format

FAQ

What is LMQL?

LMQL is a programming language for LLM interaction that uses types, templates, constraints, and an optimizing runtime to make prompting robust and modular.

What backends does LMQL support?

LMQL supports llama.cpp, OpenAI, and Hugging Face Transformers. You can switch between them with a single line of code.

How does LMQL enforce constraints on model output?

Constraints are enforced using 'where' clauses that can limit output length, require specific stop tokens, or restrict generation to particular types such as integers or regex patterns.

Can LMQL be used for chatbots?

Yes, LMQL includes support for chatbot applications, as indicated by the chatbot feature in its documentation.