
Introduction: Installing SQLMesh locally provides data engineers and developers with a controlled environment for testing, development, and offline work. A local installation enables users to experiment with SQLMesh’s features—such as its SQL-based transformation and model deployment tools—without affecting production environments. This guide will walk through SQLMesh’s local installation methods using Docker and several popular programming languages.
1. Docker Installation
Why Docker?
Using Docker to install SQLMesh allows you to set up a containerized environment that isolates dependencies and configurations, making it easier to replicate setups across different systems.
Prerequisites
- Docker Desktop installed on your system
Steps
Pull the SQLMesh Docker Image
docker pull sqlmesh/sqlmesh:latest
Run the SQLMesh Container
docker run -d --name sqlmesh-container -p 8080:8080 sqlmesh/sqlmesh:latest
Verify Installation: Once the container is running, verify SQLMesh by accessing it through the command line:
docker exec -it sqlmesh-container sqlmesh --version
2. Installing SQLMesh with Pip (Python)
Python is widely used in data engineering, making pip
a convenient method for installing SQLMesh for Python users.
Prerequisites
- Python 3.7+
- pip installed
Steps
Install SQLMesh via pip
pip install sqlmesh
Verify Installation: Confirm SQLMesh is installed by checking the version:
sqlmesh --version
Example Usage: After installation, you can start using SQLMesh in a Python script or Jupyter Notebook:
import sqlmesh
3. Installing SQLMesh with npm (Node.js)
If you’re a Node.js user integrating SQLMesh into JavaScript-based data pipelines, you can install it with npm
.
Prerequisites
Steps
Install SQLMesh via npm
npm install sqlmesh
Verify Installation: Check if SQLMesh is installed correctly by running:
sqlmesh --version
Example Usage in Node.js: Import SQLMesh in your Node.js project:
const sqlmesh = require('sqlmesh');
4. Installing SQLMesh with Gem (Ruby)
Although SQLMesh is less commonly used with Ruby, a gem installation can make integration easier for Ruby-based projects.
Prerequisites
Steps
Install SQLMesh using Gem
gem install sqlmesh
Verify Installation: Confirm SQLMesh installation by checking the version:
sqlmesh --version
5. Installing SQLMesh with Maven or Gradle (Java)
For Java-based applications, SQLMesh can be integrated via dependency management tools like Maven or Gradle.
Using Maven
Add SQLMesh to Your Project: In your pom.xml
, add SQLMesh as a dependency:
<dependency>
<groupId>com.sqlmesh</groupId>
<artifactId>sqlmesh</artifactId>
<version>latest</version>
</dependency>
Build Project
Once added, build your project to download and link SQLMesh:
mvn clean install
Using Gradle
Add SQLMesh to build.gradle
:
implementation 'com.sqlmesh:sqlmesh:latest'
Sync Project
Sync your Gradle project to fetch the SQLMesh dependency.
Verify Installation
In your Java code, you should now be able to import SQLMesh:
import com.sqlmesh.*;
Managing and Verifying Installations
Once installed, managing and verifying SQLMesh can ensure consistent performance and avoid version conflicts.
- Confirm the Installation Path: Use commands like
which sqlmesh
(Linux/macOS) orwhere sqlmesh
(Windows) to locate your SQLMesh binary or executable. - Test Compatibility with Other Tools: If integrating SQLMesh with other data tools (e.g., Apache Kafka, Airflow), perform a test run to check for any compatibility issues.
- Keep Dependencies Updated: Regularly update SQLMesh and any related dependencies using package managers (
pip
,npm
, etc.) to maintain compatibility with your environment.
Conclusion: Installing SQLMesh locally is essential for those looking to develop, test, and maintain SQL-based data transformations independently of production environments. With support for Docker, Python, Node.js, Ruby, and Java, SQLMesh offers flexibility across various tech stacks. By following these installation steps and managing dependencies, you can ensure SQLMesh functions smoothly in your workflow.
Last Releases
- v0.195.0✨ Features Allow virtual environments to be given dedicated catalogs (#4742) @erindru (vscode) Ability to stop server (#4775) @benfdking Add formatter options to render command (#4753) @georgesittas 🪲 Bug fixes… Read more: v0.195.0
- v0.187.9🧹 Chore Bump sqlglot to 26.26.2 @georgesittas Source: https://github.com/TobikoData/sqlmesh/releases/tag/v0.187.9
- v0.194.1✨ Features Use new tcloud sqlmesh_lsp call (#4762) @benfdking (vscode) .env file into python env (#4764) @benfdking 🪲 Bug fixes (vscode) Allow uv pip executable called through tcloud (#4766) @benfdking… Read more: v0.194.1