
Introduction
Open Metadata is an open-source tool designed to centralize and streamline metadata management for organizations handling complex data workflows. It provides a unified catalog, data lineage, governance features, and seamless integration with popular data tools like Apache Airflow, Kafka, and BI platforms. For local development and testing, Open Metadata can be deployed using Docker-Compose, a straightforward method that allows users to experiment with the platform’s features without setting up a production environment.
This guide walks you through running Open Metadata locally using Docker-Compose, covering prerequisites, step-by-step setup, and post-installation configuration.
Why Run Open Metadata Locally?
Running Open Metadata locally has several advantages:
- Experimentation: Test features and configurations in a controlled environment.
- Development: Build custom integrations or workflows without affecting production.
- Learning: Familiarize yourself with the platform before deploying it at scale.
Docker-Compose simplifies the process by bundling all dependencies and services (like the metadata server, database, and search engine) into a single configuration, ensuring a seamless setup experience.
Prerequisites
Before you start, ensure the following are installed on your system:
- Docker: Install Docker from the official website.
- Docker-Compose: Included in Docker Desktop or install separately for Linux systems.
- System Requirements: At least 4 CPU cores, 8 GB RAM, and sufficient disk space for Docker images and logs.
Step-by-Step Setup
Follow these steps to set up and run Open Metadata locally using Docker-Compose:
1. Clone the Repository
Start by cloning the Open Metadata GitHub repository:
git clone https://github.com/open-metadata/OpenMetadata.git
cd OpenMetadata/docker/docker-compose
The docker-compose
directory contains all necessary configuration files to spin up the Open Metadata services.
2. Modify the Environment Configuration
Open the .env
file in the docker-compose
directory to review or adjust the default settings. Key variables include:
- MYSQL_ROOT_PASSWORD: Set the root password for MySQL.
- ELASTICSEARCH_CONFIG: Adjust Elasticsearch settings if needed.
- OMD_SERVER_PORT: Define the port for the Open Metadata server (default is 8585).
For most local setups, the defaults work fine, but these can be modified to match your requirements.
3. Start the Services
Run the following command to start Open Metadata and its dependencies:
docker-compose up -d
This command pulls all required Docker images (e.g., MySQL, Elasticsearch, Open Metadata server) and starts the containers in detached mode (-d
).
4. Verify the Setup
Once the services are running, verify the installation:
- Access the Web UI: Open your browser and navigate to
http://localhost:8585
. - Login Credentials: The default credentials are:
- Username:
admin@open-metadata.org
- Password:
admin
- Username:
- Check Container Status: Use
docker ps
to ensure all containers are running.
Post-Installation Configuration
After logging in, configure Open Metadata to suit your environment:
- Ingest Data Sources: Use the UI or CLI to add data sources like databases, pipelines, or BI tools.
- Set Up Lineage Tracking: Configure the lineage feature to track data flows through your pipelines.
- Customize Metadata Schema: Tailor the metadata schema to fit your organization’s requirements.
Managing and Troubleshooting the Setup
Common Docker Commands
Stop Services
docker-compose down
View Logs
docker-compose logs -f
Restart a Specific Container
docker-compose restart <container_name>
Troubleshooting Tips
- Port Conflicts: If the default ports (e.g., 8585) are in use, modify the
docker-compose.yml
file. - Slow Startup: Allocate more system resources to Docker if services take too long to start.
- Connection Issues: Ensure MySQL and Elasticsearch containers are running by checking their logs.
Final Thoughts
Running Open Metadata locally with Docker-Compose is an efficient way to explore its features, test integrations, and develop workflows. This setup provides a sandbox environment that mirrors production, enabling teams to experiment without risk.
By following this guide, you’ll have a fully functional instance of Open Metadata running locally, ready to support your metadata management needs. With its extensive features and flexibility, Open Metadata is a valuable tool for data professionals aiming to optimize their data ecosystems.
Last Releases
- 1.7.5-releaseRelease: Add what’s new content for 1.7.5 (#21860) (cherry picked from commit 1439081) Source: https://github.com/open-metadata/OpenMetadata/releases/tag/1.7.5-release
- 1.6.13-releasefix #21394 Custom properties bug update with existing wrong values (#… …21825) * fix #21394 allow only updated extension fields to be validated * fix #21394 add tests * fix… Read more: 1.6.13-release
- 1.7.4-releaseAdd what’s new content for 1.7.4 (#21794) (cherry picked from commit 74851f8) Source: https://github.com/open-metadata/OpenMetadata/releases/tag/1.7.4-release