How to Install StarRocks Locally: A Comprehensive Guide for Developers and Data Engineers

StarRocks

Introduction

Installing data tools locally is crucial for testing, developing, or experimenting with their functionality in a controlled environment. StarRocks, an open-source distributed SQL engine optimized for real-time analytics, is no exception. A full local installation involves deploying all its components: the Frontend (FE), Backend (BE), and optionally the Broker (for external file system support). This guide walks you through using a Docker Compose setup to deploy all components efficiently, along with programming language-specific options to interact with StarRocks.


1. Installing StarRocks Locally with Docker Compose

To deploy the full StarRocks stack locally, a Docker Compose configuration is the easiest and most efficient method. Docker Compose allows you to set up multiple containers for the Frontend, Backend, and Broker services, ensuring proper communication between them.

Step 1: Prerequisites

Step 2: Create a Docker Compose File

Create a docker-compose.yml file in an empty directory with the following content:

version: '3.8'
services:
  fe:
    image: starrocks/starrocks-fe:latest
    container_name: starrocks-fe
    ports:
      - "9030:9030"
      - "8040:8040"
    environment:
      - FE_METADIR=/opt/starrocks/meta
    volumes:
      - fe-meta:/opt/starrocks/meta
    depends_on:
      - be

  be:
    image: starrocks/starrocks-be:latest
    container_name: starrocks-be
    ports:
      - "9060:9060"
    environment:
      - BE_DATADIR=/opt/starrocks/storage
    volumes:
      - be-data:/opt/starrocks/storage

  broker:
    image: starrocks/starrocks-broker:latest
    container_name: starrocks-broker
    ports:
      - "8000:8000"

volumes:
  fe-meta:
  be-data:

This configuration ensures that the FE, BE, and Broker services are deployed and properly interconnected.

Step 3: Start the Cluster

Run the following command in the same directory as your docker-compose.yml file:

docker-compose up -d

Docker Compose will pull the required images and start the containers.

Step 4: Verify the Installation

Open a browser and navigate to http://localhost:8040. This will show the StarRocks FE admin console.

Use the following command to check the running containers:

docker ps

You should see starrocks-fe, starrocks-be, and starrocks-broker containers listed.

Step 5: Register Backend Nodes

By default, the FE and BE services need to be linked. Access the StarRocks FE container and register the BE node:

docker exec -it starrocks-fe bash
mysql -uroot -h127.0.0.1 -P9030 -e "ALTER SYSTEM ADD BACKEND 'starrocks-be:9060';"

This command informs the FE of the BE’s existence, enabling proper data storage and retrieval.


2. Programming Language-Specific Installations

Once the full StarRocks stack is running, you can interact with it using your preferred programming language. Below are examples for Python, Node.js, Ruby, and Java.


Python (pip)

Step 1: Install the Python Client

pip install starrocks-python

Step 2: Connect to StarRocks

import starrocks

# Connect to StarRocks
connection = starrocks.connect(
    host="localhost",
    port=9030,
    user="root",
    password="",
    database="default"
)

# Execute a query
cursor = connection.cursor()
cursor.execute("SELECT * FROM some_table;")
print(cursor.fetchall())

Node.js (npm)

Step 1: Set Up a Node.js Project

npm init -y
npm install axios

Step 2: Interact with StarRocks

const axios = require('axios');

axios.post('http://localhost:8040/api/query', {
  sql: 'SELECT * FROM some_table LIMIT 10;'
})
.then(response => console.log(response.data))
.catch(error => console.error(error));

Ruby (gem)

Step 1: Install the Rest-Client Gem

gem install rest-client

Step 2: Query StarRocks

require 'rest-client'
require 'json'

url = 'http://localhost:8040/api/query'
payload = { sql: 'SELECT * FROM some_table LIMIT 10;' }

response = RestClient.post(url, payload.to_json, {content_type: :json, accept: :json})
puts JSON.parse(response.body)

Java (Maven/Gradle)

Step 1: Add JDBC Dependency

For Maven, include the following in your pom.xml:

<dependency>
  <groupId>com.starrocks</groupId>
  <artifactId>starrocks-jdbc</artifactId>
  <version>latest_version</version>
</dependency>

For Gradle, add this to your build.gradle:

implementation 'com.starrocks:starrocks-jdbc:latest_version'

Step 2: Connect to StarRocks

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;

public class StarRocksExample {
    public static void main(String[] args) {
        try {
            Connection connection = DriverManager.getConnection(
                "jdbc:starrocks://localhost:9030", "root", "");

            Statement statement = connection.createStatement();
            ResultSet resultSet = statement.executeQuery("SELECT * FROM some_table LIMIT 10;");

            while (resultSet.next()) {
                System.out.println(resultSet.getString(1));
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

3. Managing and Verifying Your Installation

Once installed, you can manage and verify your StarRocks setup with the following tips:

  • Monitor Containers: Use docker ps to ensure all components are running. Restart individual containers with docker restart <container_name>.
  • Check Logs: Use docker logs <container_name> to troubleshoot issues in FE, BE, or Broker components.
  • Run Sample Queries: Use the StarRocks admin console or your preferred programming language to execute basic queries like SHOW DATABASES; to confirm the cluster is functioning correctly.

Conclusion

Deploying StarRocks locally with all its components (FE, BE, and Broker) ensures a complete setup for real-time analytics testing and development. Using Docker Compose streamlines the process, while programming language-specific integrations allow for seamless interaction with the database. By following this guide, you’ll have a robust local StarRocks environment ready to handle your data challenges.

Last Releases

  • 3.3.15
    [BugFix] Revert Catch cancel status from be to retry (#59642) (#59988)   Source: https://github.com/StarRocks/starrocks/releases/tag/3.3.15
  • 3.5.0: [BugFix] Revert PR #59009 (backport #59815) (#59827)
    Signed-off-by: sevev qiangzh95@gmail.com Co-authored-by: zhangqiang qiangzh95@gmail.com   Source: https://github.com/StarRocks/starrocks/releases/tag/3.5.0
  • 3.4.4
    [BugFix] Fix invalid database id when recycling partition (backport #…   Source: https://github.com/StarRocks/starrocks/releases/tag/3.4.4

More From Author

Leave a Reply

Recent Comments

No comments to show.