How to Install Apache Beam Locally

• v2.66.0

Introduction

Apache Beam is a unified framework for processing both batch and streaming data, supporting multiple execution engines such as Apache Flink, Apache Spark, and Google Cloud Dataflow. Installing Apache Beam locally allows developers to test pipelines, develop new features, and work offline without relying on cloud environments. This guide walks through installing Apache Beam using various methods, including Docker and language-specific package managers.

Installing Apache Beam with Docker

Using Docker is one of the easiest ways to run Apache Beam without configuring dependencies manually. Ensure you have Docker installed before proceeding.

To pull the latest Apache Beam Python container, use:

docker pull apache/beam_python3.8_sdk:latest

To start an interactive session:

docker run -it apache/beam_python3.8_sdk:latest /bin/bash

For Java-based development, use:

docker pull apache/beam_java_sdk:latest

This approach ensures a clean, dependency-free environment for running Beam pipelines.

Installing Apache Beam for Python

The simplest way to install Apache Beam for Python is through pip. First, create and activate a virtual environment:

python -m venv beam_env
source beam_env/bin/activate  # On Windows, use beam_env\Scripts\activate

Then install Apache Beam:

pip install apache-beam

To verify the installation, run:

python -c "import apache_beam; print(apache_beam.__version__)"

Installing Apache Beam for Node.js

Although Apache Beam does not have native Node.js support, you can interact with it through gRPC or by using Java/Python-based APIs. If you need to process Beam jobs from Node.js, install grpc-tools:

npm install -g grpc-tools

Then, set up a gRPC connection to a Beam pipeline.

Installing Apache Beam for Java

For Java-based development, Apache Beam can be installed using Maven or Gradle.

Using Maven

Ensure Maven is installed, then create a new Java project and add the Beam dependency:

<dependency>
  <groupId>org.apache.beam</groupId>
  <artifactId>beam-sdks-java-core</artifactId>
  <version>2.50.0</version>
</dependency>

To build and verify the installation:

mvn compile

Using Gradle

For Gradle-based projects, add the following to build.gradle:

dependencies {
    implementation 'org.apache.beam:beam-sdks-java-core:2.50.0'
}

Then, build the project:

gradle build

Managing and Verifying Apache Beam Installation

After installation, verify that Apache Beam is correctly set up by running a simple pipeline. For example, in Python:

import apache_beam as beam

with beam.Pipeline() as p:
    (p | beam.Create(['Hello', 'Beam']) | beam.Map(print))

For Java, compile and execute a Beam pipeline to ensure dependencies are correctly installed.

Conclusion

Installing Apache Beam locally provides a flexible development and testing environment. Whether using Docker, pip, Maven, or Gradle, these methods ensure smooth installation across different programming languages. By following the steps above, you can confidently set up Apache Beam and start building data pipelines.

Last Releases

• v2.66.0July 1, 2025
  Tagging release   Source: https://github.com/apache/beam/releases/tag/v2.66.0
• Beam 2.66.0 releaseJune 23, 2025
  We are happy to present the new 2.66.0 release of Beam. This release includes both improvements and new functionality. For more information on changes in 2.66.0, check out the detailed… Read more: Beam 2.66.0 release
• Beam 2.66.0 releaseJune 18, 2025
  We are happy to present the new 2.66.0 release of Beam. This release includes both improvements and new functionality. For more information on changes in 2.66.0, check out the detailed… Read more: Beam 2.66.0 release