Apache Storm Development Environment Setup in Eclipse with Maven

This tutorial will guide you through the process of setting up new Apache Storm project in your Eclipse IDE.

Requirements:

1. Install latest version of JDK
2. Install latest version of Maven
3. Install latest version of Eclipse IDE

Follow the steps below to create new sample project;

1. Create a new Maven Project
File-->Project-->Maven Project
2. Define arbitrarily “groupId” and “artifactId” of your project.
3. Open “pom.xml” file which will be appear in your project folder.
4. Define the following dependencies for a starter Storm project. These dependencies may vary according to your library needs for your project. Included libraries are;

  • Apache Storm Core(1.0.2)
  • Apache Storm-Kafka Integration (1.0.2)
  • Apache Kafka (0.8.1)

    • You need to exclude “Logging” and “Zookeeper” libraries in Kafka because Apache Storm Core already includes these libraries. Otherwise, your project will fail due to including same libraries more than once.
    • Apache Storm compatible with Apache Kafka 0.8.1, if you include Apache Kafka 0.8.1+, your application will crash.

         <dependencies>
          <dependency>
              <groupId>junit</groupId>
              <artifactId>junit</artifactId>
              <version>3.8.1</version>
              <scope>test</scope>
          </dependency>
          <dependency>
              <groupId>org.apache.storm</groupId>
              <artifactId>storm-core</artifactId>
              <version>1.0.2</version>
              <scope>provided</scope>
          </dependency>
          <dependency>
              <groupId>org.apache.storm</groupId>
              <artifactId>storm-kafka</artifactId>
              <version>1.0.2</version>
              <exclusions>
                  <exclusion>
                      <groupId>log4j</groupId>
                      <artifactId>log4j</artifactId>
                  </exclusion>
              </exclusions>
          </dependency>
          <dependency>
              <groupId>org.apache.kafka</groupId>
              <artifactId>kafka_2.10</artifactId>
              <version>0.8.1.1</version>
              <exclusions>
                  <exclusion>
                      <groupId>org.apache.zookeeper</groupId>
                      <artifactId>zookeeper</artifactId>
                  </exclusion>
                  <exclusion>
                      <groupId>org.slf4j</groupId>
                      <artifactId>slf4j-log4j12</artifactId>
                  </exclusion>
                  <exclusion>
                      <groupId>log4j</groupId>
                      <artifactId>log4j</artifactId>
                  </exclusion>
              </exclusions>
          </dependency>
          <dependency>
              <groupId>commons-collections</groupId>
              <artifactId>commons-collections</artifactId>
              <version>3.2.1</version>
          </dependency>
          <dependency>
              <groupId>org.apache.curator</groupId>
              <artifactId>curator-test</artifactId>
              <version>2.4.0</version>
              <exclusions>
                  <exclusion>
                      <groupId>org.jboss.netty</groupId>
                      <artifactId>netty</artifactId>
                  </exclusion>
                  <exclusion>
                      <groupId>org.slf4j</groupId>
                      <artifactId>slf4j-log4j12</artifactId>
                  </exclusion>
              </exclusions>
          </dependency>
      </dependencies>
      
  • In production of Apache Storm projects, you need to generate and upload a “Jar” file which includes all of the dependencies. You can easily generate “Jar” file using Maven with the following lines;

    <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>2.3.2</version>
                    <configuration>
                        <source>1.7</source>
                        <target>1.7</target>
                    </configuration>
                </plugin>
                <!-- Maven Assembly Plugin -->
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-assembly-plugin</artifactId>
                    <version>2.4.1</version>
                    <configuration>
                        <!-- get all project dependencies -->
                        <descriptorRefs>
                            <descriptorRef>jar-with-dependencies</descriptorRef>
                        </descriptorRefs>
                        <!-- MainClass in mainfest make a executable jar -->
                        <archive>
                            <manifest>
                                <mainClass>org.odtu.Storm</mainClass>
                            </manifest>
                        </archive>
                    </configuration>
                    <executions>
                        <execution>
                            <id>make-assembly</id>
                            <!-- bind to the packaging phase -->
                            <phase>package</phase>
                            <goals>
                                <goal>single</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    
  • Run Maven Clean
  • Run Maven Install
  • Your development environment is ready to run your application.

Mert Gökalp

Read more posts by this author.

Ankara/Turkey

Subscribe to Pseudo Factory

Get the latest posts delivered right to your inbox.

or subscribe via RSS with Feedly!