Hazelcast – Setting up multi-node instances

Given that Hazelcast is a distributed IMDG and typically is set up on multiple machines, it requires access to the internal/external network. The most important use-case being the discovery of Hazelcast nodes within a cluster.

Hazelcast requires the following ports −

  • 1 inbound port to receive pings/data from other Hazelcast nodes/clients
  • n number of outbound ports which are required to send ping/data to other members of the cluster.

This node discovery happens in few ways −

  • Multicast
  • TCP/IP
  • Amazon EC2 auto discovery

Of this, we will look at Multicast and TCP/IP

Multicast

Multicast joining mechanism is enabled by default. https://en.wikipedia.org/wiki/Multicast is a way of communication form in which a message is transmitted to all the nodes in a group. And this is what Hazelcast uses to discover other members of the cluster. All the examples that we have looked at earlier use multicast to discover members.

Example

Let’s now explicitly turn it on. Save the following in hazelcast-multicast.xml

<hazelcast
   xsi:schemaLocation="http://www.hazelcast.com/schema/config
   http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd"
   xmlns="http://www.hazelcast.com/schema/config"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 
   <network>
      <join>
         <multicast enabled="true" />
      </join>
   </network>
</hazelcast>

And then, let us execute the following −

java -Dhazelcast.config=hazelcast-multicast.xml -cp .\target\demo-0.0.1-
SNAPSHOT.jar com.example.demo.XMLConfigLoadExample

Output

In the output, we notice the following lines from Hazelcast which effectively means that multicast joiner is used to discover the members.

Jan 30, 2021 5:26:15 PM com.hazelcast.instance.Node
INFO: [localhost]:5701 [dev] [3.12.12] Creating MulticastJoiner

Multicast, by default, accepts communication from all the machines in the multicast group. This may be a security concern and that is why typically, on-premise, multicast communication is firewalled. So, while multicast is good for development work, in production, it is best to use TCP/IP-based discovery.

TCP/IP

Due to the drawbacks stated for Multicast, TCP/IP is the preferred way for communication. In the case of TCP/IP, a member can connect to only known/listed members.

Example

Let’s use TCP/IP for discovery mechanisms. Save the following in hazelcast-tcp.xml

<hazelcast
   xsi:schemaLocation="http://www.hazelcast.com/schema/config
   http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd"
   xmlns="http://www.hazelcast.com/schema/config"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 
   <network>
      <join>
         <multicast enabled="false" />
         <tcp-ip enabled="true">
            <members>localhost</members>
         </tcp-ip>
      </join>
   </network>
</hazelcast>

And then, let’s execute the following command −

java -Dhazelcast.config=hazelcast-tcp.xml -cp .\target\demo-0.0.1-SNAPSHOT.jar
com.example.demo.XMLConfigLoadExample

Output

The output is following −

INFO: [localhost]:5701 [dev] [3.12.12] Creating TcpIpJoiner
Jan 30, 2021 8:09:29 PM
com.hazelcast.spi.impl.operationexecutor.impl.OperationExecutorImpl

The above output shows that a TCP/IP joiner is used to join two members.

And if you execute the following command on two different shells −

java '-Dhazelcast.config=hazelcast-tcp.xml' -cp .\target\demo-0.0.1-SNAPSHOT.jar
com.example.demo.MultiInstanceHazelcastExample

We see the following output −

Members {size:2, ver:2} [
   Member [localhost]:5701 - 62eedeae-2701-4df0-843c-7c3655e16b0f
   Member [localhost]:5702 - 859c1b46-06e6-495a-8565-7320f7738dd1 this
]

The above output means that the nodes were able to join using TCP/IP and both are using localhost as the IP address.

Note that we can specify more IPs or the machine names (which would be resolved by DNS) in the XML configuration file.

<hazelcast
   xsi:schemaLocation="http://www.hazelcast.com/schema/config
   http://www.hazelcast.com/schema/config/hazelcast-config-3.12.12.xsd"
   xmlns="http://www.hazelcast.com/schema/config"
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 
   <network>
      <join>
         <multicast enabled="false" />
         <tcp-ip enabled="true">
            <members>machine1, machine2....</members>
         </tcp-ip>
      </join>
   </network>
</hazelcast>

Leave a Reply