Hazelcast – IMap

The java.util.concurrent.The map provides an interface that supports storing key-value pairs in a single JVM. While java.util.concurrent.ConcurrentMap extends this to support thread safety in a single JVM with multiple threads.

Similarly, IMap extends the ConcurrentHashMap and provides an interface that makes the map thread-safe across JVMs. It provides similar functions: put, get, etc.

The IMap supports synchronous backup as well as asynchronous backup. Synchronous backup ensures that even if the JVM holding the queue goes down, all elements would be preserved and available from the backup.

Let’s look at an example of the useful functions.

Creation & Read/Write

Adding elements and reading elements. Let’s execute the following code on two JVMs. The producer code on one and one consumer code on the other.

Example

The first piece is the producer code which creates a map and adds items to it.

public static void main(String... args) throws IOException, InterruptedException {
   //initialize hazelcast instance
   HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
   // create a map
   IMap<String, String> hzStock = hazelcast.getMap("stock");
   hzStock.put("Mango", "4");
   hzStock.put("Apple", "1");
   hzStock.put("Banana", "7");
   hzStock.put("Watermelon", "10");
   Thread.sleep(5000);
   System.exit(0);
}

The second piece is of consumer code which reads the elements.

public static void main(String... args) throws IOException, InterruptedException {
   //initialize hazelcast instance
   HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
   // create a map
   IMap<String, String> hzStock = hazelcast.getMap("stock");
   for(Map.Entry<String, String> entry: hzStock.entrySet()){
      System.out.println(entry.getKey() + ":" + entry.getValue());
   }
   Thread.sleep(5000);
   System.exit(0);
}

Output

The output for the code for the consumer −

Mango:4
Apple:1
Banana:7
Watermelon:10

Useful Methods

Sr.NoFunction Name & Description
1put(K key, V value) Add an element to the map
2remove(K key) Remove an element from the map
3keySet() Return a copy of all the keys in the map
4localKeySet() Return a copy of all keys which are present in the local partition
5values() Return a copy of all the values in the map
6size() Return the count of elements in the map
7containsKey(K key) Return true if the key is present
8executeOnEnteries(EntryProcessor processor) Applies the processor on all the map’s keys and returns the output of this application. We will look at an example for the same in the upcoming section.
9addEntryListener(EntryListener listener, value) Notifies the subscriber of an element being removed/added/modified in the map.
10addLocalEntryListener(EntryListener listener, value) Notifies the subscriber of an element being removed/added/modified in the local partitions

Eviction

By default, keys in Hazelcast stay indefinitely in the IMap. If we have a very large set of keys, then we need to ensure that the keys which are heavily used are stored in the IMap as compared to the ones which are used less often, in order to have better performance and efficient memory usage.

For this purpose, one can manually delete keys via remove()/evict() functions for the keys which are not used that often. However, Hazelcast also provides automatic eviction of keys based on various eviction algorithms.

This policy can be set by XML or programmatically. Let’s look at an example for the same −

<map name="stock">
   <max-size policy="FREE_HEAP_PERCENTAGE">30</max-size>
   <eviction-policy>LFU</eviction-policy>
</map>

There are two attributes in the above configuration.

  • Max-size − Policy which is used to communicate to Hazelcast the limit at which we claim that max size of the map “stock” has reached.
  • Eviction-policy − Once the above max-size policy is hit, what algorithm to use to remove/evict the key.

Here are some of the useful max_size policies.

Sr.NoMax Size Policy & Description
1PER_NODE Max number of entries per JVM for the map which is the default policy.
2FREE_HEAP Minimum free heap memory to be kept aside (in MBytes) in the JVM
3FREE_HEAP_PERCENTAGE Minimum free heap memory to be kept aside (in percent) in the JVM
4take() Return the head of the queue or wait till the element becomes available
5USED_HEAP Maximum allowed heap memory used in the JVM (in MBytes)
6USED_HEAP_PERCENTAGE Maximum allowed heap memory used in the JVM (in percent)

Here are some of the useful eviction policies −

Sr.NoEviction Policy & Description
1NONE No eviction will be made which is the default policy
2LFU Least frequently-used would be evicted
3LRU Least recently used key would be evicted

Another useful parameter for eviction is also time-to-live-seconds, i.e., TTL. With this, we can ask Hazelcast to remove any key which is older than X seconds. This ensures that we are proactive in removing older keys before the max-size policy is hit.

Partitioned data and High Availability

One important point to note about IMap is that, unlike other collections, the data is partitioned across JVMs. All the data doesn’t need to be stored/present on a single JVM. Complete data is still accessible to all JVMs. This gives Hazelcast a way to scale linearly across available JVMs and not be constrained by the memory of a single JVM.

The IMap instances are divided into multiple partitions. By default, the map is divided into 271 partitions. And these partitions are distributed across Hazelcast members available. Each entry in which is added to the map is stored in a single partition.

Let’s execute this code on 2 JVMs.

public static void main(String... args) throws IOException, InterruptedException {
   //initialize hazelcast instance
   HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
   // create a map
   IMap<String, String> hzStock = hazelcast.getMap("stock");
   hzStock.put("Mango", "4");
   hzStock.put("Apple", "1");
   hzStock.put("Banana", "7");
   hzStock.put("Watermelon", "10");
   Thread.sleep(5000);
   // print the keys which are local to these instance
   hzStock.localKeySet().forEach(System.out::println);
   System.exit(0);
}

Output

As seen in the following output, the consumer 1 prints its own partition which contains 2 keys −

Mango
Watermelon

Consumer 2 owns the partition which has the other 2 keys −

Banana
Apple

By default, IMap has one synchronous backup, which means that even if one node/member goes down, the data would not get lost. There are two types of back up.

  • Synchronous − The map.put(key, value) would not succeed till the key is also backed up on another node/member. Sync backups are blocking and thus impact the performance of the put call.
  • Async − The backup of the stored key is performed eventually. Async backups are non-blocking and fast but they do not guarantee existence of the data if a member were to go down.

The value can be configured using XML configuration. For example, let’s do it for out stock map −

<map name="stock">
   <backup-count>1</backup-count>
   <async-backup-count>1<async-backup-count>
</map>

Hashcode and Equals

Example

In Java-based HashMap, key comparison happens by checking equality of the hashCode() and equals() method. For example, a vehicle may have serialId and the model to keep it simple.

public class Vehicle implements Serializable{
   private static final long serialVersionUID = 1L;
   private int serialId;
   private String model;
 
   public Vehicle(int serialId, String model) {
      super();
      this.serialId = serialId;
      this.model = model;
   }
   public int getId() {
      return serialId;
   }
   public String getModel() {
      return model;
   }
   @Override
   public int hashCode() {
      final int prime = 31;
      int result = 1;
      result = prime * result + serialId;
      return result;
   }
   @Override
   public boolean equals(Object obj) {
      if (this == obj)
         return true;
      if (obj == null)
         return false;
      if (getClass() != obj.getClass())
         return false;
      Vehicle other = (Vehicle) obj;
      if (serialId != other.serialId)
         return false;
         return true;
   }
}

When we try using the above class as the key for HashMap and IMap, we see the difference in comparison.

public static void main(String... args) throws IOException, InterruptedException {
   // create a Java based hash map
   Map<Vehicle, String> vehicleOwner = new HashMap<>();
   Vehicle v1 = new Vehicle(123, "Honda");
   vehicleOwner.put(v1, "John");
        
   Vehicle v2 = new Vehicle(123, null);
   System.out.println(vehicleOwner.containsKey(v2));
   
   // create a hazelcast map
   HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
   IMap<Vehicle, String> hzVehicleOwner = hazelcast.getMap("owner");
   hzVehicleOwner.put(v1, "John");
   System.out.println(hzVehicleOwner.containsKey(v2));
   System.exit(0);
}

Now, why does Hazelcast give the answer as false?

Hazelcast serializes the key and stores it as a byte array in binary format. As these keys are serialized, the comparison cannot be made based on equals() and hashcode().

Serializing and Deserializing are required in case of Hazelcast because the function get(), containsKey(), etc. may be invoked on the node which does not own the key, so remote call is required.

Serializing and Deserializng are expensive operations and so, instead of using equals() method, Hazelcast compares byte arrays.

What this means is that all the attributes of the Vehicle class should match not just id. So, let’s execute the following code −

Example

public static void main(String... args) throws IOException, InterruptedException {
   Vehicle v1 = new Vehicle(123, "Honda");
   // create a hazelcast map
   HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
   IMap<Vehicle, String> hzVehicleOwner = hazelcast.getMap("owner");
   Vehicle v3 = new Vehicle(123, "Honda");
   System.out.println(hzVehicleOwner.containsKey(v3));
   System.exit(0);
}

Output

The output of the above code is −

true

This output means all the attributes of Vehicle should match for equality.

EntryProcessor

EntryProcessor is a construct which supports sending of code to the data instead of bringing data to the code. It supports serializing, transferring, and the execution of function on the node which owns the IMap keys instead of bringing in the data to the node which initiates the execution of the function.

Example

Let’s understand this with an example. Let’s say we create an IMap of Vehicle -> Owner. And now, we want to store lowercase for the owner. So, how do we do that?

public static void main(String... args) throws IOException, InterruptedException {
   // create a hazelcast map
   HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
   IMap<Vehicle, String> hzVehicleOwner = hazelcast.getMap("owner");
   hzVehicleOwner.put(new Vehicle(123, "Honda"), "John");
   hzVehicleOwner.put(new Vehicle(23, "Hyundai"), "Betty");
   hzVehicleOwner.put(new Vehicle(103, "Mercedes"), "Jane");
   for(Map.Entry<Vehicle, String> entry:  hzVehicleOwner.entrySet())
   hzVehicleOwner.put(entry.getKey(), entry.getValue().toLowerCase());
   for(Map.Entry<Vehicle, String> entry: hzVehicleOwner.entrySet())
   System.out.println(entry.getValue());
   System.exit(0);
}

Output

The output of the above code is −

john
jane
betty

While this code seems simple, it has a major drawback in terms of scale if there are high number of keys −

  • Processing would happen on the single/caller node instead of being distributed across nodes.
  • More time as well as memory would be needed to get the key information on the caller node.

That is where the EntryProcessor helps. We send the function of converting to lowercase to each node which holds the key. This makes the processing parallel and keeps the memory requirements in check.

Example

public static void main(String... args) throws IOException, InterruptedException {
   // create a hazelcast map
   HazelcastInstance hazelcast = Hazelcast.newHazelcastInstance();
   IMap<Vehicle, String> hzVehicleOwner = hazelcast.getMap("owner");
   hzVehicleOwner.put(new Vehicle(123, "Honda"), "John");
   hzVehicleOwner.put(new Vehicle(23, "Hyundai"), "Betty");
   hzVehicleOwner.put(new Vehicle(103, "Mercedes"), "Jane");
   hzVehicleOwner.executeOnEntries(new OwnerToLowerCaseEntryProcessor());
   for(Map.Entry<Vehicle, String> entry: hzVehicleOwner.entrySet())
   System.out.println(entry.getValue());
   System.exit(0);
}
static class OwnerToLowerCaseEntryProcessor extends
AbstractEntryProcessor<Vehicle, String> {
   @Override
   public Object process(Map.Entry<Vehicle, String> entry) {
      String ownerName = entry.getValue();
      entry.setValue(ownerName.toLowerCase());
      return null;
   }
}

Output

The output of the above code is −

john
jane
betty

Leave a Reply