{"id":11130,"date":"2018-05-16T09:37:23","date_gmt":"2018-05-16T09:37:23","guid":{"rendered":"http:\/\/stblog.lunaeme.com\/?p=11130"},"modified":"2023-09-20T12:56:22","modified_gmt":"2023-09-20T12:56:22","slug":"apache-ignite-cache","status":"publish","type":"post","link":"https:\/\/www.stratio.com\/blog\/apache-ignite-cache\/","title":{"rendered":"Apache Ignite: More than a simple cache"},"content":{"rendered":"

Let us suppose that we start to develop a webserver for our IOT App with a few endpoints, like POST for receive events, GET devicesBySensorType, GET all, and PUT for update device metadata, etc.<\/p>\n

<\/p>\n

At first, a cache for common data could seem like a secondary issue, but if we start to think long term and if we want to improve performance and\/or to decrease response time (e.g when a service retrieve data from a database), we realize that a cache is a mandatory requirement.<\/p>\n

An initial solution could be to use the always reliable HashMap class (or ConcurrentHashMap after the first ConcurrentModificationException): this is a first attempt that, by caching objects in different maps in memory, may give us quick benefits. But it is not enough.<\/p>\n

Hello Data Grid<\/h2>\n

With Ignite<\/strong>, this is simple, only add the dependency on your scala\/java project:<\/p>\n

> sbt\n\n       \"org.apache.ignite\" % \"ignite-core\" % \"2.4.0\"\n\n> maven\n\n    <dependency>\n          <groupId>org.apache.ignite<\/groupId>\n          <artifactId>ignite-core<\/artifactId>\n          <version>2.4.0<\/version>\n   <\/dependency><\/pre>\n
Scala code for the example:<\/span><\/p>\n
object IgniteSimpleDataGrid extends App {\n\n     val igniteConfig = new IgniteConfiguration()\n     val cacheConfig = new CacheConfiguration(\"ignite\")\n     igniteConfig.setCacheConfiguration(cacheConfig)\n     val ignite = Ignition.start(igniteConfig)\n     val cache: IgniteCache[Int, String] = ignite.getOrCreateCache[Int, String](\"ignite\")\n     for (i <- 1 to 10) {\n       cache.put(i, s\"value-$i\")\n     }\n     println(s\"From cache 7 -> ${cache.get(7)}\")\n     println(s\"Not exists 20 -> ${cache.get(20)}\")\n}<\/pre>\n
Run the class and see the output
\n(Note: ONLY one running Ignite instance per JVM)<\/strong><\/p>\n
\"Screen<\/p>\n
Perfect, we have our first single version of Ignite Data Grid, but what if we need HA and to deploy another instance of the app? Or it may sound familiar a common situation where that metadata endpoint is related to another service and uses the same information so are in need of sharing data between the apps and, sadly, the maps cache schema solution does not work anymore.<\/p>\n
What if I tell you that with in-memory data grid<\/strong> data could be distributed and every node in the cluster (forming a ring topology) can access and maintain the shared information, besides computing remote functions or custom predicates? Would you believe that this is possible? The answer is absolutely yes<\/strong> and here is where Ignite stands out among other solutions.<\/p>\n
Let\u2019s test IgniteSimpleDataGrid <\/em>class with two instances(nodes) : node1 is already running so before we can start the app again (node2), we need to comment the for cycle (block in the code snippet), because node1 has already populated the cache with values and this means that node2 will only be reading from cache.<\/p>\n
\/* for (i <- 1 to 10) {\n    cache.put(i, s\"value-$i\")\n    } *\/\n   println(s\"From cache 7 -> ${cache.get(7)}\")\n   println(s\"Not exists 20 -> ${cache.get(20)}\")<\/pre>\n
Pay attention to the output in node2: the console prints almost the same data as node1 in the first example (look at the id, it is different).<\/p>\n
\"Screen<\/p>\n
And now, look at the output in node1:<\/p>\n
\"Screen<\/p>\n
Both nodes print this line:<\/p>\n
\"Screen<\/p>\n
Great!! That means that we have our Ignite in-memory distributed key-value store cluster up and running!! (more than a simple cache, and we see why).
\nNodes have been self-discovered and all of them see the same data. If we start more nodes, the topology will keep on growing up, each node will be assigned with a unique id and the server number will increase ( e.g. servers =3, servers=4 , etc..)<\/p>\n
\"Screen<\/p>\n
If we stop node1, topology decreases and the rest of the running nodes print this change, because they are listening to topology change events.<\/p>\n
\"Screen<\/p>\n
Ignite implements high availability, if a node goes down, the data is then re-balanced around the cluster without any user operation involved.<\/p>\n
Let us clarify this point.<\/p>\n
My definition of Ignite is that it’s a distributed in-memory cache, query and processing platform for working with large-scale data sets in real-time (leaving aside, streaming processing, Spark integration, Machine learning grid, Ignite FileSystem, persistence, transactions…)<\/strong><\/p><\/blockquote>\n
How can Ignite automagically<\/strong> create a cluster? Well, it provides TcpDiscoverySpi as a default implementation of DiscoverySpi that uses TCP\/IP for node discovery. Discovery SPI can be configured for Multicast- and Static IP-based node discovery. (Spi = especially Ip Finder)<\/p>\n
It means that nodes use multicast to find each other:<\/p>\n
val igniteConfig = new IgniteConfiguration()\nval spi = new TcpDiscoverySpi()\nval ipFinder = new TcpDiscoveryMulticastIpFinder()\nipFinder.setMulticastGroup(\"228.10.10.157\") \/\/ Default value = DFLT_MCAST_GROUP = \"228.1.2.4\";\nspi.setIpFinder(ipFinder)\ncfg.setDiscoverySpi(spi)\nval ignite = Ignition.start(igniteConfig)\nval cache: IgniteCache[Int, String] = ignite.getOrCreateCache[Int, String](\"ignite\")<\/pre>\n
Ignite Ring Topology<\/span><\/h2>\n
\"Screen<\/p>\n
In the image the nodes form a ring of server<\/strong> nodes but, inside an Ignite cluster, nodes can either have roles and belong to a cluster group or they could be client<\/strong> nodes (this means “outside” of the ring and without the possibility of maintaining cached data. This may prove useful for external applications.<\/p>\n
Nodes can broadcast messages to other nodes in a particular group:<\/p>\n
val igniteCluster = ignite.cluster()\n\/\/Send to cluster remotes nodes (ignoring this) this Code\nignite.compute(igniteCluster.forRemotes()).broadcast(\n      new IgniteRunnable {\n          override def run(): Unit = println(s\"Hello node ${igniteCluster.localNode()},this message had been send by igniteCompute broadcast\")\n       }\n)<\/pre>\n
As this example shows, when a node starts, it sends a message to the cluster group forRemotes<\/strong> and to all other nodes (except from itself) that have been configured in mode=server<\/strong>. You might see an exception on the first node at the beginning when the cluster has just been created but don’t worry because it is normal.<\/p>\n
Furthermore it is possible to define nodes with your custom attributes:<\/p>\n
igniteConfig.setUserAttributes(Map[String,Any](\"ROLE\" -> \"MASTER\").asJava)\nignite.compute(igniteCluster.forAttribute(\"ROLE\",\"WORKER\")).broadcast(new IgniteRunnable {\n        override def run(): Unit = println(s\"Hello worker node ${igniteCluster.localNode()},\" +s\"this message had been send by ignite master node\")\n})<\/pre>\n
Here the node has attribute ROLE = MASTER<\/strong>, and broadcast only to nodes with ROLE = WORKER<\/strong><\/span><\/p>\n
Nodes with:<\/span><\/p>\n
igniteConfig.setUserAttributes(Map[String,Any](\"ROLE\" -> \"WORKER\").asJava)<\/pre>\n
will receive the message:<\/span><\/p>\n
As long as your cluster is alive, Ignite will guarantee that the data between different cluster nodes will always remain consistent regardless of crashes or topology changes.<\/strong><\/p><\/blockquote>\n
So far so good: we have seen nodes, topology, cluster groups, broadcast and now let us look forward to one of the best feature of Ignite (PS: durable memory, persistence, spark RDD, streaming, transactions will be tackled in the next posts XD): SQL Queries<\/strong>!!
\nDoes it sounds crazy? Yes, SQLQueries! Over your data!
\nIgnite is fully ANSI-99 compliant and it supports Text queries and Predicate-based Scan Queries.<\/p>\n
\"Screen<\/p>\n
Cache Queries<\/span><\/h2>\n
Before starting to play with code, a few considerations:<\/p>\n