Platform for Analyzing Fast Data
Stream Processing and Batch Processing

jKool cloud - Streaming Analytics
Data streams from your applications to jKoolCloud. Use jKQL (jKool Query language) to ask questions about your data and turn raw machine data into charts, graphs, tables, comparisons, topology and more.

There is a rich set of analytical operators used to analyze your data: aggregation, summaries, comparison and more including: count, min, max, average, top, bottom, best, worst, bucketing, filtering, bollinger bands and more.

jKQL uses the latest BigData and clustered computing technologies such as CEP, DataStax Enterprise (Apache Cassandra) to store and analyze data streams. jKool simulator can be used to generate live streams of events, metrics and transactions without having concrete data sources.

jKool has a unique dual-processing approach for all incoming data streams: stream & history (batch) in one. The jKQL “subscribe” verb allows subscription to real-time data streams with sliding time windows. History is available for all streamed data based on TTL (time-to-live).


How jKool Works

3 simple steps:

1. Stream time series data (time stamped data points) (logs, events, transactions, metrics)
2. Let jKool store, track and analyze your streams
3. “Talk with your data” using jKool Elastic Views + jKQL

No installations, no hardware, no database, no data management, no data schema, no headaches.

Let jKoolCloud do the heavy lifting for you.

jKool-technology_flow

The jKool Cloud Service

visualization

Visualization

jKool’s elastic presentation layer utilizes queries to drive the presentation. Instead of the typical dashboard that you then configure to display data, jKQL queries define what is called Viewlets. Viewlets are individual presentations of queries that ask questions about your data.

compute_grid

Compute Grid

jKool is built on a distributed Big Data Repository, an Elastic Grid and a Query Grid. The underlying NoSQL repository is optimized for time-series data and delivers transparent scale, performance and storage.

query_grid

Query Grid

The elastic Query Grid is used to handle the incoming streams of data, ingest and then digest them. Massive parallel processing is used to enable this data to be usable in near real-time. jKool automatically summarizes, aggregates, counts and computes. In addition it handles replication, encryption and compression providing fast response across large data sets.

dashboard

Dashboard

  • A dashboard is made up of multiple windows called Viewlets, which can be moved, rearranged and resized.
  • Charts, graphs, analysis: all in one elastic pane
jKQL

jKQL

  • An English like query language that is there to let you “Talk with your data”.
  • Easy to learn: just stream, run queries, get insight
  • Query Builder is used to auto complete or build full queries without any knowledge of jKQL
Big-data-events
heatmap


The jKool Data Model

schema-less_model

Schema-less model based on:

  • Tree like structure of activity -> event(s) -> snapshot(s)
  • Each activity/event has a set of defined attributes at given time (snapshot)
  • Each one collectively called a Data Point (DP)
activity_event_model

Activity/Event Model

  • Activity is a collection of related events, activities
  • Event is an atomic occurrence of something in time
  • Snapshot is a set of key=value pairs in time
activity_examples

Activity Examples:

  • Activity is a marketing campaign, events are individual task within a campaign
  • Activity is an order process and events are credit check, shipping, price quote.

Correlation

automated_grouping_events_activities

Automated grouping of events and activities

  • Based on user defined tokens (correlators, such as order ids, tags, other user identifiers)
  • Connect various data points across runtime boundaries into common data sets.
track_your_data_in_motion

Track your data in motion

  • Track morphing and splitting of activities across execution path: order -> payment -> shipping
  • Track geo locations, servers, devices, applications, users
  • Track end-to-end performance for activities, events across applications, servers, locations, users.
  • Annotate, enrich event streams with user defined attributes and correlate them in time

Collectors

Open Source Collectors

Spark

HDFS

  • jKool, using the HDFS collector can natively read and parse logs located in HDFS
Log4j

Logstash

  • Logstash, used to centralize data processing of logs and metrics can send data to jKool for analysis. The Logstash collector will take advantage of existing platforms leveraging existing input and output plugins in order to stream data to jKool.
StreamJMX

Apache Flume

  • Apache Flume, used to centralize data processing of logs and metrics can send data to jKool for analysis. The Apache Flume collector will take advantage of existing platforms leveraging existing input and output plugins in order to stream data to jKool.
StreamJMX

Kafka

  • The Kafka collector reads off queues and streams data to jKool. Application specific data, including events and metrics enrich other data sources as part of application analytics.
TrackingFilter

JMS

  • The JMS collector reads off queues and streams data to jKool. Application specific data, including events and metrics enrich other data sources as part of application analytics.
Spark

Spark (tnt4j-spark)

  • Stream Spark data to jKool for real-time visualization
  • Implements SparkListener for SparkContext
Log4j

Log4j/SLF4J/Logback

  • Consolidate all application logs on a single pane of glass
  • Visualize your logs to help with diagnostics & root cause
  • Search your log entries across applications, servers
  • Annotate your log entries for improved diagnostics
  • Adhoc queries and summaries for all your log entries

View App Log Screenshots

StreamJMX

Syslog (tnt4j-syslogd)

  • Consolidate syslogs from multiple servers on a single pane of glass
  • Visualize the consolidated syslog to help with diagnostics & root cause
  • Search your syslog entries across applications, servers
  • Annotate your syslog entries for improved diagnostics
  • Adhoc queries and summaries for all your syslog entries

View Syslog Screenshot

StreamJMX

StreamJMX (tnt4-stream-jmx)

  • Keep history of JMX metrics across multiple JVM’s
  • Understand JVM capacity, performance
  • Compare how JVM performance changes with changes to your applications
  • Adhoc queries, summaries for all your JMX metrics
TrackingFilter

TrackingFilter (tnt4j-servlet-filter)

  • Measure performance of your JEE Web applications
  • Keep history of response times
  • Track end-user behavior, location
  • Adhoc queries, summaries for your web traffic
Stream Garbage Collection

Stream GC (stream-gc)

  • Stream Garbage Collection from single or multiple JVMs
  • Measure Garbage Collection frequency, duration
  • Minor, Major, Full collections
  • Measure Memory Pool Utilization
  • Before & After GC collection
  • Provides metrics required for GC troubleshooting & tuning
Simulator

Simulator

  • Simulate the data you would see using a collector without deploying one
  • Learn in advance the benefits you will receive
JESL

JESL (Java Event Streaming Library)

  • JESL allows application developers stream time-series data to jKool Cloud.
  • The JESL Simulator provides the ability to generate tracking activities and events.
  • TNT4J Open Source API + jKool Event Simulator

Download Collectors

jKool API

jKool provides development tools for capturing events and streaming them to the jKool service, in the form of:

RESTful API

RESTful API

  • RESTful API Allows you to send any data you need to your jKool repository
  • Java objects will automatically convert to the proper JSON format when POSTed.

Find out more about the jKool RESTful API.

Download API’s

Please join our developer community, contribute and help evolve the open source project.

Get Started Free