Platform for Analyzing Fast Data
Stream Processing and Batch Processing
Data streams from your applications to jKoolCloud. Use jKQL (jKool Query language) to ask questions about your data and turn raw machine data into charts, graphs, tables, comparisons, topology and more.
There is a rich set of analytical operators used to analyze your data: aggregation, summaries, comparison and more including: count, min, max, average, top, bottom, best, worst, bucketing, filtering, bollinger bands and more.
jKQL uses the latest BigData and clustered computing technologies such as CEP, DataStax Enterprise (Apache Cassandra) to store and analyze data streams. jKool simulator can be used to generate live streams of events, metrics and transactions without having concrete data sources.
jKool has a unique dual-processing approach for all incoming data streams: stream & history (batch) in one. The jKQL “subscribe” verb allows subscription to real-time data streams with sliding time windows. History is available for all streamed data based on TTL (time-to-live).
How jKool Works
3 simple steps:
1. Stream time series data (time stamped data points) (logs, events, transactions, metrics)
2. Let jKool store, track and analyze your streams
3. “Talk with your data” using jKool Elastic Views + jKQL
No installations, no hardware, no database, no data management, no data schema, no headaches.
Let jKoolCloud do the heavy lifting for you.
The jKool Cloud Service

Visualization
jKool’s elastic presentation layer utilizes queries to drive the presentation. Instead of the typical dashboard that you then configure to display data, jKQL queries define what is called Viewlets. Viewlets are individual presentations of queries that ask questions about your data.

Compute Grid
jKool is built on a distributed Big Data Repository, an Elastic Grid and a Query Grid. The underlying NoSQL repository is optimized for time-series data and delivers transparent scale, performance and storage.

Query Grid
The elastic Query Grid is used to handle the incoming streams of data, ingest and then digest them. Massive parallel processing is used to enable this data to be usable in near real-time. jKool automatically summarizes, aggregates, counts and computes. In addition it handles replication, encryption and compression providing fast response across large data sets.

Dashboard
- A dashboard is made up of multiple windows called Viewlets, which can be moved, rearranged and resized.
- Charts, graphs, analysis: all in one elastic pane
The jKool Data Model

Schema-less model based on:
- Tree like structure of activity -> event(s) -> snapshot(s)
- Each activity/event has a set of defined attributes at given time (snapshot)
- Each one collectively called a Data Point (DP)

Activity/Event Model
- Activity is a collection of related events, activities
- Event is an atomic occurrence of something in time
- Snapshot is a set of key=value pairs in time

Activity Examples:
- Activity is a marketing campaign, events are individual task within a campaign
- Activity is an order process and events are credit check, shipping, price quote.
Correlation

Automated grouping of events and activities
- Based on user defined tokens (correlators, such as order ids, tags, other user identifiers)
- Connect various data points across runtime boundaries into common data sets.

Track your data in motion
- Track morphing and splitting of activities across execution path: order -> payment -> shipping
- Track geo locations, servers, devices, applications, users
- Track end-to-end performance for activities, events across applications, servers, locations, users.
- Annotate, enrich event streams with user defined attributes and correlate them in time
Open Source Collectors

HDFS
- jKool, using the HDFS collector can natively read and parse logs located in HDFS

Logstash
- Logstash, used to centralize data processing of logs and metrics can send data to jKool for analysis. The Logstash collector will take advantage of existing platforms leveraging existing input and output plugins in order to stream data to jKool.

Apache Flume
- Apache Flume, used to centralize data processing of logs and metrics can send data to jKool for analysis. The Apache Flume collector will take advantage of existing platforms leveraging existing input and output plugins in order to stream data to jKool.

Kafka
- The Kafka collector reads off queues and streams data to jKool. Application specific data, including events and metrics enrich other data sources as part of application analytics.

JMS
- The JMS collector reads off queues and streams data to jKool. Application specific data, including events and metrics enrich other data sources as part of application analytics.

Spark (tnt4j-spark)
- Stream Spark data to jKool for real-time visualization
- Implements SparkListener for SparkContext

Log4j/SLF4J/Logback
- Consolidate all application logs on a single pane of glass
- Visualize your logs to help with diagnostics & root cause
- Search your log entries across applications, servers
- Annotate your log entries for improved diagnostics
- Adhoc queries and summaries for all your log entries

Syslog (tnt4j-syslogd)
- Consolidate syslogs from multiple servers on a single pane of glass
- Visualize the consolidated syslog to help with diagnostics & root cause
- Search your syslog entries across applications, servers
- Annotate your syslog entries for improved diagnostics
- Adhoc queries and summaries for all your syslog entries

StreamJMX (tnt4j-stream-jmx)
- Keep history of JMX metrics across multiple JVM’s
- Understand JVM capacity, performance
- Compare how JVM performance changes with changes to your applications
- Adhoc queries, summaries for all your JMX metrics

TrackingFilter (tnt4j-servlet-filter)
- Measure performance of your JEE Web applications
- Keep history of response times
- Track end-user behavior, location
- Adhoc queries, summaries for your web traffic

Stream GC (stream-gc)
- Stream Garbage Collection from single or multiple JVMs
- Measure Garbage Collection frequency, duration
- Minor, Major, Full collections
- Measure Memory Pool Utilization
- Before & After GC collection
- Provides metrics required for GC troubleshooting & tuning

Simulator
- Simulate the data you would see using a collector without deploying one
- Learn in advance the benefits you will receive

JESL (Java Event Streaming Library)
- JESL allows application developers to stream time-series data to jKool Cloud.
- The JESL Simulator provides the ability to generate tracking activities and events.
- TNT4J Open Source API + jKool Event Simulator
jKool API
jKool provides development tools for capturing events and streaming them to the jKool service, in the form of:

RESTful API
- RESTful API Allows you to send any data you need to your jKool repository
- Java objects will automatically convert to the proper JSON format when POSTed.
Please join our developer community, contribute and help evolve the open source project.