Streaming Ingestion (SI)
To use data, a system needs to be able to discover, integrate, and ingest all available data from the machines that produce it, as fast as it’s being produced, in any format, and at any quality.
A streaming data ingestion framework doesn’t simply move data from source to destination like traditional ETL solutions. With SI, all data formats including log files, web logs, sensor or IoT data, machine data, and CDC from databases, may be ingested, filtered, corrected, parsed, and simultaneously enriched with structured, stored data, from any or many sources for analysis.
Streaming Analytics
In order to provide accurate and relevant business intelligence, a system needs to be able to analyze all data available continuously, concurrently, and in real time.
Streaming analytics differs from batch processing in that results are updated continuously as more data enters the system. This record-by-record model is more accurate and relevant because it analyzes more data, faster.
Streaming Action
The analytics results can trigger the correct business process or operation on a per-record basis at the point of insight.
Unlike traditional models, streaming technologies employ a continuous load process. That allows the systems to mitigate anomalies, report results, and seize opportunities when and where they happen. Automating actions (whether pushing data into streaming applications, or into storage for future use) also means avoiding systems overload.