Streaming Join State Growth Simulation Dataset
#State Prediction #State Size Estimation #Memory Pressure Prediction #Output Rate Estimation #Stream Processing System Design #Streaming Join Operator Tuning #Memory Resource Planning #Stream Processing Performance Analysis
- Records
- 4460
- Size
- 0.3GB
- Format
- JSON
- Provider
MOBIUSI INC- Updated
- v1.02026-11-04
AI Analysis & Value Prop
This dataset simulates streaming join behavior using parameters such as the arrival rates of two input streams, join-key cardinality, time-window length, out-of-order degree, match rate, and state cleanup policy. Each record combines scenario inputs with estimates of state records, pending unmatched records, memory usage, memory pressure, cleanup rate, and output rate. It supports state modeling, memory resource assessment, operator tuning, and stream processing system design.
Overview
Dataset Insights
Preview
Sample Examples
sample.json
{}
Schema
Technical Specifications
| Field | Type | Description |
|---|---|---|
| file_name | string | File name |
| match_rate | number | Proportion of records from the two streams that successfully match within the join window, from 0 to 1. |
| key_cardinality | integer | Number of distinct keys participating in the join during the simulation period. |
| total_arrival_rate | number | Combined record arrival rate of Stream A and Stream B, in records per second. |
| out_of_order_degree | number | Degree of input record disorder, from 0 to 1; higher values indicate more pronounced disorder. |
| time_window_seconds | number | Length of the time window used to retain and match records in the streaming join, in seconds. |
| state_cleanup_policy | string | Policy used by the join operator to remove expired state or state for records whose matching is complete. |
| estimated_output_rate | number | Estimated rate of matched output records produced by the join operator, in records per second. |
| memory_pressure_ratio | number | State memory usage relative to available memory capacity or a configured memory budget. |
| stream_a_arrival_rate | number | Number of records arriving from Stream A per second. |
| stream_b_arrival_rate | number | Number of records arriving from Stream B per second. |
| estimated_memory_usage_mb | number | Estimated memory used by the join operator based on the size of its state, in megabytes. |
| simulation_duration_seconds | number | Duration of the streaming join simulation, in seconds. |
| estimated_state_size_records | number | Estimated number of state records that the join operator must maintain during the simulation. |
| estimated_cleanup_release_rate | number | Estimated number of expired or fully matched records released by the state cleanup policy per second. |
| effective_state_retention_seconds | number | Estimated effective state retention duration after accounting for the time window, out-of-order degree, and cleanup policy, in seconds. |
| estimated_pending_unmatched_records | number | Estimated number of records that have not yet matched under the current window and match rate. |
Trust
Compliance Statement
Authorization Type
Proprietary - Commercial AI Training License (No Redistribution)
Commercial Use
Requires exclusive subscription or authorization contract (monthly or per-invocation charging)
Privacy and Anonymization
No PII, no real company names, simulated scenarios follow industry standards
Compliance System
Compliant with China's Data Security Law / EU GDPR / supports enterprise data access logs
FAQ
Frequently Asked Questions
- What does the dataset contain?
- It describes simulated streaming join scenarios through input parameters and estimated outcomes. Inputs include two-stream arrival rates, key cardinality, window length, out-of-order degree, match rate, and cleanup policy. Estimated metrics include state records, pending unmatched records, memory usage, cleanup rate, and output rate.
- Are these measurements from production systems?
- The data is intended for simulation learning and scientific modeling. Its fields represent simulated scenario inputs and estimates, so they should not be assumed to be measurements from production systems. Validate or calibrate the data against actual workloads before using it to guide deployment decisions.
- Which modeling and evaluation tasks does it support?
- It can support state prediction, state size estimation, memory pressure prediction, and output rate estimation for stateful streaming joins. It can also help compare operator behavior under different windows, workloads, and cleanup policies.
- How can I access the data, and what are the licensing terms?
- Refer to the dataset product page for available download methods, formats, and license terms. Confirm the applicable permissions and restrictions before using the data in commercial projects or redistributing it.
Can't find the data you need?
Post a request and let data providers reach out to you.
Get this Dataset
High-quality data, ready for your next AI breakthrough.
Cite this Work
@dataset{Mobiusi2026,
title={Streaming Join State Growth Simulation Dataset},
author={MOBIUSI INC},
year={2026},
url={https://www.mobiusi.com/datasets/7a7fb01b51f7b60be4a1270b7f95956d?dataset_task_cate_id=11},
urldate={2026-10-27},
keywords={streaming join simulation data,stream processing state prediction,join state growth,stream operator memory estimation,two-stream join modeling,state cleanup policy,stream processing performance evaluation,pending unmatched record prediction},
version={1.0}
}Using this in research? Please cite us.
placeholder
placeholder
placeholder
placeholder
placeholder
placeholder
placeholder