How LoadGen's No-Code Workload Engine Drives the Open Benchmark for User Experience

OBUX aims to create a transparent, robust, and vendor-neutral benchmark for evaluating digital workspaces. This initiative focuses on several key areas:

System performance (CPU, disk, memory, GPU)

User experience (responsiveness, satisfaction, frustration)

Environmental impact (e.g., CO₂ consumption)

These dimensions are synthesized into a composite score that reflects not just the system's capabilities but also the user's perceptual experience. This dual focus enables IT leaders, architects, and engineers to make well-rounded decisions based on both technical data and human experience.

To achieve precise and reproducible benchmark results, realistic, consistent, and scalable user simulation is essential. This is where LoadGen excels.

With LoadGen’s no-code workload engine, OBUX can effortlessly simulate typical user activities. These activities include:

This entire process is configured through LoadGen’s intuitive visual interface, requiring no scripting and making it accessible to engineers, consultants, and QA teams alike.

Each workload consists of modular transactions-the fundamental units of user activity. These transactions are orchestrated to form user scenarios that can be executed across multiple virtual users and environments. Transactions may cover:

Each action is timed, logged, and scored, allowing for a comprehensive assessment of responsiveness, stability, and consistency.

The OBUX scoring system blends objective system metrics with subjective user perception. This balanced approach produces a more comprehensive understanding of performance.

System Score (Absolute):

Based on hardware and OS-level performance metrics.

Includes CPU load, memory utilization, disk latency, and system response times.

User Experience Score (Relative):

Advances perceived UX on a scale from 0.00 (unacceptable) to 0.94 (perfect).

Anchored to human-centric categories:

This scoring formula ensures that both technical efficiency and subjective quality contribute to the final benchmark results.

Every benchmark generated by LoadGen is composed of multiple sessions with repeated transactions. These transactions are the core data points that inform OBUX metrics.

Each transaction is:

Timestamped and compared to expected performance thresholds.

Measured for duration, jitter, and resource utilization.

Linked to environmental impact (e.g., energy consumed → CO₂ equivalent).

These metrics provide vital insights, allowing OBUX to correlate system strain, user perception, and environmental footprint in a unified benchmark result.

LoadGen is integrated into OBUX as an open and transparent workload generator, ensuring the entire testing flow is reproducible and verifiable:

Benchmark definitions are published openly for scrutiny.

Metrics are stored in InfluxDB.

Visualizations are crafted using Grafana.

Community contributions are encouraged on GitHub.

Additionally, the LoadGen engine supports plugin-based extensions, enabling contributors to introduce custom applications, edge cases, or vertical-specific workloads, such as healthcare EMRs, financial dashboards, or CAD software.

With LoadGen leading workload simulation and OBUX establishing the structure and scoring logic, the first public version of the benchmark is set to launch at the end of 2024. The target venues for this launch include EUCworld and E2EVC.

Here are some avenues we are exploring:

Offering free and discounted LoadGen licenses for OBUX contributors.

Introducing pre-built workload packs for popular use cases.

Enhancing real-time feedback utilizing AI/LLM-assisted UX interpretation.

The collaboration between LoadGen and OBUX signifies a powerful synergy where precision engineering meets open benchmarking. Together, we empower IT professionals to look beyond hardware metrics and focus on the real-world user experience.

Interested in being a part of this groundbreaking evolution?

Visit obux.it Explore LoadGen