<?xml version="1.0" encoding="UTF-8"?>
<article xmlns:xlink="http://www.w3.org/1999/xlink">
<front><article-meta>
<title-group><article-title>Uncertainty Aware Heuristic Scheduling for Resource Constrained Distributed Data Processing</article-title></title-group>
<contrib-group>
<contrib contrib-type="author"><name><surname>Park</surname><given-names>Min-Chul</given-names></name><xref ref-type="aff" rid="aff1"/></contrib>
<contrib contrib-type="author"><name><surname>Kim</surname><given-names>Seo-Yeon</given-names></name><xref ref-type="aff" rid="aff1"/></contrib>
<contrib contrib-type="author"><name><surname>Conti</surname><given-names>Alessandro</given-names></name><xref ref-type="aff" rid="aff2"/></contrib>
<contrib contrib-type="author"><name><surname>Hassan</surname><given-names>Samar Q.</given-names></name><xref ref-type="aff" rid="aff3"/></contrib>
<contrib contrib-type="author"><name><surname>De Wit</surname><given-names>Pieter J.</given-names></name><xref ref-type="aff" rid="aff4"/></contrib>
<aff id="aff1">Department of Mechanical Design Engineering, Korea Advanced Institute of Engineering Design (KAIED), Daejeon, Republic of Korea.</aff>
<aff id="aff2">Dipartimento di Meccanica, Politecnico di Milano, Milano, Italy.</aff>
<aff id="aff3">Department of Industrial Engineering, Jordan University of Science and Technology, Irbid, Jordan.</aff>
<aff id="aff4">Department of Mechanical Engineering, KU Leuven, Leuven, Belgium.</aff>
</contrib-group>
<abstract><p>Distributed task scheduling under variable durations requires distinguishing local risk-aware placement from a workflow-level deadline guarantee. We formulate a chance-constrained scheduling framework and examine reported results for CHARM, a restricted quantile-guided heuristic on 100 random layered Directed Acyclic Graphs (DAGs) of 30–60 tasks and an eight-worker heterogeneous cluster. Worker calibration traces inform per-task placement and a bounded local repair pass. At log-normal noise σ = 0.3, mean realized makespan is 9.80 time units versus 9.94 for a Heterogeneous Earliest Finish Time (HEFT) assignment evaluated under common dispatch semantics, a 1.4% reduction, with nominal makespan 8.07 versus 7.82. The fraction exceeding 1.1 times each method&apos;s own nominal makespan is 0.797 versus 0.891; this method-dependent threshold is a plan-relative measure rather than a common external deadline. Removing the local chance gate raises realized makespan by 0.13 and the plan-relative exceedance fraction by 0.029. Removing local repair leaves the displayed realized mean unchanged. The experiment does not certify the workflow chance constraint, evaluate runtime rescheduling, or test production traces. Reported statistical results and duration conventions require raw-outcome verification, including an inconsistency in the miss-probability signed-rank summary.</p></abstract>
<kwd-group kwd-group-type="author"><kwd>Task scheduling</kwd><kwd>Uncertainty</kwd><kwd>Chance constraints</kwd><kwd>Heterogeneous computing</kwd><kwd>Makespan robustness</kwd><kwd>Heterogeneous earliest finish time</kwd></kwd-group>
</article-meta></front>
<body>
<sec id="sec-1"><label>1</label><title>Introduction</title>
<p>Every evening a national weather service must complete an ensemble data-assimilation and forecasting pipeline before the bulletin deadline, and every night a bioinformatics centre regenerates variant-calling pipelines whose results steer the next day&apos;s experiments. These workloads are expressed as Directed Acyclic Graphs (DAGs) of tasks with data dependencies and are executed on shared clusters whose effective capacity is far smaller than the sum of their parts. Production experience with systems such as Pegasus shows how variable scientific workloads are in practice <xref ref-type="bibr" rid="ref-1">[1]</xref>, and characterisation studies report task durations spanning orders of magnitude, with stragglers common rather than exceptional <xref ref-type="bibr" rid="ref-2">[2]</xref>. When a deadline-bearing workflow lands on a contended cluster, the scheduler&apos;s treatment of that variability largely decides whether the deadline is met.</p>
<p>The Heterogeneous Earliest Finish Time (HEFT) heuristic ranks tasks and assigns them using point processing-time estimates <xref ref-type="bibr" rid="ref-3">[3]</xref>. Production cluster analysis demonstrates that heterogeneity and time variation affect machine performance <xref ref-type="bibr" rid="ref-4">[4]</xref>. Workflow managers and scheduling models supply the wider setting <xref ref-type="bibr" rid="ref-1">[1]</xref>, <xref ref-type="bibr" rid="ref-2">[2]</xref>. The present simulation abstracts those systems into a fixed heterogeneous cluster and omits communication and contention.</p>
<p>Precedence-constrained scheduling is computationally difficult even for deterministic processing times <xref ref-type="bibr" rid="ref-5">[5]</xref>, and stochastic models introduce decisions about distributions as well as assignment <xref ref-type="bibr" rid="ref-6">[6]</xref>. Chance constraints state a probability requirement, while robust optimization protects against a specified uncertainty set. The present study examines a tractable risk-aware heuristic; it does not establish that existing scheduling research lacks explicit uncertainty treatment.</p>
<p>The contribution is a framework that separates distribution estimation, schedule construction, and risk assessment, together with reported evidence for a simpler worker-quantile implementation. This separation makes clear which design features are proposed and which are evaluated.</p>
<p>The formal model expresses workflow deadline risk as a chance constraint. A local 90% task-placement gate is examined experimentally but is not equivalent to that joint workflow requirement.</p>
<p>Worker calibration traces inform the implemented placement rule. Per-class Bayesian shrinkage and transfer-distribution estimation are general extensions, not tested components.</p>
<p>A bounded local repair pass is evaluated during schedule construction. Runtime monitoring and posterior-based rescheduling remain conceptual extensions.</p>
<p>The analysis states the conditions required for probability certification and the cost of estimating risk; no unconditional reliability certificate is claimed.</p>
<p>Reported experiments compare CHARM, random assignment, EFT list scheduling, HEFT-based assignment, and static greedy on 100 synthetic DAGs under three noise levels.</p>
<p>The rest of the paper is arranged in the following order. Section 2 reviews deterministic and stochastic scheduling research together with the workflow systems that motivate the problem, and Section 3 introduces the model, the research questions, and the chance-constrained formulation. Section 4 presents the CHARM framework and Algorithm 1. Section 5 reports the executed experimental campaign and ends with an epistemic status note. Section 6 examines limitations and alternative positions, and Section 7 concludes with empirical answers and a roadmap.</p>
</sec>
<sec id="sec-2"><label>2</label><title>Related Work</title>
<sec id="sec-3"><label>2.1</label><title>Deterministic Directed Acyclic Graph and List Scheduling</title>
<p>List scheduling assigns task priorities and places ready tasks greedily; Kwok and Ahmad <xref ref-type="bibr" rid="ref-7">[7]</xref> survey deterministic graph scheduling. HEFT uses upward ranks based on average durations and insertion-based earliest finish <xref ref-type="bibr" rid="ref-3">[3]</xref>. Stochastic priority-policy analysis addresses related random-duration scheduling problems <xref ref-type="bibr" rid="ref-6">[6]</xref>, while textbook treatments establish the broader theory <xref ref-type="bibr" rid="ref-5">[5]</xref>. The problems and guarantees of those sources are not transferred to the heterogeneous DAG heuristic tested here.</p>
<p>The deterministic methods considered here consume point processing-time estimates and produce schedules quickly. CHARM retains a list-scheduling structure and uses empirical worker multipliers to guide placement. Broader stochastic scheduling literature already studies uncertain durations; the proposed comparison concerns this specific computational construction rather than a universal distinction between deterministic and uncertainty-aware methods.</p>
<p>Insertion, duplication, and lookahead alter how a list scheduler allocates tasks <xref ref-type="bibr" rid="ref-7">[7]</xref>. These design choices can interact with duration models. The reported comparison evaluates static assignment and worker priority under a shared dispatcher, so it does not directly compare all insertion schedules as constructed or establish that uncertainty treatment is independent of dispatch semantics.</p>
</sec>
<sec id="sec-4"><label>2.2</label><title>Scheduling under Uncertainty</title>
<p>Stochastic scheduling analyses priority policies under random processing times <xref ref-type="bibr" rid="ref-5">[5]</xref>, <xref ref-type="bibr" rid="ref-6">[6]</xref>. Chance-constrained programming expresses a requirement that a constraint hold with a specified probability <xref ref-type="bibr" rid="ref-8">[8]</xref>. Robust optimization protects against a specified uncertainty set <xref ref-type="bibr" rid="ref-9">[9]</xref>, <xref ref-type="bibr" rid="ref-10">[10]</xref>. Concentration inequalities support sampling error bounds under their assumptions <xref ref-type="bibr" rid="ref-11">[11]</xref>, and Bayesian modelling offers methods for estimating predictive distributions <xref ref-type="bibr" rid="ref-12">[12]</xref>. These tools provide a formal context for the general framework.</p>
<p>Closer to systems practice, cloud scheduling research has examined deadline-constrained provisioning, including partial critical-path algorithms for infrastructure-as-a-service clouds <xref ref-type="bibr" rid="ref-13">[13]</xref> and auto-scaling approaches that trade cost against deadlines <xref ref-type="bibr" rid="ref-14">[14]</xref>. These works largely treat the deadline as deterministic given provisioned resources, and uncertainty enters through provisioning margins rather than through the schedule itself. Our position is complementary: we take the cluster as given and resource-constrained, and move the uncertainty treatment inside the scheduling computation, where trace-informed distributions can be exploited rather than padded away.</p>
<p>The reported experiment releases one DAG on a fixed cluster. Online arrivals, queue stability across jobs, and runtime adaptation require separate models and are outside its scope. The static local repair pass should not be interpreted as an evaluated online scheduler.</p>
</sec>
<sec id="sec-5"><label>2.3</label><title>Workflow Systems Traces and Simulation</title>
<p>Pegasus and workflow profiling studies supply operational context <xref ref-type="bibr" rid="ref-1">[1]</xref>, <xref ref-type="bibr" rid="ref-2">[2]</xref>, and production cluster analysis illustrates heterogeneity <xref ref-type="bibr" rid="ref-4">[4]</xref>. The campaign uses synthetic layered DAGs and a custom dispatcher rather than production replay. This allows controlled pairing but leaves operational workload diversity and communication costs untested.</p>
<p>Trace-based evaluation can connect scheduling models to observed task variability. Controlled paired simulation also supports comparisons on identical generated instances. The reported experiment uses common random draws across methods, but only ten realized draws per DAG and noise level, so its per-DAG exceedance estimates are coarse. Production-trace validation and calibration uncertainty remain necessary extensions.</p>
</sec>
<sec id="sec-6"><label>2.4</label><title>Synthesis and Positioning</title>
<p>CHARM combines a list-scheduling backbone with distribution-informed placement and repair. The general framework targets a workflow chance constraint; the implemented heuristic uses empirical worker quantiles and a local placement gate. Its reported advantage must be assessed separately from whether it meets an externally specified deadline risk.</p>
</sec>
</sec>
<sec id="sec-7"><label>3</label><title>Problem Formulation and Research Questions</title>
<sec id="sec-8"><label>3.1</label><title>Research Questions and Hypotheses</title>
<p>The research questions concern uncertainty representation, scheduling benefit, and distribution misspecification. The reported experiment addresses restricted versions of them.</p>
<p>RQ1 asks how predictive duration distributions and their estimation uncertainty can inform scheduling. The evaluated implementation uses worker multiplier quantiles rather than a fitted per-class posterior.</p>
<p>RQ2 asks whether risk-aware construction improves realized performance and workflow deadline reliability at an acceptable cost. A common external deadline is required to evaluate the latter directly.</p>
<p>RQ3 asks how performance responds to distribution error, heavy tails, and correlated contention. The reported three-level noise sweep tests only a limited form of distribution shift.</p>
<p>H1 predicts an improvement from risk-aware construction relative to mean-based HEFT. The experiment measures realized makespan and an own-plan exceedance fraction, rather than joint deadline compliance. H2 predicts a benefit from runtime repair. The reported ablation concerns a static local repair pass, so it cannot test runtime rescheduling in its stated form.</p>
<p>H3 predicts controlled degradation as the estimated distribution departs from the true distribution. Three noise levels do not establish linearity in an estimation-error metric, heavy-tail behaviour, or robustness to correlated contention.</p>
<p>These hypotheses distinguish design aims from findings. No verified preregistration record was supplied. Sections 3 and 4 describe the general framework; Section 5 identifies the narrower implementation and evidence.</p>
</sec>
<sec id="sec-9"><label>3.2</label><title>System Model and Notation</title>
<p>A job is a DAG G = (T, E) of tasks i in T connected by data edges (i, j) in E. The platform consists of heterogeneous resources r in R, each with a speed normalisation and a resource-time price. The duration of task i on resource r is a random variable D_{i, r} whose distribution is estimated from traces, and the transfer volume along edge (i, j) is likewise random when its endpoints are placed on different resources. The job carries a deadline delta and, optionally, a resource budget B. A schedule s assigns each task to a resource and a start order, and its makespan C_max(s) is the completion time of the final task, itself a random variable induced by the durations. <xref ref-type="table" rid="tab1">Table 1</xref> lists the principal symbols.</p>
<table-wrap id="tab1"><label>Table 1</label><caption><p>Principal notation for the chance-constrained scheduling problem.</p></caption><graphic content-type="pdf-region" xlink:href="pdf-region:3:66,309.2,537.1,354.7"/><p content-type="note">This table is shown in the PDF version of the article.</p></table-wrap>
<p>D_{i,r} Random duration of task i on resource r Time units W_{ij} Random transfer volume on edge (i, j) MB R Set of heterogeneous resources - delta Job deadline Time units epsilon Tolerated violation probability - C_max(s) Makespan of schedule s Time units rho_r Resource-time price of resource r Cost/time unit alpha Rank quantile level - z_{1-alpha} Standard normal quantile at level 1-alpha - K Independent draw count for conceptual risk assessment - L_i Latest allowable start of task i Time units</p>
</sec>
<sec id="sec-10"><label>3.3</label><title>Formal Problem Statement</title>
<p>A schedule assigns each task to a resource and a nonpreemptive worker order, respecting precedence and at most one executing task per worker. With resource-time price ρ_r, the general problem minimizes expected execution cost subject to a fixed deadline δ and risk tolerance ε:</p>
<p>minimize_s E[Σ_{i∈T} ρ_{s(i)} D_{i,s(i)}]</p>
<p>s.t (1)</p>
<p>Pr{C_max(s) &gt; δ} ≤ ε, precedence and worker nonoverlap constraints.</p>
<p>Eq. (1) is a general chance-constrained model <xref ref-type="bibr" rid="ref-8">[8]</xref>. A simulated exceedance fraction is an estimator of its risk, not automatically a certificate. For a fixed schedule evaluated using K independent draws from a specified model, let p̂_viol be the empirical fraction:</p>
<p>p̂ _viol(s) = K⁻¹ Σ_{k=1}^{K} 1{C_max^(k)(s) &gt; δ}; p̂ _viol(s) + √[log(1/β)/(2K)] ≤ ε. (2)</p>
<p>The margin in Eq. (2) is a one-sided Hoeffding bound at confidence 1−β, where 0 &lt; β &lt; 1, for a fixed schedule under the sampling model <xref ref-type="bibr" rid="ref-11">[11]</xref>. It does not account for misspecified duration laws, estimated model parameters, correlated draws, or repeated adaptive schedule selection. Independent final certification or a simultaneous bound is needed when many candidates are searched. The reported implementation neither uses this workflow certification rule nor demonstrates the deadline guarantee in Eq. (1).</p>
</sec>
</sec>
<sec id="sec-11"><label>4</label><title>Proposed Methodology</title>
<sec id="sec-12"><label>4.1</label><title>Framework Overview</title>
<p>The general framework has estimation, construction, and monitoring layers. Estimated duration and transfer models supply schedule candidates; risk evaluation determines whether they meet a fixed deadline; monitoring can initiate rescheduling with updated information. These are conceptual interfaces. The reported implementation uses empirical worker multiplier traces, a local deadline gate, and a static bounded repair pass. Offline estimation and event-triggered updates can reduce runtime overhead, but their cost and calibration must be measured. No production monitoring or runtime-event experiment is reported here.</p>
</sec>
<sec id="sec-13"><label>4.2</label><title>Distributional Estimation from Traces</title>
<p>A per-class log-normal model with parameter uncertainty and shrinkage is a possible estimation layer <xref ref-type="bibr" rid="ref-12">[12]</xref>. Such a model would require an estimation procedure, priors, calibration checks, and a predictive distribution before it could support a chance constraint. Empirical quantiles are another option, with uncertainty depending on trace size and relevance. Finite moment diagnostics cannot prove that a distribution has infinite variance. The reported campaign uses 500 multiplier samples per worker at σ_cal = 0.3, without evaluating per-class shrinkage or transfer estimation.</p>
</sec>
<sec id="sec-14"><label>4.3</label><title>Chance Aware Construction</title>
<p>A list scheduler can prioritize tasks using a duration score that combines a predictive mean μ with variability σ. Eq. (3) is a normal-quantile surrogate, not a log-normal quantile or a bound on a path maximum:</p>
<p>d̃ _{i,r} = μ_{i,r} + z_{1−α} σ_{i,r}, 0 &lt; α &lt; 1. (3)</p>
<p>For D ∼ LogNormal(a,b²), the exact marginal quantile is exp(a + b z_{1−α}). Summing marginal quantiles does not in general give the same quantile of a sum or workflow makespan. Joint deadline assessment must include precedence, worker queueing, dependence, and communication assumptions. The tested heuristic derives task deadlines from downstream slack and uses empirical worker multiplier quantiles; it does not propagate exact workflow distributions.</p>
<p>If no candidate placement meets a risk requirement, the framework must report infeasibility or use a specified escalation whose revised risk and resource cost are evaluated. Replication or relaxing ε changes the decision problem and cannot silently preserve the original guarantee. A local per-task gate with probability 0.9 does not imply 90% workflow completion probability; a joint certificate or valid allocation of total risk is required.</p>
</sec>
<sec id="sec-15"><label>4.4</label><title>Runtime Repair</title>
<p>Runtime repair would condition remaining durations on observed progress and reschedule only unexecuted tasks while respecting completed and running work. Its effectiveness depends on engine support, repair cost, trigger definitions, and re-evaluation of joint risk. This is a proposed extension. The reported repair pass modifies a static schedule using a quantile-inflated makespan estimate, with at most five moves, and does not demonstrate event-driven runtime adaptation.</p>
</sec>
<sec id="sec-16"><label>4.5</label><title>The CHARM Algorithm</title>
<p>Algorithm 1 states the general construction-and-monitoring interfaces, not the procedure executed in Section 5. Probability checks refer to independent or simultaneous risk assessment under a specified joint model. The narrower worker-quantile heuristic is described in the experimental setup.</p>
<p>Algorithm 1. Conceptual workflow risk assessment and rescheduling.</p>
<p>Input: DAG G, resources R, fixed deadline δ, risk ε, predictive model, certification confidence 1: fit or load specified predictive models and calibration evidence 2: generate a precedence-feasible schedule using a list heuristic 3: assess workflow risk using independent or simultaneous certification 4: while no admissible schedule has been found do 5: generate a new candidate within a stated search budget 6: verify worker nonoverlap and precedence constraints 7: evaluate expected cost and joint deadline risk 8: if the risk condition is met, retain the candidate 9: if the search budget is exhausted, report failure or an explicit escalation 10: end construction loop; if no admissible schedule exists, report failure and stop 11: otherwise dispatch the retained schedule; optionally monitor execution progress 12: while unfinished work remains do 13: update the conditional duration model without changing completed work 14: if a specified repair trigger is met, generate a feasible remaining-work schedule 15: reassess its risk using the stated certification procedure 16: apply it only within the repair and resource budgets 17: report estimated risk, confidence scope, and any escalation Output: schedule and risk assessment conditional on the supplied model</p>
</sec>
<sec id="sec-17"><label>4.6</label><title>Complexity Assumptions and Failure Conditions</title>
<p>A HEFT-style construction has O(|T|²|R|) scheduling cost under its standard implementation. Dense task– resource costs require O(|T||R|) storage, in addition to O(|T|+|E|) graph and schedule state. Simulating K workflow realizations adds the cost of K full dispatch runs; it cannot generally be bounded by only the number of nominal critical paths, because resource contention can change the realized path. Candidate search, model estimation, and repeated repairs multiply these costs. The reported experiment supplies total harness timing, without a separate scheduler-cost benchmark.</p>
<p>The simulation assumes the stated task multipliers and fixed worker factors. Correlated contention requires an explicit joint model and is untested.</p>
<p>Calibration traces must remain relevant to future durations. The outer noise levels intentionally change the multiplier distribution.</p>
<p>Task deadlines derived from slack are heuristic allocations; near-tied paths and queueing can invalidate a local interpretation as a joint guarantee.</p>
<p>Repair requires an explicit move and computation budget. The tested pass allows at most five moves. These conditions define the proposed risk assessment. Monitoring does not by itself establish that every failure regime is detectable or that a cold-start fallback is safe.</p>
<p>Heavy-tailed durations may require empirical or model-based quantiles rather than moment scores; their adequacy must be validated.</p>
<p>Shared contention can couple task durations and defeat an independence-based assessment.</p>
<p>A cold start without defensible duration information leaves reliability unquantified. A conservative schedule cannot be called safe without a justified uncertainty set.</p>
</sec>
</sec>
<sec id="sec-18"><label>5</label><title>Experimental Results and Analysis</title>
<sec id="sec-19"><label>5.1</label><title>Setup</title>
<p>The manuscript reports 100 random layered DAGs with one entry and one exit, 30–60 tasks (mean 45.3), 3– 6 middle layers, and inter-layer edge probabilities in [0.2, 0.4]. Eight workers have speed factors in [0.53, 1.91] and fixed noise factors f_w in [0.66, 1.48]. A nominal task duration is base duration divided by speed; realized duration multiplies it by LogNormal(0,σ f_w), with σ ∈ {0.2,0.3,0.4}. Under this reported parameterization the multiplier mean is exp((σ f_w)²/2), greater than one, so the nominal duration is not the stochastic mean. The table label “expected” denotes nominal simulation values here. Whether the code instead centers multipliers requires verification.</p>
<p>Every method is reported as receiving nominal durations and a 500-task multiplier calibration trace per worker at σ_cal = 0.3. CHARM sets T* to 1.05 times the HEFT nominal construction makespan, derives task deadlines from downstream slack, and chooses a worker passing an empirical 90% local finish gate, preferring the earliest median-scaled finish. A static repair pass responds to a quantile-inflated makespan estimate above T*. The exact feasibility fallback, task-deadline formula, and repair moves require the implementation record. The 0.9 task gate is an estimated local condition, not a certified workflow chance constraint.</p>
<p>All methods are evaluated without communication cost by a shared event-driven dispatcher. A static assignment and per-worker priority order are converted into execution by starting the highest-priority ready task on each idle worker. This may differ from HEFT&apos;s constructed insertion schedule, so the comparison is between assignments under common dispatch rather than faithful insertion-schedule execution. Ten common random draws per DAG and noise level pair methods. The reported miss fraction uses a different threshold for each method, 1.1 times its own nominal makespan; robustness is exactly one minus that fraction and supplies no independent outcome. No common external deadline or T* compliance is reported. Software references identify NumPy, SciPy, and statsmodels <xref ref-type="bibr" rid="ref-15">[15–17]</xref>, and the stated tests use Wilcoxon <xref ref-type="bibr" rid="ref-18">[18]</xref>, Demšar <xref ref-type="bibr" rid="ref-19">[19]</xref>, and Holm <xref ref-type="bibr" rid="ref-20">[20]</xref>.</p>
<table-wrap id="tab2"><label>Table 2</label><caption><p>Executed experimental setup: workload, cluster, information model, methods, evaluation, and software, as run on 2026-10-11.</p></caption><graphic content-type="pdf-region" xlink:href="pdf-region:6:66,211.7,537.1,267.1"/><graphic content-type="pdf-region" xlink:href="pdf-region:7:66,672.4,537.1,756.2"/><p content-type="note">This table is shown in the PDF version of the article.</p></table-wrap>
<p>Cluster 8 heterogeneous workers, speed factors in [0.53, 1.91], realized duration = expected x LogNormal(0, sigma x f_w) with per-worker noise factor f_w in [0.66, 1.48] Information model Schedulers see expected durations only, plus a 500-task calibration trace per worker (empirical multiplier quantiles) taken at sigma_cal = 0.3 Noise levels Sigma in {0.2, 0.3, 0.4}, 10 common-random-number duration draws per DAG and level Methods Random assignment, EFT list scheduling (insertion-based), HEFT (upward rank + insertion-based EFT), static greedy on expected durations, CHARM (deadline from downstream slack, P(finish &lt;= deadline) &gt;= 0.9 via worker quantiles, local repair) Evaluation One shared event-driven simulator (static assignment, dynamic ready dispatch per worker in priority order) scores all methods on expected and realized durations Metrics Expected makespan, realized makespan (mean +/- std), deadline-miss probability P(realized &gt; 1.1 x expected), robustness index P(realized &lt;= 1.1 x expected)</p>
</sec>
<sec id="sec-20"><label>5.2</label><title>Main Comparison RQ1 RQ2</title>
<p>At σ = 0.3, <xref ref-type="table" rid="tab3">Table 3</xref> gives mean realized makespan 9.80 for CHARM and 9.94 for HEFT-based assignment under the shared dispatcher, a reduction of 1.4%. Reported within-DAG standard deviations are 0.94 and 1.02. The source gives paired Wilcoxon W = 1413, p = 0.000132 across 100 DAG means, with CHARM lower on 66 DAGs. The medians are nearly equal, 9.68 and 9.67. Random, EFT, and static greedy have larger realized means. CHARM&apos;s nominal makespan is 8.07 versus 7.82 for HEFT, a 3.2% increase. This comparison describes an assignment trade-off under the simulator, rather than an increase in expected monetary or energy cost.</p>
<table-wrap id="tab3"><label>Table 3</label><caption><p>Main comparison at sigma = 0.3 over 100 DAGs: expected makespan, realized makespan (mean over 10 draws per DAG), deadline-miss probability, and robustness index. Makespans in cluster time</p></caption><p content-type="note">This table is shown in the PDF version of the article.</p></table-wrap>
<fig id="fig1"><label>Figure 1</label><caption><p>Reported empirical CDF of realized makespan at σ = 0.3, normalized by the stated critical-path lower bound, pooling 1,000 DAG–draw observations per method. The close CHARM and HEFT curves do not establish stochastic dominance; draws within a DAG are related observations.</p></caption><graphic content-type="pdf-region" xlink:href="pdf-region:7:66,228.1,537.1,509.6"/><p content-type="note">This figure is shown in the PDF version of the article.</p></fig>
<p>Plan-relative exceedance fractions are 0.797 for CHARM and 0.891 for HEFT; their complementary robustness values are 0.203 and 0.109. CHARM&apos;s threshold is larger because its nominal makespan is larger. Random assignment has a smaller exceedance fraction despite much slower execution because its own plan is substantially longer. These values measure conformity to each method&apos;s own nominal plan, not the probability of meeting one shared application deadline. The source&apos;s signed-rank summary for this comparison is internally inconsistent and remains a verification dependency in <xref ref-type="table" rid="tab7">Table 7</xref>.</p>
</sec>
<sec id="sec-21"><label>5.3</label><title>Ablation Which Component Carries the Gain</title>
<p>Removing the local chance gate raises displayed realized makespan from 9.80 to 9.93, with reported W = 1348 and p = 5.19 × 10⁻⁵; its own-plan exceedance fraction rises by 0.029, from 0.797 to 0.826. The ablated nominal makespan (8.04) differs from HEFT&apos;s (7.82), so this variant cannot be described as identical to HEFT. Removing static repair leaves both displayed realized means at 9.80 (reported p = 0.69) and changes exceedance fraction from 0.797 to 0.842 while also lowering the nominal plan from 8.07 to 7.93. This change does not by itself establish tail protection at a common threshold. The source reports repair on 46% of DAGs, 0.91 moves per DAG, capped at five. The experiment supports a benefit of the local gate on realized makespan within this design; runtime-repair H2 remains untested.</p>
<table-wrap id="tab4"><label>Table 4</label><caption><p>Ablation at sigma = 0.3: CHARM without the chance constraint and without the repair loop, against the full method.</p></caption><graphic content-type="pdf-region" xlink:href="pdf-region:8:66,489.3,537.1,596.2"/><p content-type="note">This table is shown in the PDF version of the article.</p></table-wrap>
<p><xref ref-type="table" rid="tab5">Table 5</xref> varies target factor γ on the first 30 DAGs. Nominal means range from 8.32 to 8.36, realized means from 10.01 to 10.11, and own-plan exceedance fractions from 0.797 to 0.810. The small variation suggests limited sensitivity in this particular subset and parameter range. It does not establish that the gate saturates or that γ controls external deadline reliability.</p>
<table-wrap id="tab5"><label>Table 5</label><caption><p>Target-factor sensitivity on the first 30 DAGs at σ = 0.3.</p></caption><graphic content-type="pdf-region" xlink:href="pdf-region:8:66,308.6,537.1,409.9"/><p content-type="note">This table is shown in the PDF version of the article.</p></table-wrap>
</sec>
<sec id="sec-22"><label>5.4</label><title>Stress across Noise Levels RQ3</title>
<p><xref ref-type="table" rid="tab6">Table 6</xref> and <xref ref-type="fig" rid="fig2">Fig. 2</xref> vary true noise while calibration stays at σ_cal = 0.3. CHARM&apos;s realized mean is 8.89 versus HEFT&apos;s 8.90 at σ = 0.2, 9.80 versus 9.94 at 0.3, and 11.05 versus 11.33 at 0.4. The gap grows over these three conditions. The multiplier parameterization also changes its mean, so this is not a pure variance or estimation-error experiment. Own-plan exceedance fractions rise for every method; at σ = 0.4 they exceed 0.9 for HEFT and CHARM, but remain 0.662, 0.837, and 0.820 for random, EFT, and greedy. These different thresholds preclude a cross-method deadline-reliability ranking. The sweep does not establish linear degradation, heavy-tail robustness, or correlated-contention performance.</p>
<table-wrap id="tab6"><label>Table 6</label><caption><p>Stress across noise levels: realized makespan and deadline-miss probability per method at sigma = 0.2, 0.3, 0.4.</p></caption><graphic content-type="pdf-region" xlink:href="pdf-region:8:66,47.7,537.1,145.1"/><p content-type="note">This table is shown in the PDF version of the article.</p></table-wrap>
<fig id="fig2"><label>Figure 2</label><caption><p>Deadline-miss probability versus noise level sigma (mean over 100 DAGs; error bars are standard errors).</p></caption><graphic content-type="pdf-region" xlink:href="pdf-region:9:66,624.2,537.1,778.9"/><p content-type="note">This figure is shown in the PDF version of the article.</p></fig>
<p>Note (<xref ref-type="fig" rid="fig2">Fig. 2</xref>): The inversion between the tight schedules (HEFT worst) and the conservative ones (random best) is the own-expectation artifact discussed in Section 5.2; CHARM sits below HEFT at every level.</p>
</sec>
<sec id="sec-23"><label>5.5</label><title>Statistical Analysis</title>
<p><xref ref-type="table" rid="tab7">Table 7</xref> retains the reported paired Wilcoxon and Friedman results. The primary realized-makespan comparison uses 100 DAG means, rather than treating 1,000 duration draws as independent instances. A four-comparison CHARM-versus-baseline family uses Holm correction; the source&apos;s “five-way family” wording refers to five methods, not five pairwise tests. The miss-probability row reports 54 better and 46 tied DAGs alongside W = 455; for the usual signed-rank statistic, 54 same-direction nonzero differences would have minimum rank sum zero. Raw paired outcomes and the statistic convention are needed to resolve this inconsistency. Displayed p = 0.000 entries denote rounded values, not zero probability. Until regenerated, these statistics remain source-reported calculations rather than independently confirmed tests.</p>
<table-wrap id="tab7"><label>Table 7</label><caption><p>Statistical analysis at sigma = 0.3: paired Wilcoxon signed-rank tests (primary endpoint, miss probability, and the Holm-corrected five-way family) and the Friedman test.</p></caption><graphic content-type="pdf-region" xlink:href="pdf-region:9:65.9,74.8,537.1,385.4"/><p content-type="note">This table is shown in the PDF version of the article.</p></table-wrap>
</sec>
<sec id="sec-24"><label>5.6</label><title>Discussion Findings Losses and Scope</title>
<p>The reported realized-makespan means favour CHARM over HEFT-based assignment at intermediate and high noise, with a modest nominal-plan increase. The local-gate ablation supports a construction-time effect in this simulation. Static repair has no demonstrated mean effect at the displayed precision, and its own-plan exceedance change cannot establish fixed-deadline tail protection. These are the empirical findings supported The close realized medians and 66 favourable DAGs show that CHARM does not improve every instance. HEFT has the smaller nominal plan, whereas plan-relative exceedance measures adherence to that plan and uses method-dependent thresholds. Target-factor variation is small on the 30-DAG subset. None of these observations provides a certified workflow reliability guarantee or an operating recommendation for a production cluster.</p>
<p>The evidence covers 100 synthetic DAGs, one eight-worker cluster, no communication cost, log-normal multipliers, and a makespan objective. Production traces, correlated stragglers, heavy-tailed models, runtime rescheduling, replication, elasticity, energy, and multi-job fairness are untested. A future experiment should evaluate all schedulers at shared external deadlines and report feasibility, calibration, scheduler overhead, and outcome uncertainty.</p>
<p>The numerical results are reported at proof-of-concept scale. Independent regeneration requires the harness, raw draws, calibration samples, assignments, and execution logs. Resolution of the signed-rank inconsistency remains necessary; a verified release is not specified.</p>
</sec>
</sec>
<sec id="sec-25"><label>6</label><title>Discussion Limitations and Counterarguments</title>
<sec id="sec-26"><label>6.1</label><title>Limitations</title>
<p>Risk assessment is conditional on a duration model and its calibration. Shared contention can introduce dependence absent from the simulation. Quantiles estimated from 500 samples have sampling uncertainty, and changes in workload or cluster conditions can invalidate them. The local task gate used here is weaker than the joint deadline condition in Eq. (1), while the own-plan exceedance measure does not assess that condition. Small-scale synthetic results cannot certify production reliability.</p>
<p>The general formulation concerns one job on a fixed cluster. Multi-job fairness, preemption, and elasticity require additional constraints. The tested repair pass is static and does not require demonstrated runtime preemption or replication. Runtime extensions should specify engine support and measure their cost.</p>
</sec>
<sec id="sec-27"><label>6.1</label><title>Alternative Viewpoints and Responses</title>
<p>Robust optimization protects against an uncertainty set <xref ref-type="bibr" rid="ref-9">[9]</xref>, <xref ref-type="bibr" rid="ref-10">[10]</xref>, while chance constraints depend on a probability law. Their relative conservatism depends on how the set, law, and risk tolerance are chosen. Neither inherently collapses the risk–cost trade-off to a single setting. A comparison should expose these choices and assess both at shared deadline and resource conditions.</p>
<p>Learned schedulers provide another possible comparator, whose calibration, sample needs, and distribution-shift behaviour require evidence. Transparent local risk scores are inspectable, but are not automatically probability certificates. The reported ablations isolate a static local gate and repair pass, without testing learning-based policies or reactive runtime rescheduling.</p>
</sec>
</sec>
<sec id="sec-28"><label>7</label><title>Conclusion and Future Directions</title>
<p>CHARM uses worker-calibrated quantiles to guide static assignment and local schedule repair. Reported results on 100 synthetic DAGs give mean realized makespan 9.80 versus 9.94 for HEFT-based assignment at σ = 0.3 under common dispatch, with nominal makespan 8.07 versus 7.82. The local-gate ablation raises realized mean by 0.13, whereas static repair leaves the displayed mean unchanged. The gap in realized means grows across the three tested noise levels. These results support further evaluation of the heuristic, without demonstrating the general framework&apos;s chance constraint or runtime-repair aims.</p>
<p>The next evaluation should first regenerate the paired statistics and specify whether nominal durations are centered stochastic means. Shared external deadlines are needed to compare actual deadline reliability, and joint independent certification is needed before using a risk guarantee. Faithful insertion-schedule execution, communication costs, trace-derived workloads, calibration uncertainty, and correlated durations would strengthen the benchmark. Runtime repair should then be tested as a separate implemented mechanism, with its computation and resource costs reported.</p>
</sec>
<sec id="sec-29"><title>Author Contributions</title>
<p>M. P.: Study conception and design, data collection, data analysis, interpretation of results, and drafting of the original manuscript. S. K.: Study conception and design, data collection, data analysis, interpretation of results, and drafting of the original manuscript. A. C.: Data analysis, interpretation of findings, and critical revision of the manuscript. S. Q. H.: Data analysis, interpretation of findings, and critical revision of the manuscript. P. J. D. W.: Study supervision, critical review, and editing of the manuscript. All authors read and approved the final version of the manuscript.</p>
</sec>
<sec id="sec-30"><title>Funding</title>
<p>This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.</p>
</sec>
<sec id="sec-31"><title>Data Availability</title>
<p>The reported workload consists of 100 generated layered DAGs on an eight-worker simulated cluster. Exact graph instances, worker factors, multiplier calibration traces, and raw duration draws require the experiment record.</p>
</sec>
<sec id="sec-32"><title>Code Availability</title>
<p>A verified public release of the simulation harness and raw outcomes is not specified.</p>
</sec>
<sec id="sec-33"><title>Conflicts of Interest</title>
<p>The authors declare no conflicts of interest.</p>
</sec>
<sec id="sec-34"><title>Generative AI Disclosure</title>
<p>AI-assisted tools were used in drafting and revising the manuscript text. The authors retain responsibility for verification of the scientific content and the applicable disclosure.</p>
</sec>
</body>
<back><ref-list><title>References</title>
<ref id="ref-1"><label>1</label><mixed-citation>Deelman, E., Vahi, K., Juve, G., Rynge, M., Callaghan, S., Maechling, P. J., … ., &amp; Wenger, K. (2015). Pegasus, a workflow management system for science automation. Future generation computer systems, 46, 17–35. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1016/j.future.2014.10.008">https://doi.org/10.1016/j.future.2014.10.008</ext-link></mixed-citation></ref>
<ref id="ref-2"><label>2</label><mixed-citation>Juve, G., Chervenak, A., Deelman, E., Bharathi, S., Mehta, G., &amp; Vahi, K. (2013). Characterizing and profiling scientific workflows. Future generation computer systems, 29(3), 682–692. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1016/j.future.2012.08.015">https://doi.org/10.1016/j.future.2012.08.015</ext-link></mixed-citation></ref>
<ref id="ref-3"><label>3</label><mixed-citation>Topcuoglu, H., Hariri, S., &amp; Wu, M.-Y. (2002). Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE transactions on parallel and distributed systems, 13(3), 260–274. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1109/71.993206">https://doi.org/10.1109/71.993206</ext-link></mixed-citation></ref>
<ref id="ref-4"><label>4</label><mixed-citation>Reiss, C., Tumanov, A., Ganger, G. R., Katz, R. H., &amp; Kozuch, M. A. (2012). Heterogeneity and dynamicity of clouds at scale: Google trace analysis. Proceedings of the third ACM symposium on cloud computing (pp. 1– 13). Association for Computing Machinery. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1145/2391229.2391236">https://doi.org/10.1145/2391229.2391236</ext-link></mixed-citation></ref>
<ref id="ref-5"><label>5</label><mixed-citation>Pinedo, M. L. (2012). Scheduling: Theory, algorithms, and systems. Springer. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1007/978-1-4614-2361-4">https://doi.org/10.1007/978-1-4614-2361-4</ext-link></mixed-citation></ref>
<ref id="ref-6"><label>6</label><mixed-citation>Möhring, R. H., Schulz, A. S., &amp; Uetz, M. (1999). Approximation in stochastic scheduling: The power of LP-based priority policies. Journal of the association for computing machinery,, 46(6), 924–942. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1145/331524.331530">https://doi.org/10.1145/331524.331530</ext-link></mixed-citation></ref>
<ref id="ref-7"><label>7</label><mixed-citation>Kwok, Y. K., &amp; Ahmad, I. (1999). Static scheduling algorithms for allocating directed task graphs to multiprocessors. ACM computing surveys., 31(4), 406–471. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1145/344588.344618">https://doi.org/10.1145/344588.344618</ext-link></mixed-citation></ref>
<ref id="ref-8"><label>8</label><mixed-citation>Prékopa, A. (1995). Stochastic programming. Springer. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1007/978-94-017-3087-7">https://doi.org/10.1007/978-94-017-3087-7</ext-link></mixed-citation></ref>
<ref id="ref-9"><label>9</label><mixed-citation>Bertsimas, D., &amp; Sim, M. (2004). The price of robustness. Operations research, 52(1), 35–53. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1287/opre.1030.0065">https://doi.org/10.1287/opre.1030.0065</ext-link></mixed-citation></ref>
<ref id="ref-10"><label>10</label><mixed-citation>Ben-Tal, A., Nemirovski, A., &amp; El Ghaoui, L. (2009). Robust optimization. Princeton university press. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1515/9781400831050">https://doi.org/10.1515/9781400831050</ext-link></mixed-citation></ref>
<ref id="ref-11"><label>11</label><mixed-citation>Boucheron, S., Lugosi, G., &amp; Massart, P. (2013). Concentration inequalities: A nonasymptotic theory of independence. Oxford University Press. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1093/acprof:oso/9780199535255.001.0001">https://doi.org/10.1093/acprof:oso/9780199535255.001.0001</ext-link></mixed-citation></ref>
<ref id="ref-12"><label>12</label><mixed-citation>Gelman, A., Carlin, J. B., Stern, H. S., &amp; Rubin, D. B. (2013). Bayesian data analysis. Chapman and hall/cRc. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1201/b16018">https://doi.org/10.1201/b16018</ext-link></mixed-citation></ref>
<ref id="ref-13"><label>13</label><mixed-citation>Abrishami, S., Naghibzadeh, M., &amp; Epema, D. H. J. (2013). Deadline-constrained workflow scheduling algorithms for Infrastructure as a Service Clouds. Future generation computer systems, 29(1), 158–169. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1016/j.future.2012.05.004">https://doi.org/10.1016/j.future.2012.05.004</ext-link></mixed-citation></ref>
<ref id="ref-14"><label>14</label><mixed-citation>Mao, M., &amp; Humphrey, M. (2011). Auto-scaling to minimize cost and meet application deadlines in cloud workflows. Proceedings of 2011 international conference for high performance computing, networking, storage and analysis. New York, NY, USA: Association for Computing Machinery. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1145/2063384.2063449">https://doi.org/10.1145/2063384.2063449</ext-link></mixed-citation></ref>
<ref id="ref-15"><label>15</label><mixed-citation>Harris, C. R., Millman, K. J., van der Walt, S. J., Gommers, R., Virtanen, P., Cournapeau, D., … ., &amp; Oliphant, T. E. (2020). Array programming with NumPy. Nature, 585(7825), 357–362. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1038/s41586-020-2649-2">https://doi.org/10.1038/s41586-020-2649-2</ext-link></mixed-citation></ref>
<ref id="ref-16"><label>16</label><mixed-citation>Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., … ., &amp; Contributors, S. 1. . (2020). SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nature methods, 17(3), 261–272. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.1038/s41592-019-0686-2">https://doi.org/10.1038/s41592-019-0686-2</ext-link></mixed-citation></ref>
<ref id="ref-17"><label>17</label><mixed-citation>Seabold, S., Perktold, J., &amp; others. (2010). Statsmodels: Econometric and statistical modeling with python. Proceedings of the python in science conference (pp. 92–96). SciPy. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.25080/Majora-92bf1922-">https://doi.org/10.25080/Majora-92bf1922-</ext-link></mixed-citation></ref>
<ref id="ref-18"><label>18</label><mixed-citation>Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics bulletin, 1(6), 80–83. <ext-link ext-link-type="uri" xlink:href="https://doi.org/10.2307/3001968">https://doi.org/10.2307/3001968</ext-link></mixed-citation></ref>
<ref id="ref-19"><label>19</label><mixed-citation>Demšar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of machine learning research, 7(1), 1–30. <ext-link ext-link-type="uri" xlink:href="https://jmlr.org/papers/v7/demsar06a.html">https://jmlr.org/papers/v7/demsar06a.html</ext-link></mixed-citation></ref>
<ref id="ref-20"><label>20</label><mixed-citation>Holm, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics, 6(2), 65–70. <ext-link ext-link-type="uri" xlink:href="https://www.jstor.org/stable/4615733">https://www.jstor.org/stable/4615733</ext-link></mixed-citation></ref>
</ref-list></back>
</article>