Transformation
Transformation3 min readBy The Data Workers Team

Move from legacy schedulers to modern orchestration without a freeze

Control-M, Autosys and aging Airflow deployments hold thousands of jobs nobody wants to migrate. Data Workers maps every job and dependency, moves them in waves and verifies every SLA.

Before and after

TodayWith Data Workers
Thousands of jobs in a legacy schedulerEvery job and dependency mapped
Dependencies nobody can drawJobs moved in reversible waves
Upgrades postponed for yearsSLAs verified through the move
Freezes for every migrationNo freeze

How it works

1. Map every job

Data Context & Catalog maps jobs, schedules and dependencies.

2. Move in waves

Data Migration rebuilds jobs as DAGs in the target orchestrator, wave by wave.

3. Verify SLAs

Conductor watches every SLA and rolls back any wave that drifts.

The agents behind the orchestrator modernization: Data Context & Catalog agent → Data Migration agent → Pipeline Building agent → Autonomous Data-Conductor.
Fig. 1:The agents behind the orchestrator modernization: Data Context & Catalog agent → Data Migration agent → Pipeline Building agent → Autonomous Data-Conductor.

The impact metrics we move

  • •Job inventory: from about a month to 8 h
  • •Wave of jobs moved: from about a month to ~1 wk
  • •SLA verified: from about a day to 8-15 min
Time to complete, current practice (black) vs. with Data Workers (green). Illustrative.
Fig. 2:Time to complete, current practice (black) vs. with Data Workers (green). Illustrative.

Where teams start

  • •Control-M and Autosys to Airflow or Dagster. Jobs rebuilt as DAGs.
  • •Airflow 2 to 3. Upgrades without a freeze.
  • •Cron and scripts. Consolidated under one orchestrator.
  • •Cross-team dependencies. Drawn and enforced.
  • •SLA monitoring. Verified on every run.

The climb: from assisted to autonomous to agentic

Every transformation runs on the same ladder from our thesis: agents first inventory and propose with receipts, then move work in approved waves, and the finished platform keeps running itself once the move is done.

The pilot

  • •Scope. Map the estate and move one wave of jobs.
  • •First week. Agents run at L1, observing only, and show the fixes they would have made, with receipts.
  • •By week two. The routine work above runs on its own; anything sensitive stays with a named approver.
  • •At day 30. We review the impact metrics against your baseline:
  • •Job inventory: from about a month to 8 h
  • •Wave of jobs moved: from about a month to ~1 wk
  • •SLA verified: from about a day to 8-15 min
  • •Cost. Start with a pilot ($7,500 one-time, credited in full against the first year). Scale is from $1,000 a month and Enterprise from $3,000 a month, billed annually, with unlimited seats, no usage meter and no markup on model spend. See pricing.

FAQ

Do we need a freeze? No; waves are small.

What about cross-team dependencies? They are mapped before anything moves.

What if a wave misses an SLA? It rolls back automatically.