Transformation
Transformation3 min readBy The Data Workers Team

Move Informatica, SSIS and DataStage jobs to tested, reviewable code

Legacy ETL tools hold decades of business logic in proprietary mappings. Data Workers reads the jobs, rebuilds them as dbt and orchestrated code, proves the outputs match and retires the old tool.

Before and after

TodayWith Data Workers
Business logic in proprietary mappingsLogic as reviewed dbt and SQL
Licenses that renew every yearLicenses retired
Jobs nobody wants to touchEvery job tested and documented
Rewrites estimated in yearsA rewrite measured in job families

How it works

1. Read every job

Data Context & Catalog documents mappings, sources, targets and schedules.

2. Rebuild as code

Data Migration and Pipeline Building rebuild each job family as dbt models and orchestrated tasks with tests.

3. Prove and retire

Quality Monitoring proves outputs match; the legacy jobs are switched off family by family.

The agents behind the legacy etl modernization: Data Context & Catalog agent → Data Migration agent → Pipeline Building agent → Quality Monitoring agent.
Fig. 1:The agents behind the legacy etl modernization: Data Context & Catalog agent → Data Migration agent → Pipeline Building agent → Quality Monitoring agent.

The impact metrics we move

  • •Job family documented: from about 2 weeks to 4 h
  • •Job family rebuilt: from about 2 months to 3 days
  • •Output parity: from about 2 days to under 1 h
Time to complete, current practice (black) vs. with Data Workers (green). Illustrative.
Fig. 2:Time to complete, current practice (black) vs. with Data Workers (green). Illustrative.

Where teams start

  • •Informatica PowerCenter to dbt. Mappings rebuilt as models.
  • •SSIS packages to orchestrated code. Packages rebuilt as tasks.
  • •DataStage retirement. Jobs rebuilt and licenses dropped.
  • •Stored procedures to models. Logic made testable.
  • •Scheduler consolidation. Jobs moved to one orchestrator.

The climb: from assisted to autonomous to agentic

Every transformation runs on the same ladder from our thesis: agents first inventory and propose with receipts, then move work in approved waves, and the finished platform keeps running itself once the move is done.

The pilot

  • •Scope. Rebuild one job family and prove its outputs.
  • •First week. Agents run at L1, observing only, and show the fixes they would have made, with receipts.
  • •By week two. The routine work above runs on its own; anything sensitive stays with a named approver.
  • •At day 30. We review the impact metrics against your baseline:
  • •Job family documented: from about 2 weeks to 4 h
  • •Job family rebuilt: from about 2 months to 3 days
  • •Output parity: from about 2 days to under 1 h
  • •Cost. Start with a pilot ($7,500 one-time, credited in full against the first year). Scale is from $1,000 a month and Enterprise from $3,000 a month, billed annually, with unlimited seats, no usage meter and no markup on model spend. See pricing.

FAQ

Do we rewrite everything at once? No; job families move one at a time.

How do we know outputs match? Parity tests on every output table.

Who reviews the code? Your engineers, as pull requests.