← Back to Stories

Stories · Energy & Utilities

AEP cuts its interval meter data pipeline cost 35% and runs it 17x faster

Mactores and AWS rebuilt the non-VEE pipeline behind AEP’s interval-metering analytics on Amazon EMR and Apache Iceberg in 13 weeks.

Data Platform
Energy & Utilities
Amazon EMR
Apache Iceberg
AWS Glue
Amazon Bedrock AgentCore

AEP SERVES

~5.6M

customers

states

What Mactores delivered for AEP

Cost

Pipeline cost, indexed to 100

Before

100

Now

65, with the commitment at 70

Runtime

The longest batch jobs

Before

17 hours

Now

15–20 minutes

Each track is the figure from before the rebuild. Hover a Now bar for its value.

The customer

About AEP

American Electric Power (AEP) is one of the largest utilities in the United States, and it runs a multi-petabyte data lake that powers its interval-metering analytics.

Customers About 5.6 million
Service area 11 states
Industry Energy · Utilities
Pipeline in scope Non-VEE interval metering

AEP at a glance. Source: AEP

The challenge · 01

The problem

AEP’s non-VEE interval-metering pipeline had two problems: how it ran, and how it had been handled so far.

The symptoms 17-hour jobs and failed weekends Batch jobs ran for 17 hours, and the weekend batches failed outright.
The earlier approach Months of tuning, no redesign For 9 to 10 months the previous team patched EMR configuration while the architecture stayed as it was.

Three goals for the engagement

AWS brought Mactores in with these three goals.

Find the root cause that some 18 months of earlier effort had not.
Rebuild the pipeline around its real workload.
Reach production with at least 30% lower cost and at least 50% better performance.

[Quote 1: what Linden Lab needed from AI, and why it chose Claude through Mactores given its commitment on resident content]”

How Mactores delivered · 02

AI agents found the fault in hours; engineers rebuilt the pipeline

Mactores ran the work on its agent-native delivery model, with AI agents on the diagnosis and forward-deployed engineers on everything that followed.

The diagnosis

Mactores AI agents

Under 2 hours

HiveQL on HDFS was structurally wrong for this workload.

Not an EMR version bug

The rebuild

Forward-deployed engineers

Layer	Before	Change	After
Compute	Amazon EMR	→	Amazon EMR Kept
Job code	HiveQL	→	PySpark
Storage	~~HDFS~~ Retired	→	Apache Iceberg on Amazon S3 In production

Mactores’ agent-native delivery

What the AI agents found, and what the engineers changed because of it.

The work

The AWS services behind the rebuild

This is Mactores’ Data Platform Modernization work in practice, with each service doing one job in the new pipeline.

Amazon EMR: Runs the pipeline’s PySpark jobs
Amazon S3 with Apache Iceberg: Stores the interval-metering tables
AWS Glue Data Catalog: Keeps the table definitions the jobs read
Amazon Bedrock AgentCore with Claude: Runs the AI agents that traced the fault

Delivery · 03

One week to a prototype, twelve to production

AWS and Mactores delivered the program jointly, from 22 February to 29 May 2026.

Thirteen weeks: week 1 for assessment, weeks 2 to 13 for the production build

Each segment is one week.

Phase 1 · Week 1

Assessment

AI agents: Root-cause analysis of the failing jobs
Engineers: A working prototype by the end of the week

Phase 2 · Weeks 2 to 13

Production build

Forward-deployed engineers

Architecture for the rebuilt pipeline
HiveQL rewritten as PySpark on Iceberg
Code validation
Cutover through DEV, UAT and production

In production · 04

The outcome

The rebuilt pipeline beat both targets set for it, and every production job ran green for the first four weeks after cutover.

Committed against delivered

Target	Committed	Delivered
Cost reduction	At least 30%	35%
Performance gain	At least 50%	17x faster

Day 1 EMR was trusted again as AEP’s analytics platform of record.