Stories · TMEGS
Plume cuts analytics costs 58% by moving off Databricks to an AWS lakehouse
- Data Platform
- TMEGS
- Amazon EMR on EKS
- Delta Lake
- Lake Formation
- SageMaker Unified Studio
Workspaces moved
13
production
7
non-production
What Mactores delivered for Plume
- The figure is an average taken across every workspace Plume moved
- No workspace had to fall back to Databricks after its cutover.
- Fourteen weeks for production, four for everything else.
- There was no Mactores-run transition period after go live.
Before
$1,513K
projected six-month run-rate on Databricks
After
$635K
six-month run-rate on the AWS lakehouse
Saved
$878K
projected over six months
$1,513K on Databricks
Each track spans $1,513K. Hover the After bar for the value.
The customer
About Plume
- Headquarters Palo Alto, US West
- Industry Telecom · ISP tech
- Reach Hundreds of operator deployments, worldwide
- Before the move 20 Databricks workspaces
The challenge · 01
The problem
- Correctness Every number had to survive the move A figure that changed in transit would surface in front of the ISPs Plume serves.
- Operational readiness Someone had to run it on day one A new platform would only pay off if Plume’s own team could operate it the moment the workloads switched.
Three conditions on the move
- Migrate all 20 workspaces end to end, production and non-production alike.
-
Validate parity before cutover, to take the risk out of each switch.
-
Hand Plume AWS-native runbooks it could use from the first day after cutover.
How Mactores delivered · 02
An agent took the volume work; engineers made the calls
Mactores ran the move on its agent-native delivery model: the agent and the engineers took turns on every workspace until it was running on AWS.
-
1Agent
Step 1, agent: Discovery
Maps what each workspace holds before anything moves
-
2Engineers
Step 2, engineers: Architecture
Designs the target lakehouse on AWS
-
3Agent
Step 3, agent: Conversion
Rewrites workloads for Amazon EMR on EKS
-
4Agent
Step 4, agent: Parity validation
Compares results on AWS with Databricks
Parity validation A workspace moved only after its outputs on AWS matched its outputs on Databricks. -
Parity gate No cutover without a match
- Agent track Mactores migration agent
- Engineer track 8 forward-deployed engineers
-
5Engineers
Step 5, engineers: Cutover
Switches each workspace once parity passes
-
6Engineers
Step 6, engineers: Hand-off
Turns the platform over to Plume’s team
Plume runs it from day one
- Amazon EMR on EKS
- Delta Lake on Amazon S3
- AWS Lake Formation
- Amazon SageMaker Unified Studio
The work
What runs Plume’s analytics now
This is Mactores’ Data Platform Modernization work in practice: Databricks out, AWS services in, each with one job.
- Amazon EMR on EKS
Runs the converted analytics workloads on Kubernetes
- Delta Lake on Amazon S3
Stores the tables in an open format
- AWS Lake Formation
Controls who can read which data
- Amazon SageMaker Unified Studio
Gives data teams one place to work
Delivery · 03
Production first, then the rest of the estate
-
01
Kickoff
Eight forward-deployed engineers and the migration agent begin the move
-
02
Phase 1: production
Thirteen workspaces that carry live analytics move first
-
03
Phase 2: non-production
Seven development and test workspaces follow on the proven path
-
04
Go-live
The last workspace goes live on the AWS lakehouse
Phase 1 · production Phase 2 · non-production
- Weeks
- 14 4 14 in phase 1 and 4 in phase 2, of 18
- Workspaces
- 13 7 13 in phase 1 and 7 in phase 2, of 20
In production · 04
The outcome
Production Non-production
20 of 20 · 0 rolled back
What we ship · Data Platform Modernization
Moving your own data platform to AWS?
Mactores plans and runs data platform moves on AWS, from the target architecture to the day your team takes over.
Curious about the migration agent behind this move? See Mactores’ AI agent expertise.