The Problem
RWE's energy services platform depended on legacy Java components that were difficult for the team to extend. Energy market data had to be queried, filtered, and exported daily. Large dataset exports took 45 minutes, and response times were slow enough to affect day-to-day analyst workflows.
The business needed faster data access and a codebase a small team could evolve safely. My work focused on the Node.js/NestJS replacement path, the data filtering pipeline, and the tests needed to keep migration work controlled.
How I Approached It
We moved replacement work into Node.js with NestJS and MongoDB. Energy projects have dynamic properties, so the filtering system needed to generate columns from each project's characteristics. I built and tuned this path with MongoDB aggregations, using profiling to find the expensive parts before changing them.
The biggest technical constraint was heavy processing on the main thread. I implemented Worker Pools to move large dataset operations into parallel work, which brought export time from 45 minutes down to 12. The profiler helped separate real bottlenecks from guesses.
I also introduced TDD early for the features I owned. On a migration project, tests are not ceremony; they are how you avoid replacing known behavior with subtle regressions. That discipline became a useful bridge into later teams where high coverage was already expected.
Key Technical Decisions
Worker Pools for Exports
Large dataset operations were blocking the event loop. Worker Pools moved heavy processing to parallel threads, keeping API work responsive while exports ran separately. 45min became 12min.
Profiler-Driven Optimization
I used the Node.js profiler to identify where time was spent in the column filtering pipeline, then optimized the MongoDB aggregations that were actually responsible for the slowdown.
Dynamic Column Generation
Energy projects have variable properties. The filtering system generates columns dynamically based on project characteristics — flexible enough for any project shape, fast enough for daily use.
TDD from Day One
On a migration project, replacing working behavior is the risk. I used test-driven development for the features I owned so changes could be reviewed against expected behavior instead of intent alone.
Business Impact
The 65% API improvement made market-data workflows faster for users who queried the system daily. The export-time reduction changed exports from an overnight-style task into something analysts could run during the workday. The NestJS structure also gave the 4-person team a clearer path for follow-up work.
For a contract buyer, the relevance is direct: this is the kind of messy, high-context backend work where a senior engineer needs to improve the system without stopping day-to-day delivery.
What I Took Away
This project taught me the value of consistency in delivery. On a small team replacing a production system, there's no room for erratic output. Every week needed to move the migration forward predictably. I learned to scope work tightly, ship reliably, and communicate progress clearly.
Technically, it deepened my conviction that performance optimization starts with measurement. The profiler-first approach — finding the real bottleneck before writing a single optimization — saved weeks of wasted effort and became a permanent part of how I work.