Research-grade data needs research-grade infrastructure. The team had a script. We replaced it with a pipeline that can be rerun in a single command and produces versioned, schema-validated output every time.
Pipeline output schema The challenge
PCU researchers needed clean, structured OJK BPR data on a repeating cadence. The existing approach was a one-off scrape that broke whenever the source changed.