I set up a data pipeline that dumps data from our RDS postgres db to a S3 location but the column headers are not included. I would like to avoid using the solution outlined here because it seems brittle (if the columns change, also want to avoid casting columns as string). Thanks in advance
I’m not sure how do you set the pipeline (you did not describe it), but the standard way of including column headers – with bare PostgreSQL – is
COPY (SELECT some,fields FROM any.table ORDER BY something) TO STDOUT WITH (FORMAT csv, HEADER on);
It might be tricky to receive such stream, not all DI tools support it – but the engine will generate it in proper CSV format.