Including column headers in CSV export from RDS postgres db using AWS Data Pipeline?

Home / Uncategorized / Including column headers in CSV export from RDS postgres db using AWS Data Pipeline?

Question:
I set up a data pipeline that dumps data from our RDS postgres db to a S3 location but the column headers are not included. I would like to avoid using the solution outlined here because it seems brittle (if the columns change, also want to avoid casting columns as string). Thanks in advance


Answer:
I’m not sure how do you set the pipeline (you did not describe it), but the standard way of including column headers – with bare PostgreSQL – is
COPY (SELECT some,fields FROM any.table ORDER BY something) TO STDOUT WITH (FORMAT csv, HEADER on);

It might be tricky to receive such stream, not all DI tools support it – but the engine will generate it in proper CSV format.
Read more

Leave a Reply

Your email address will not be published. Required fields are marked *