A data engineer must ingest a source of structured data that is in .csv format into an Amazon S3 data lake. The .csv files contain 15 columns. Data analysts need to run Amazon Athena queries on one or two columns of the dataset. The data analysts rarely query the entire file. Which solution will meet these requirements MOST cost-effectively?
- AUse an AWS Glue PySpark job to ingest the source data into the data lake in .csv format.
- BCreate an AWS Glue extract, transform, and load (ETL) job to read from the .csv structured data source. Configure the job to ingest the data into the data lake in JSON format.
- CUse an AWS Glue PySpark job to ingest the source data into the data lake in Apache Avro format.
- DCreate an AWS Glue extract, transform, and load (ETL) job to read from the .csv structured data source. Configure the job to write the data into the data lake in Apache Parquet format. (correct answer)
Reveal answer & explanationHide answer
The correct answer is D. Option D: Create an AWS Glue extract, transform, and load (ETL) job to read from the.csv structured data source. Configure the job to write the data into the data lake in Apache Parquet format.
Explanation
AWS Glue is a serverless ETL and data-catalog service that prepares and integrates data. AWS Config tracks resource configuration changes and evaluates compliance. This option delivers the requirement at the lowest cost.