A company is collecting a large amount of data from a fleet of IoT devices. Data is stored as Optimized Row Columnar (ORC) files in the Hadoop Distributed File System (HDFS) on a persistent Amazon EMR cluster. The company's data analytics team queries the data by using SQL in Apache Presto deployed on the same EMR cluster. Queries scan large amounts of data, always run for less than 15 minutes, and run only between 5 PM and 10 PM. The company is concerned about the high cost associated with the current solution. A solutions architect must propose the most cost-effective solution that will allow SQL data queries. Which solution will meet these requirements?
- AStore data in Amazon S3. Use Amazon Redshift Spectrum to query data.
- BStore data in Amazon S3. Use the AWS Glue Data Catalog and Amazon Athena to query data. (correct answer)
- CStore data in EMR File System (EMRFS). Use Presto in Amazon EMR to query data.
- DStore data in Amazon Redshift. Use Amazon Redshift to query data.
Reveal answer & explanationHide answer
The correct answer is B. Option B: Store data in Amazon S3. Use the AWS Glue Data Catalog and Amazon Athena to query data.
Explanation
Amazon S3 provides durable, scalable object storage that is fully managed. Amazon Athena runs serverless SQL queries directly on data in Amazon S3 with no infrastructure to manage. AWS Glue is a serverless ETL and data-catalog service that prepares and integrates data. This option delivers the requirement at the lowest cost.