CloudGuild · Blog · Cheat sheets · Lessons · Certifications
AWS Certified Data Engineer - Associate (DEA-C01) Exam Question Walkthrough
Explore a common AWS DEA-C01 exam question about Kinesis Data Streams, its options, and why the right choice matters.
When it comes to managing streaming data in AWS, many candidates stumble on how to ensure data integrity with Kinesis Data Streams. This question tests your understanding of data consumption and processing efficiency. Let’s break it down.
The question
Your team is using Amazon Kinesis Data Streams to collect and process streaming data. What is the best way to ensure that your consumers can handle the data without losing any records?
A. Use a single consumer to read all the data.
B. Increase the number of shards in the stream.
C. Implement a retry mechanism for consumers.
D. Use Amazon SQS to buffer the messages.
Think before you scroll
Consider how data is processed in Kinesis. Each option offers a different approach to managing data flow. Focus on scalability and data handling efficiency when making your choice.
The answer
The correct option is B: Increase the number of shards in the stream. This approach allows for parallel processing of data, enabling multiple consumers to read from different shards simultaneously. This setup reduces the risk of losing records, as it enhances the system's ability to keep up with incoming data rates.
Why the other options lose
A. Use a single consumer to read all the data.
This option creates a bottleneck. A single consumer cannot keep pace with a high volume of incoming data, increasing the likelihood of data loss.
C. Implement a retry mechanism for consumers.
While retries can help recover from transient errors, they do not solve the underlying issue of data lag. If consumers can't keep up, retries won’t prevent data loss.
D. Use Amazon SQS to buffer the messages.
Introducing SQS adds unnecessary complexity to the Kinesis data flow. Kinesis is designed to handle streaming data directly, and using SQS is not a straightforward solution for managing Kinesis data.
The concept behind it
The key principle here is scalability. By increasing the number of shards, you distribute the data load across multiple consumers. This parallel processing capability is essential for maintaining data integrity in high-throughput scenarios.
Exam trap to remember
Remember: More shards equal better data handling. Always consider how to scale consumer capacity when working with streaming data.