CloudGuild · Blog · Cheat sheets · Lessons · Certifications
Navigating Data Governance in Databricks
Learn how to effectively log data access in Databricks to meet governance requirements. We break down the question and options for clarity.
Your organization is adopting data governance policies that require all data access to be logged for auditing purposes. How can you achieve this in Databricks?
The question
- A: Enable cluster logging in the workspace settings.
- B: Use Databricks' built-in audit logs.
- C: Implement a custom logging solution using Delta Lake.
- D: Configure job logs to capture access information.
Think before you scroll
When evaluating these options, consider which approach provides a comprehensive solution for logging data access specifically. The goal is to ensure compliance with data governance policies without unnecessary overhead.
The answer
The correct option is B: Use Databricks' built-in audit logs. This choice directly addresses the requirement for logging all data access for auditing purposes. The built-in audit logs automatically capture relevant activities, making them the most effective solution for this scenario.
Why the other options lose
- A: Enable cluster logging in the workspace settings. While cluster logging may offer some insights, it does not specifically focus on data access events. This option lacks the specificity required for auditing data access comprehensively.
- C: Implement a custom logging solution using Delta Lake. This option requires additional development and management efforts. It does not guarantee that all data access is logged automatically, which is essential for compliance.
- D: Configure job logs to capture access information. Job logs provide information about job executions but do not focus solely on data access. This means they may miss key events necessary for thorough auditing.
The concept behind it
The principle at play here is the importance of using built-in features that cater to specific compliance needs. Audit logs are designed for this purpose, ensuring that all data access is tracked without requiring extra configuration or management. Understanding how Databricks structures its logging capabilities can help you choose the right method in various scenarios.
Exam trap to remember
Always opt for built-in solutions for compliance requirements. They save time and reduce the risk of missing critical data access logs.