Data and Metadata Governance (Brainstorming)

Data silos are not good. They cause lots of problems like inconsistencies between different silos, increased data management costs, difficulty in connecting data across siloes, difficulty in understanding and using data, and other such issues.  In the new era of artificial intelligence, bad data can be particularly problematic.

Barc explains data governance thus:

Data Governance includes the people, processes and technologies needed to manage and protect the company’s data assets in order to guarantee generally understandable, correct, complete, trustworthy, secure and discoverable corporate data. The topics encompassed by data governance are shown in the following graphic provided by Barc and include:


Amazon Web Services (AWS) explains data governance thus:

Data governance includes the processes and policies that ensure data is in the proper condition to support business initiatives and operations. Modern organizations collect data from various sources at scale to enhance operations and service delivery. However, data-driven decision-making is effective only when data meets required quality and integrity standards.

Data governance determines roles, responsibilities, and standards for data usage. It outlines who can take what action, upon what data, using what methods, and in what situations. With more data being used to support artificial intelligence (AI) and machine learning (ML) use cases, it has become critical that all data usage meets regulatory and ethical requirements. Data governance balances data security with tactical and strategic objectives to ensure maximum effectiveness.

Data governance programs have historically been employed to lock down data in silos to prevent data leakage or misuse. However, the consequence of data silos is that legitimate users must navigate barriers to get access to data when they need it. Inadvertently, data-driven innovation gets stifled.

AWS provides this graphic which explains the parts of data governance:


Once you have your data governance down, then you need to start thinking about metadata governance. Secoda provides this explanation of metadata governance:

Metadata governance is a critical component of any successful data management process. Regardless of the size or scope of your data, metadata governance is essential for ensuring the accuracy, integrity, security, and privacy of your data. Metadata is considered the core of data governance, because it is the layer of information that provides an understanding of the structure, relationships, and function of the data.


Metadata governance is not about all information or about making all of your metadata publicly available.

There will always be information that is local, sensitive, highly specialized, or that for some other reason should not, or need not, be managed at an enterprise level. Metadata Governance is about forming a culture where high-value show information is consistent and shared across the organization. This is especially important at a time when content discovery technologies, such as voice search and voice assistants, are now metadata-driven.


Data democratization makes information in a digital format accessible to the average end user. The goal of data democratization is to enable nonspecialists to gather and analyze data without requiring outside help.

In the forthcoming age of artificial intelligence which will use data and metadata; all these ideas need to be applied to the important publicly available data and metadata for financial reporting standards such as US GAAP and IFRS.

A "public metadata governance framework" refers to a set of established principles, policies, and procedures that a public sector organization uses to manage and control its metadata, ensuring it is accurate, consistently formatted, accessible, and used appropriately to enhance data quality, privacy, and compliance within the public domain; essentially, it's a structured approach to managing data about data within a public entity, providing transparency and accountability to citizens regarding how information is collected, stored, and utilized.

Additional Information:

Comments

Popular posts from this blog

Microsoft CEO: "AI Agents will Replace All Software"

New Tool for Accountants, Auditors, Analysts

Named Graphs