Competency Question

A competency question is a test question you write before building an ontology or a theory to make sure the ontology/theory will be able to answer the kinds of questions the stakeholders of a system care about and need the ontology/theory to answer. A competency question is both a specification for what needs to exist in an ontology/theory and an acceptance test for the constructed ontology to make certain epistemic risk is minimized.

A competency question is a mistake proofing tool.  A competency question is a lot like a unit test and the notion of extreme programming which is referred to as test driven development. The use of competency questions is an ontology engineering best practice.

I became aware of the notion of the competency question from the article The Question is the Contract by Jessica Talisman.  I became aware of extreme programming and Agile software development probably 20 years ago from a software engineer. The importance of these proactive approaches I learned in college when I was studying for my MBA where I learned about what is currently referred to as Lean Six Sigma. All these things are part of what people refer to as "Agile Accounting".

The fundamental idea is that it is better to spend the time to build a system correctly than it is to not do so and then to have to fix mistakes or deal with the ramifications or fall out of mistakes made. This video explains why all this is important, understanding the true cost of quality.

Accounting can learn and benefit a lot from Lean Six Sigma, Agile software development, and ontology engineering best practices such as the competency question. In particular, new modern accountancy needs to be built out embracing these sorts of ideas.

Ask yourself this question related to traceability and provenance considering the total cost of quality:  In your current accounting system, can you follow a financial transaction all the way back to the business event that spawned the transaction and then forward again to the exact financial statement line item where it ultimately appears? And can you also start at any financial statement line item and drill down to the full set of underlying business events and transactions that roll up into that reported amount? If you can’t move effectively and confidently in both directions; from summary to detail and from detail back to summary; what exactly is preventing that?

Semantic fragmentation refers to the phenomenon where the meaning, definitions, understanding, or context of information become scattered, inconsistent, incomplete, or siloed across different systems, teams, software applications, or spreadsheets.

Think about something.  Gartner points out that the average Fortune 1000 company uses 800 spreadsheets to create the compliance report ultimately submitted to a regulator. Those 800 spreadsheets are connected by what some refer to as a "bucket brigade" of human and computer based processes. Traditional electronic spreadsheets are known for their propensity to contain errors; this is well understood.

This situation is so bad that the United States Congress passed the Sarbanes Oxley Act which in part requires public companies that report to the Securities and Exchange Commission to explain their systems. Sarbanes Oxley addresses the symptoms that cause this problem.  But Sarbanes Oxley does nothing to address the conditions which cause the symptoms.

These data and information related issues are not unique to financial accounting and reporting.

This situation is getting so bad, some want to change the "1-10-100" rule to the "10-100-1000 Rule". Particularly in this coming age of artificial intelligence, the failure to address data and information quality issues can be extremely costly and lead to slow responses to data/information requests because they have to perform what amounts to janitorial work such as overcoming that semantic fragmentation.  Even worse; bad data/information might be used unknowingly resulting in flawed understanding resulting in flawed insights and ultimately to very costly flawed decisions.

Proactively applying tools such as competency questions and other such practices when constructing things like ontologies or theories is sometimes caused by the false belief that those proactive steps "cost too much".  But most of the time it is simply ignorance that the technique even exists.

The paper Discerning and Characterising Types of Competency Questions for Ontologies, categories competency questions into five main types which are explained in simple terms below and describe and  help evaluate the completeness of an ontology or theory which is being held out to effectively describe an area of knowledge (a.k.a. area of interest, community of practice, collective, field, domain, subject domain, universe of discourse, society): 

• Scoping (SCQ): What "things" belong in this subject domain?
• Foundational (FCQ): What types and parts of things belong in the subject domain? (a.k.a. classification)
• Relationship (RCQ): Clarify the nature of the connections between the things, types, and parts.
• Metaproperty (MpCQ): What essential properties do things have?
• Validating (VCQ): Does the subject domain know what it is supposed to know and has that knowing been proven using unit tests?

My Seattle Method covers pretty much all those characteristics and a few more specific to financial reporting.

Additional Information:

• Agile Accounting
• Rethinking and Improving the Bucket Brigade
• Epistemic Risk
• Models
• Elements of Logic for Accountants
• SKOS, SBRM, XBRL