Problem Solving Systems

The world is complex.  But sometimes humans can come together and agree enough to create something useful in meeting some important aim and satisfying some agreed upon set of goals and objectives.

When trying to solve a problem, systems thinking is useful. Systems thinking is a discipline for seeing wholes. It is a framework for seeing interrelationships and patterns.

Irreducible complexity (a.k.a. essential complexity) is a term used to describe a characteristic of complex systems whereby the complex system needs all of its individual component systems in order to effectively function. 

In other words, it is impossible to reduce the complexity of a system (or to further simplify a system) by removing any of its component parts and still maintain its functionality objective because all those component parts are essential to the proper functioning of the system. So for example, consider a simple mechanism such as a mousetrap.

That mousetrap is composed of several different parts each of which is essential to the proper functioning of the mousetrap per the use case of the mouse trap to catch mice: 

  • a flat wooden base, 
  • a spring, 
  • a horizontal bar, 
  • a catch bar, 
  • the catch, 
  • the bait,
  • and staples that hold the parts to the wooden base.
If you have all the parts and the parts are assembled together properly, the mousetrap works as it was designed to work.  You can say, and show via testing, that the mouse trap functions effectively.

But say you remove one of the parts of the mousetrap.  The mousetrap will no longer function as it was designed to function, it simply will not work.  That is irreducible complexity: the complexity of the design requires that it can't be reduced any farther without losing required functionality.

There are different types of essential complexity a.k.a. irreducible complexity a.k.a. inherent complexity.  By distinguishing between the different types one can better understand a system:

  • Organized simplicity
  • Unorganized complexity
  • Organized complexity

The same thing is true for a problem solving system.  Whether the functionality is provided using computer based functionality or manual effort; all the pieces need to exist in some form in order for the problem solving system to function effectively.  You could have specific levels of functionality.  For example, in financial reporting there is a difference between a "compilation" and a "review" and an "audit" of a financial report.

There is a difference between a "complex system" and a "simple system that is complicated".  Most software applications tend to be simple systems that may be complicated.  Fred Brooks No Silver Bullet points out that complexity comes in two forms: essential complexity and accidental complexity.  Essential complexity is what is irreducible.  Irreducible complexity = essential complexity.

A kludge is an engineering/computer science term that defines what is best described as a workaround or quick-and-dirty solution that is typically clumsy, inelegant, inefficient, difficult to extend and hard to maintain; but it gets the job done. By contrast, elegance is beauty that shows unusual effectiveness and simplicity. The nautical term for this is jury rig.

The alternative is elegant simplicity.

It is easy to create a problem solving system that is complex.  It is hard work to create a problem solving system that is simple.  People commonly confuse the terms "simple" and "simplistic". Simplistic is dumbing down a problem in order to make the problem easier to solve. Simplistic involves removing essential complexity in order to make a system easier.  Simple is something that is not complicated, that is easy to understand or do. Simple means without complications. When something is simple it looks elegant.

The challenge when constructing a problem solving system is making sure you avoid or eliminate the accidental complexity. Accidental complexity refers to challenges someone unintentionally make for themselves as a result of trying to solve a problem.

The Law of Conservation of Complexity states that, "Every software application has an inherent amount of irreducible or essential complexity. The question is who will have to deal with that complexity: the application developer, the platform developer that the software runs on, or the software user."

The Law of Requisite Variety states that in order to effectively regulate a system, the regulator of that system must possess a sufficient range of actions to counter act the variety of important potential disturbances that system might encounter.  The principle of requisite variety ensures that the system's internal state remains as close as possible to the desired goal state of the system.

The Cynefin Framework provides a tool for understanding complex knowledge: (organized simplicity and organized complexity)

  • Best practice (obvious, clear)
  • Good practice (only obvious if you have the right skills and experience, complicated)
  • Emergent practice (tend to have to have more skills and experience, then can use principles to group alternatives, complex)
  • Novel practice (tends to be unique, but describable, chaotic)

Not every system is computable. Systems can be categorized into two groups: non-complex systems which are computable and complex systems which are not computable.  Computability is the ability to solve a problem.  Intelligence is used to achieve computability. Artificial intelligence is getting a machine to perform tasks that a human would otherwise perform.

These, the graphic below, seem to be the pieces of a problem solving system which are essential.  What I have tried to do is use the same framework that was used for articulating the Semantic Web Stack or "layer cake" and generalized the pieces necessary for a problem solving system. This is what I came up with: (for more information see Smart (Cognitive) Business Applications and Services)

So, each of those pieces must exist in order to get an effective result.  If the piece is not provided within software applications, then the task or process must be performed manually.  You cannot simply just leave something out.  All this is driven by the aim of the problem solving system; what do you want the problem solving system to do. The stakeholders of the system specify goals and objectives that the system needs to be able to perform (i.e. the aim of the system).

Now, you can implement such a problem solving system using the semantic web stack, using the XBRL technology stack, or some other technology stack.  There tends to be three primary problem solving system implementation approaches:

  • Semantic web stack
  • Graph database
  • Logic programming
This graphic shows the three approaches and makes an important point.  Providing a high-level logical model, such as the Standard Business Report Model (SBRM) or the Seattle Method is an approach to enabling the conversion between these three technical implementation approaches:

When you think about this, being able to convert bidirectionally from any one technical implementation approach to the other technical implementation approaches (given the appropriate knowledge representation power); while the technical format and other technical implementation details may be different, the LOGIC of the information conveyed by the system should not change at all.

Why is this important?  Multiple technology stacks is a fact of life.  Different enterprises will implement different technology stacks internally.  But in order to exchange information effectively, information from one technology stack needs to be converted to some neutral standard or into some other technology stack in order to exchange information effectively and not create an integration hairball.

To make something like "algorithmic regulation" work, either: (a) everyone needs to convert to exactly the same technology stack internally within their enterprise or, (b) reliable conversion between different technology stacks needs to be created. Option "(a)" will not happen.

Additional Information:

Comments

Post a Comment

Popular posts from this blog

Relational Knowledge Graph System (RKGS)

Image
Relational Knowledge Graph System (RKGS) is an interesting idea that combines the relational paradigm and the knowledge graph paradigm.  This documentation, Why RKGS , explains the idea. This gets even more interesting with the notion of "relational AI".  RelationalAI is described as follows by the company of the same name, Relational.ai : “RelationalAI is a cloud-based relational knowledge graph management system, with state of the art probabilistic processing and declarative reasoning at scale to make developing Data Applications a superpower for your business.” This notion of knowledge graphs represented in relational databases fits into my belief that knowledge graphs have two aspects: the expression SYNTAX and the logic of what is expressed or SEMANTICS. While the LOGIC of a knowledge graph is always the same regardless of technical syntax used to express that logic; there are many, many different technical syntaxes that can be used to represent a knowledge graph. There
Read more

Getting Started with Auditchain Luca

Image
(Note that a newer, production version as opposed to only the development version , of Luca Suite is available) The Innovator's Dilemma points out that disruptive innovations often look like toys that are doing everything wrong. Auditchain Luca , which is part of the Auditchain Suite , is a unique approach to creating a knowledge graph .  To experience using Auditchain Luca, have a look at the Getting Started with Auditchain Luca instructions. Be sure to grab the Excel and JSON import files used for this getting started exercise.   This getting started exercise uses one of several different approaches to getting information into a tool for constructing and viewing professional quality knowledge graphs .  Additional getting started guides will be made available over time. Auditchain Luca might not make sense to you if you don't understand the paradigm shift that is occurring and if you have not shifted your mental map. For the next step in your journey, consider trying creat
Read more

Evaluating the Quality of XBRL-based Financial Reports

Image
From the time public companies started submitting XBRL-based financial reports to the U.S. Securities and Exchange Commission, I have been looking at those reports, trying to understand the XBRL-based reports, and trying to figure out how to get them right.  That information was used to figure out how to create software applications that help professional accountants get these reports right. Here is information that summarizes my testing and results which I stopped doing March 31, 2019: Analysis of the high level accounting relationships in the 10-Ks of about 5,716 financial reports  (5,063 public companies, about 89.1% got all these high level accounting relationships consistent with expectation, 623 were inconsistent and manual confirmation revealed the inconsistency was an error) Documentation that helps you understand 26 different types of errors in XBRL-based reports High quality documentation of hundreds of errors in XBRL-based reports submitted by public companies to the SEC Doc
Read more