Quality, Quality, Quality: The Status Quo is Doomed

This article, GPT-4 has a trillion parameters, helps one understand how good ChatGPT can be currently and how to make it better.  It also helps one recognize how customizable or custom LLMs will be used in the future.

The US GAAP XBRL Taxonomy published by the FASB could be an enabler for a rules-based reporting system.  But it is not currently because it does not represent the necessary rules correctly or completely.  What that XBRL taxonomy is, effectively and admittedly, is a human readable "pick list".  So a rules based system would have a hard time using that US GAAP XBRL Taxonomy or reports created using that taxonomy.  Similarly, all of the report in the SEC’s EDGAR system that use that US GAAP XBRL Taxonomy are not of the right quality for a rules-based system to make use of the reports effectively.

All of the above is similarly true of the IFRS XBRL Taxonomy and the financial reports submitted to the ESMA using that IFRS XBRL Taxonomy.

However, that information could be fed to a GPT-4 enabled system, creating a large language model (LLM) that is custom to US GAAP financial reporting and another perhaps separate model custom for IFRS financial reporting.  Both the SEC EDGAR system and ESMA report system are large enough (i.e. good model size) and an AI system can be trained using that information and that AI system would be helpful.  But, the resulting system can only be good as the quality of the financial reports.

But a LLM in natural language can never be as precise as a knowledge graph that has been created correctly and precisely by knowledgeable and skilled humans.  An autogenerated LLM that is better than a knowledge graph created by humans, in my view, is never going to happen and if it does, perhaps, happen it will be well into the future and it will be based on the rules-based knowledge graph constructed by knowledgeable, skilled human accountants that understand all those rules.  Effectively, the rules will be a part of the training data used by the LLM/GPT-4 based system. Knowledgeable, skilled humans using the right tools will always ultimately prevail where quality and precision matter.

HOWEVER, people will try the LLM approach, it will be better than nothing, and it will contribute to leading to a proper rule based model created by skilled, knowledgeable humans at some point in the future. So the less precise LLM/GPT-4 approach can help pave the way to rules based approaches.

All this stuff will “cycle” as people figure this out over the coming years.  The status quo is flat out doomed.

If you really want to understand the moving pieces of the puzzle, I would recommend really studying these three articles:

Your brain has different parts.  The part responsible for intuition is a different part than the part responsible for reasoning.  And so it will be for artificial intelligence. The artificial intelligence of your future will be a hybrid, like your brain.

As pointed out in that third article by Alan Morrison, who used to work at PWC, GPT-4 (LLMs, machine learning) and knowledge graphs (rules based) will be combined into hybrid solutions.  Something like Logical English would make this even better, more precise.  I suspect that people will try and make GPT-4 work and that will be their focus in the short term because it is easier for the user, but it is not precise enough for many things, its limitations will be understood, then rules based will be valued more, the rules will fill the gaps, and a hybrid system will serve humans well.

Rules-based systems are similar to the "reasoning" part of your brain. GPT-4 and other statically based machine learning is like your "intuition".

I suspect that each of the Big 4 CPA firms will create their own competing custom hybrid models.  Maybe not.  Maybe there will be a bunch more custom hybrid models that will be built.  Maybe others are seeing this even more clearly than I am and have even better ideas.  Perhaps.  But regardless, the status quo is definitely doomed.

As Jason Staats, CPA said in this video, Using AI To Prepare Tax Returns, "AI will  not put accountants out of work.  Accountants using AI will put accountants not using AI out of work."

Personally, I think the risk calculus changed because of GPT-4.

Additional Resources:

Comments

Post a Comment

Popular posts from this blog

Relational Knowledge Graph System (RKGS)

Image
Relational Knowledge Graph System (RKGS) is an interesting idea that combines the relational paradigm and the knowledge graph paradigm.  This documentation, Why RKGS , explains the idea. This gets even more interesting with the notion of "relational AI".  RelationalAI is described as follows by the company of the same name, Relational.ai : “RelationalAI is a cloud-based relational knowledge graph management system, with state of the art probabilistic processing and declarative reasoning at scale to make developing Data Applications a superpower for your business.” This notion of knowledge graphs represented in relational databases fits into my belief that knowledge graphs have two aspects: the expression SYNTAX and the logic of what is expressed or SEMANTICS. While the LOGIC of a knowledge graph is always the same regardless of technical syntax used to express that logic; there are many, many different technical syntaxes that can be used to represent a knowledge graph. There
Read more

Getting Started with Auditchain Luca

Image
The Innovator's Dilemma points out that disruptive innovations often look like toys that are doing everything wrong. Auditchain Luca , which is part of the Auditchain Suite , is a unique approach to creating a knowledge graph .  To experience using Auditchain Luca, have a look at the Getting Started with Auditchain Luca instructions. Be sure to grab the Excel and JSON import files used for this getting started exercise.   This getting started exercise uses one of several different approaches to getting information into a tool for constructing and viewing professional quality knowledge graphs .  Additional getting started guides will be made available over time. Auditchain Luca might not make sense to you if you don't understand the paradigm shift that is occurring and if you have not shifted your mental map. For the next step in your journey, consider trying creating a report using a financial reporting scheme template . If you are struggling to figure out how to register or
Read more

Evaluating the Quality of XBRL-based Financial Reports

Image
From the time public companies started submitting XBRL-based financial reports to the U.S. Securities and Exchange Commission, I have been looking at those reports, trying to understand the XBRL-based reports, and trying to figure out how to get them right.  That information was used to figure out how to create software applications that help professional accountants get these reports right. Here is information that summarizes my testing and results which I stopped doing March 31, 2019: Analysis of the high level accounting relationships in the 10-Ks of about 5,716 financial reports  (5,063 public companies, about 89.1% got all these high level accounting relationships consistent with expectation, 623 were inconsistent and manual confirmation revealed the inconsistency was an error) Documentation that helps you understand 26 different types of errors in XBRL-based reports High quality documentation of hundreds of errors in XBRL-based reports submitted by public companies to the SEC Doc
Read more