This is the story of why the current AI boom was inevitable from the very core of engineering and information technology.
If you think about it, AI was inevitable, it all boiled down to having an underlying system supporting faster development. It began a long time ago, in a time when engineers were designing physical objects. In the ’60s or in the ’70s when software was only supporting number crunching for accounting and not everything that moves or interacts.
There has been a series of underlying systems competing with efficient solutions. Let me define what I mean by underlying system. It is lower level abstract machine, that is designed to interpret or translate a given solution to a problem and make it more efficient.
From that definition the first subject of this writing is obvious, it is the SQL query optimization engine, which is nearly the same age as the C compiler. Both boasting more than 50 years of experience of supporting higher level abstractions.
In the beginning these systems were behind, because they introduced a new level of abstraction, a ton of compute and complexity. But as SQL refused to retire, the query optimization, behind a declarative style is no small feat. An additional example is C++’s optimizer, based on a recent podcast the time invested in a hidden layer was well worth it.
The Boar book also talks about this tendency to spend time and effort with a system that will reduce the complexity down the road, and this is what SQL engines excel at, and why they rule the database world. This got me thinking.
Let’s suppose that a task can be more efficient someday if we build THE TASK SOLVER. This machine, this task solver is a general purpose faster-maker, built not for this task, but for helping the solver think about fewer things. The task solver therefore is modular and stackable. You can have multiple task solvers in a row, similar to an automation pipeline, to solve this current task very fast.
But building the task solver is a task in itself. So when we built the task solver, we created the task solver WITH the task solver. Now we have an exponential curve of development. This is a trend in all engineering. A well engineered solution has a well defined contract, API doc or a screw’s thread size. On top of the contract we have the single responsibility principle in action e.g.: you CAN hammer on a screw, but it will honestly fight back. It is inert, it will work in a specified place, but the user will define how it will be useful.
This comparison breaks down when we try to push it more, in the physical world you cannot make an engine that will help you create a better engine by eating the blueprint. But in information technology you can. This is the game changer, all the task solvers that work in the background, mature and reliable solutions off the shelf.
IT has been working on this technology for decades now. It is second nature to have a “make everything better” machine. That is why I believe that thought of having an even higher level of abstraction – only prompts for input is really a normal change in computer science. AI was not new, it was just like the SQL query optimizer but declarative in a new sense. Less code and more definition. In a sense AI was not a stranger, but a new task solver. This is why I think AI was inevitable.