Sunday, November 6, 2011

Introducing Expert Systems and Distributed Architecure

Expert Systems.
The enterprise is typically composed of expert systems that perform core business functions and perform them well.  A retail system consists of points of sale systems, e-commerce websites, inventory management systems, accounting systems, marketing systems, customer service systems, advertising systems and business analytics.  Each supports a core function, and the primary business process ties them all together to execute its main use case.

In addition to these business expert systems, there are a number of process oriented expert systems that take care of security, notifications, logging, search, exception management, monitoring, management etc.  

In a distributed environment, the most common way to interact with expert systems is using webservices, file transfers, shared databases, or including references to expert components in an application.  The problem with most of these methods is that they strongly couple the systems to each other.  The webservice has to be available when the application needs it and if it's not there, can cause cascading failures downstream.   These failures can lead to data loss or require the development of expensive, complex, redundant systems that can handle these failures and recover at later times.  Managing changes to expert components that are referenced by other applications is very difficult, often requiring versioning when interfaces change, rebuilding dependent applications to accommodate changes, or having system wide downtimes to upgrade just one expert system.  In an enterprise, this can get out of hand very quickly.
Ideally you want to loosely couple these expert systems.  In addition to using webservices or referencing expert components directly in an application, we can add advanced messaging to our arsenal and add some stability to the system. Messaging has been around for a very long time and has a wide and varied implementation base, but it's features are limited. Advanced messaging adds several important features that change the way we use messaging, it's basically messaging re-engineered from the bottom up.

One important feature of the advanced messaging protocol is that it applies the publisher/subscriber pattern to message queues.  In the typical queue architecture an application puts a message on the queue specifically for another application who picks it up and processes the work, responding perhaps on another queue.  In the exchange model, an application puts a message in an exchange and provides a routing topic (publishing).  An application can listen to an exchange using a specific subscription key  (subscribing).  The exchange acts as a broker and matches routing keys to subscription keys.  When a match is made the message is moved to as many subscribers as match the routing key.  This mechanism does one very important thing.  It completely decouples the publisher from the subscriber.  This has large implications for how the enterprise can be organized into expert systems.  
Distributed Systems
The diagram above shows an example ecommerce website taking an order and fulfilling it with integration points between accounting, marketing and inventory management systems.   
When you compare advanced messaging to standard message queues, the main difference is the addition of an exchange.  On the surface this may seem like a small and insignificant change, but that one addition fundamentally changes the way we can architect the enterprise.  Perhaps the most complex of software systems, rich graphic, ui driven applications are all event driven.  The event driven systems allows various components in the UI landscape to change as the user changes the way they use the application, creating a very rich satisfying experience.  By comparison, enterprise systems have typically been standalone applications that communicate with other systems in a very linear, stochastic way where the weakest link truly defines the stability of the entire system.  By adding exchanges to the message bus, we've introduced an event model to the enterprise.  This will allow us to create a simple network of loosely coupled expert systems that is more stable and easier to manage that what we have had to date.  The following entries will explain how to build such a system with advanced messaging at its core.

No comments:

Post a Comment