There is undoubtedly great value to be gained by linking the company's systems together and integrating them. The benefits are endless, including:
- - Less manual work to keep all systems up to date;
- - Process automation (e.g. automatic invoicing);
- - Better reporting and analysis (see data across systems).
One of the things to be aware of when planning integrations is the level of robustness required. By robustness, I mean the risk of failure: whether it goes down and how vulnerable it is. There's a difference between a monthly report, where everything must fit within two decimal points, and a live dashboard, where a small outage doesn't matter, as it is updated 2 minutes later.
Two Types of Integrations
There are generally two types of integrations, both of which have their pros and cons:
1. Point-to-point integration (typically with REST services)
2. Message-based (typically with a service bus)
The difference between the two is their robustness. Of course, there are also some financial decisions involved in this.
Point-to-point Integration
With point-to-point integrations (the most common integration, for e.g., webshops to accounting systems), you have an online system where you have access to its data (e.g., a webhook or an API). This system is one of the points in the "point-to-point" - e.g., a webshop. So, the other point is the system you want to integrate with; this could be an accounting system.
You can connect these two points.
Typically, you develop a small piece of code in the middle to translate from one system to another, and you can synchronize this at regular intervals (such as every hour). In basic terms, the software supplier arrives with a glue gun and glues the systems together. It’s that simple. A piece of software is required to extract data from point 1, transform it, and insert it into point 2 again.
The benefits of this type of integration are countless, and for many companies, this is the only type of integration required.
Point-to-point integration, the grey circles and lines indicate several systems to be integrated. Each line is software integration.
The Pros and Cons of Point-to-Point Integration
| Pros | Cons |
|
- Quick and easy to develop and therefore also relatively cheap; - Relatively robust, all three points (point 1, point 2, and the glue gun in the middle) are often available at the same time; - Little or no business logic to consider; - High performance. |
(They typically only arise when you have other requirements, such as uptime if several more systems are introduced, or the data starts to grow).
|
Point-to-point integrations are a very fast and simple integration between two systems, and we do a lot of them at Median Aps. It is quick to develop and cheap to operate. This type of integration fits really well into smaller companies, e.g., webshops, where you need one or two integrations that connect the systems.
But it doesn’t take much before you have to consider another solution.
Message-based Integration
This type of integration has some different pros and cons compared with point-to-point integration. Generally, we can say that this type of integration is the "big brother" of point-to-point integration.
We recommend that if you start integrating your systems, a point-to-point solution may be a sensible first step in order to get a better overview of your requirements and a better idea of the benefits you’ll get from integrating. In short, you have to see if the business case holds. If this turns out to be a benefit for the company, you can easily move from a point-to-point integration to a message-based integration.
The latter works by two or more systems sending messages to each other, rather than directly integrating. The messages run through an advanced queuing system (don't worry, this is something the software developer takes care of), and at the other end of the queue (or multiple queues), there is a recipient who handles the message.
Picture it as a "straw", where there is a small piece of software at one end which keeps track of e.g., orders in the webshop, and every time a new order arrives, it picks up the order and adds a message to the queue. It is simple and has only one job to do. It increases the robustness of this part of the system. This means that the individual parts in contact with e.g., the webshop are extremely simple and therefore also more robust and easier to maintain.
At the other end of the queue, another small piece of software removes a message from the queue and inserts it into the other system. That is basically all that happens.
This "queue" in the middle is where all the benefits come from. It can contain a lot of messages, so it also acts as a kind of buffer so that the piece of software that inserts orders into the accounting system is not overloaded. This solution is somewhat more advanced, but it can solve virtually all the company's integration needs. The benefit is that you can have thousands of these queues, making it easier to have multiple integrations without increasing the complexity. This integration method is very flexible and easy to correct, adapt and introduce new integrations into.
The Pros and Cons of Message-based Integration
|
Pros (Many of the pros of this type of integration have something to do with scaling your integrations, both in relation to the number of systems you connect and the amount of e.g., orders you have in the system.) |
Cons (Many of these cons need to be seen about the simple integration we talked about in point-to-point integration.) |
|
|
- The queue is the bottleneck (most queues are in the cloud and as such are very robust) - this is typically never a problem; - More development and maintenance required (the value is also greater) - it is not "off the shelf"; - Slightly less performance (especially in situations with a lot of business logic); - Eventually consistent – accept that all data is not immediately updated, but it will be over time (eventually consistency).
|
Adapting to Message-Based Integration
Not all systems know about messaging; some systems may use other types of integration methods, and others may not be designed for integration.
In the (albeit simplified) diagram above, the ERP System or whatever other systems we base our day-to-day business, have knowledge about the message queue. This is not correct - it's designed to simplify it a bit. We can't possibly know which systems are best for our company (now? In the future?), they may change or evolve over time, and it is not certain that the next version/system fits the integration we already made. In these cases, you insert a small "adapter" between the source system and the message queue. This means that this small adapter is the one that can connect to source systems and has knowledge about the bus.
This has several benefits, including that (as a company) we start to build an enterprise architecture, which in principle can be independent of source systems (to prevent vendor-lock-in). Long-term, the goal might be to be completely independent and choose based on what is best for the company (that in itself is a long discussion about how to select systems and to a certain extend being agnostic in your architecture).
The job of an "Adapter" is to create the actual integration. When you can't guarantee that the source systems have some sensible "handles" to use to get data from (e.g. API, or services), you sometimes have no other options. e.g. to download from the system database or "screen-scraping". The latter options are far from the most optimal, but this may be the only solution if, for example, you have legacy systems that need integrating, or you retrieve data from the web that is not available in other ways.
The benefits of having this "adapter" are that you get a looser connection between the source systems and the other systems in your company, as they don't have to know each other. It also means that we start to build a strong enterprise architecture in the company to integrate all our systems. This enterprise architecture can give us a competitive advantage, as our competitors cannot copy this. It is unique to our company (all competitors can easily buy the same source systems, but the integration and knowledge in the integration - which is now increasingly reminiscent of a Service Oriented Architecture (SOA) - cannot be copied). In this case, it will also be easier to grow and replace a system in the long run. You' simply' replace the adapter and then all the systems run again.
More details on the message-oriented integration. Now with adapters.
Typical Integration Challenge in the EAM Industry
Many companies in the EAM space struggle with scattered, disconnected systems - it’s not unusual to see maintenance teams using one tool, finance another, and operations relying on spreadsheets or manual workarounds.
These disconnected setups make it hard to get a clear, real-time picture of what’s going on and often result in duplicated data, missed information, or unnecessary delays. The real challenge is connecting the systems after they’ve already been set up independently.
Without a well-thought-out integration approach, even the best EAM system can fall short of its potential. That’s why building integration into your ERP from the start, rather than bolting it on later, can make all the difference.
Recommendations
If you are in the process of integrating systems or are investigating whether there is any value in it, start by making simple point-to-point integrations.
Many companies never need to do anything other than this, as it covers all needs and generates the necessary value in the company. As you grow, you can then consider whether to go in a different direction and look at more advanced types of integration.
One of the biggest challenges we see companies experiencing is making ad-hoc integrations without really having any control over them. This means that no one knows what integrations have been made (the one who made them stopped working for the company 3 years ago). And no one knows if there are several similar integrations, there is also no one who dares touch the systems, least of all replace the old accounting system (e.g. C5/AX/NAV).
Our advice - be proactive:
- - Buy/choose systems that can actually integrate with others – favor strong APIs
- - Decide on integrations before others do it for you
- - Think out of the box - did you know e.g. that you can actually use RPA to make integrations?
- - Think more out of the box - now that you’re extracting data anyway, why not put it in a database so you can generate reports and analyses (Business Intelligence) e.g. with Microsoft Power BI?
This article is inspired by an article written by Brian Hvarregaard, owner of Median - a software development and consulting company based in Aalborg. You can find the original article here.
