Before data could be exchanged electronically, businesses traded physical paper documents and the keen eye of business owners to capture, process, and deliver customer orders. The shortcomings of that process were too many, and may seem too trivial today. The first big breakthrough came in the form of electronic exchange of data, or popularly known as EDI (Electronic Data Interchange). EDI had the potential for a commercial application and soon had standards committees laying out rules for this electronic exchange of information. Standards, such as EDIFACT, X12, HL7, HIPAA, etc., were developed for specific industries and applications, and each had their own rules, formats, character sets, etc.
Fast forward to the 2000s, the EDI standards have proliferated into complex document exchanges typically between large trading hubs that are able to mandate compliance among their typically less-powerful trading partners (often suppliers). And over the past 10 years, APIs (over the internet) started making headway as an alternative to EDI. API, or Application Programming Interface, provides for greater agility and customization without as much lock-in to figuring out how to make the EDI standards accommodate more modern business data exchanges. Instead of relying on EDI document standards and rules, APIs used XML or JSON as response formats for communication. However, whatever technical debt you might avoid with less EDItech lock-in is doubled in terms of technical-debt as you try to keep up with API versioning that is often very specific to one trading relationship and more encompassing than just data formatting.
“Before data could be exchanged electronically, all businesses had were paper records and the keen eye of business owners to capture, process, and deliver customer orders. The shortcomings of that process were too many, and may seem too trivial today.”
Each of these methods for ‘better’ B2B data integration have their own set of drawbacks. For instance, the growing and myriad EDI standards and specifications made the technology hard to scale, turning into a complex motley of requirements that was too difficult to manage as businesses expanded their relationships. With each new trading partner, businesses had to employ expensive, high skilled IT to manage and grow the EDI support system. This was a barrier to scale and digital transformation.
With an XML based approach, the message sizes can grow unwieldy, become too complex and result with inefficiencies in processing a high volume of other small messages. When parsed in memory, they are resource-intensive and potentially disruptive to other B2B message traffic.
VANs, or Value-Added Networks, were another alternative to direct EDI connectivity. Essentially, VANs functioned as communication brokers in the middle, directing EDI messages between varied partner mailboxes. It was especially useful early on before many companies had constant internet connectivity and an ability to directly send and receive electronic documents. But this was long ago, when there were dialup connections and phone-line modems. It was also useful when there were additional requirements that direct, peer-to-peer EDI couldn’t tackle because of its nature, and that’s why a connectivity broker in the center was needed. However, VANs generally were too expensive, especially for small businesses who did not have too many transactions flowing through. In most cases VANs also left a lot to be desired when it came to tracking and transaction status. And in cases where there were multiple VANs between you and any of your trading partners, you could just forget about document transfer status or visibility.
Once Walmart and other major trading hubs supported and championed (read: “mandated”) the AS2 protocol for data transfer, it provided a means by which one might circumvent the traditional, expensive VANs. However, because of its reliance on internet protocols, even this method wasn’t the silver bullet small businesses needed. If anything, AS2 complicated things by imposing additional requirements, specialized software, and digital certificate management upon trading partners. IT to the rescue. But the cure wasn’t easy or inexpensive.
In short, all these technologies solve pieces of the puzzle but brought forth their own set of new problems and limitations. One set of limitations was replaced by another as mechanisms, standards, and implementation methods changed. And with each, a greater degree of IT involvement was needed to implement new trading partner connections. The resulting set of technologies involved in trading partner connections, data transfer, data reformatting, exception handling, data security, and so on evolved what was once a business person writing out a purchase order and mailing it to a supplier into a multi-layer technology masterpiece capable of processing thousands of orders per minute. But setting up, configuring, and servicing this beast for new customers is IT-intensive and extremely deficient from a CX point-of-view.
The key in understanding what would alleviate these headaches is considering all four dimensions of customer data integration. Instead of chasing some minor technical improvement or incremental gain in IT productivity by partially automating one of a hundred steps, perhaps it’s time to take a step back from the entirety of the process and technology and reassess the bigger picture.