ETL is critical for automating process integration. Data is often received in different formats and needs to be converted into other formats for target applications or for users to review. Complex mapping rules need to be defined as these formats are converted. This usually requires tedious and extensive coding while the tools available to simplify this process are expensive and only offer limited capabilities. Adeptia provides functionality to do all this quite easily and without any need for coding. This data transformation capability is a strength of Adeptia and differentiates it from other BPM products in the market.
The following two components provide the ETL functionality of the Adeptia Integration Suite: Schema and Data Mapper.
Data Mapping Demo (6 minutes streaming video) – See how developers graphically create purchase order data maps between XML and database.
Schema Parser
Schema is a structure of a file format and it specifies information on the different data fields and records that are in a message or data file. Designing a schema is the process of providing metadata information. Schema Designer is a wizard driven Web-interface that allows IT users to design and manage data formats. The Schema Designer allows IT users to specify metadata for all commonly used file formats such as Flat Files, Fixed-length positional files, EDI, XML, Microsoft Excel, as well as relational databases like Oracle, SQL Suite, IBM DB2 and MySQL etc. Complex metadata information can be provided including data field element names, attributes, field types as well as hierarchical record structure.
Schema Parser allows data messages and files of different formats to be automatically processed based on the schema definitions. This is a run-time service that can be orchestrated by a user as a task in a process flow. It uses schema (metadata) rules as defined in the Schema Designer to identify the format and structure of a message file and so parse that file into records and data elements. This parsed data can then be sent to the Mapping engine to be transformed into another format Data Mapper.
Data Mapper
The Data Mapper is a Web-based interface for managing data transformation rules. It is a graphical tool that enables visual and rapid specification of complex business rules in easy to understand English rather than code. It auto-generates data transformation code in a fraction of time compared to custom manual coding. The Data Mapper eliminates the need for the intricate, time consuming and error-prone process of manually writing thousands of lines of data mapping code. Collaboration and mapping rules management is simplified by auto-generation of transformation rules in PDF format for easy documentation, business user sign-offs and archival.
Data Mapper simplifies management of mapping rules because these rules are:
- Defined using a visual, drag-and-drop interface.
- Specified in easy to understand English.
- Documented in PDF directly from the code.
- Archived for later edits.
- Directly converted into program executable code.
Mapping rules are easy to specify. For any selected target data field, a text area is provided where a user can build complex transformation rules for that field. For ease of use, the Data Mapper consists of built-in libraries of Math, String, Aggregate, Conditional and Boolean functions for data manipulation. Text area is also provided for users to specify comments where they can note the versioning information, history and any other context for the mapping rules for that field. This simplifies edits and future manageability of the mapping rules.
The Mapping Engine is the heart of the data transformation module of the Integration Suite. It takes the mapping rules as defined in the Data Mapper and applies those rules to the parsed data to create the transformed output. Adeptia has designed the Mapping Engine to be functionally powerful so it can apply complex rules to complex data structures.
Managing Bulk Data
Adeptia Integration Suite supports processing of both single-record messages as well as bulk data in the form of multiple-record data files. The Parser and Mapping engine have highly scalable performance so the Integration Suite can scale up to handling hundreds of megabytes of data files that may contain hundreds of thousands of records. The Integration Suite does this in an efficient manner allowing it to offer speed and throughput that is only matched by dedicated hardware appliance-type of solutions.
Please also check the other Adeptia products:
- ETL Suite: An easy-to-use data mapping solution for aggregating data from multiple sources to populate databases and data warehouses.
- BPM Suite: Business process management and workflow automation product.