ETL is critical for automating process integration. The Data Mapping engine in ETL Suite allows any-to-any transformations between different data formats. It includes complex data functions such as string, math, and conditional operations as well as DB and XML file look-up. Mapping rules can be applied from multiple sources to multiple targets irrespective of their data format. So, data can be aggregated from multiple sources. Content can also be distributed to multiple destinations automatically. Simulation ability allows verification before deployment. ETL Suite provides functionality to do all this quite easily and without any need for coding. This ETL capability is strength of Adeptia and differentiates it from other middleware products in the market.
The following two components provide the ETL functionality of the Adeptia Integration Suite: Schema and Data Mapper.
Data Mapping Demo (6 minutes streaming video) – See how developers graphically create purchase order data maps between XML and database.
Schema Parser
Schema is a structure of a file format and it specifies information on the different data fields and records that are in a message or data file. Designing a schema is the process of providing metadata information. Schema Designer is a wizard driven Web-interface that allows IT users to design and manage data formats. The Schema Designer allows IT users to specify metadata for all commonly used file formats such as Flat Files, Fixed-length positional files, EDI, XML, Microsoft Excel, as well as support for relational databases like Oracle, SQL Suite, IBM DB2 and MySQL etc. Complex metadata information can be provided including data field element names, attributes, field types as well as hierarchical record structure.
Data Mapper
The Data Mapper is a Web-based interface for managing ETL rules. It is a graphical tool that enables visual and rapid specification of complex business rules in easy to understand English rather than code. It auto-generates ETL code in a fraction of time compared to custom manual coding. The Data Mapper eliminates the need for the intricate, time consuming and error-prone process of manually writing thousands of lines of data mapping code. Collaboration and mapping rules management is simplified by auto-generation of transformation rules in PDF format for easy documentation, business user sign-offs and archival.
Data Mapper simplifies management of mapping rules because these rules are:
- Defined using a visual, drag-and-drop interface.
- Specified in easy to understand English.
- Documented in PDF directly from the code.
- Archived for later edits.
- Directly converted into program executable code.
Mapping rules are easy to specify. For any selected target data field, a text area is provided where a user can build complex transformation rules for that field. For ease of use, the Data Mapper consists of built-in libraries of Math, String, Aggregate, Conditional and Boolean functions for data manipulation. Text area is also provided for users to specify comments where they can note the versioning information, history and any other context for the mapping rules for that field. This simplifies edits and future manageability of the mapping rules.
The Mapping Engine is the heart of the ETL module of the ETL Suite. It takes the mapping rules as defined in the Data Mapper and applies those rules to the parsed data to create the transformed output. Adeptia has designed the Mapping Engine to be functionally powerful so it can apply complex rules to complex data structures.
Managing Bulk Data
ETL Suite supports handling and processing of very large data files. The schema parser and mapping engine have scalable performance that can scale up to handle hundreds of megabytes of data files that may contain hundreds of thousands of records. This is done in an efficient manner at a speed and throughput that is only matched by dedicated hardware appliance-type of solutions.
Please also check the other Adeptia products:
- Integration Suite: Complete integration solution designed to integrate ANY APPLICATION with ANY DATA at ANY LOCATION.
- BPM Suite: Business process management and workflow automation product.