XML transformation overview

In this diagram a description of the datapipe from an (webservice) or XML file source to the target datamodel is described. This is based on the transformation of a XML model to an intermediate tabular or relational model and this is then processed in an ETL process to transform the source data model in a number of steps to the required target model.

Version 1.0
Created date 02-05-2021

Data target

Data target based on a standardized data model. Based on this model and the included data entities one or more interfaces can be implemented for the consumers.

Author Bert Dingemans
Alias --
Stereotypes DataObject
Details of Data target

Extract

Extract functionality for receiving data from the source systems. This has a XML format and the source is either a webservice or a XML file. The data is extracted and prepared to be transformed

Author Bert Dingemans
Alias --
Stereotypes ApplicationFunction
Details of Extract

Load

Load functionality for transferring the data to a data target and load it there. This can either be an ETL process or the implementation in a message handler

Author Bert Dingemans
Alias --
Stereotypes ApplicationFunction
Details of Load

Logical modeling

The logfical or conceptual model is abstract in nature and is therefore independent from the implementation platform. In most cases this model must be readable for non IT stakeholderrs.

Author Bert Dingemans
Alias --
Stereotypes ApplicationFunction
Details of Logical modeling

Modeling

For the transformation of source to target datamodels a description of the data elements has to be modeled.

Author Bert Dingemans
Alias --
Stereotypes ApplicationFunction
Details of Modeling

Physical modeling

Physical model for the description of a technical implementatin model. This is platform specfic and is used for the creation of SQL DDL and ETL scripts and eventually for technical documentation.

Author Bert Dingemans
Alias --
Stereotypes ApplicationFunction
Details of Physical modeling

Transform

Transformation functionality for transforming the data content from a (application) specific datamodel to a generic datamodel based on a standardized datamodel (see the service diagrams). This can be impllemented either in an ETL transform process or in a

Author Bert Dingemans
Alias --
Stereotypes ApplicationFunction
Details of Transform

Workflow management

Workflow management for the scheduling and iteration of transformation tasks

Author Bert Dingemans
Alias --
Stereotypes ApplicationFunction
Details of Workflow management

XML based webservice

--

Author Bert Dingemans
Alias --
Stereotypes DataObject
Details of XML based webservice

XML File

File with a validated XML content that can be processed to a structured model and pessed to an ETL process

Author Bert Dingemans
Alias --
Stereotypes DataObject
Details of XML File

Relational databases

Data entities saved in a relational (staging) database

Author Bert Dingemans
Alias --
Stereotypes DataObject
Details of Relational databases

Relational transformation

Transformation of relational data entities (stored in a relational database) to the standardized data target.

Author Bert Dingemans
Alias --
Stereotypes ApplicationFunction
Details of Relational transformation

Transformation

Transformation function from the specific source data model to the standardized target data model. This can be a model or a protocol transformation.
Often this transformation is divided in sub steps and these sub steps will be analyzed and modeled in the

Author Bert Dingemans
Alias --
Stereotypes ApplicationFunction
Details of Transformation

XML Transformation

Transformation of an XML stream or file to a relational model

Author Bert Dingemans
Alias --
Stereotypes ApplicationFunction
Details of XML Transformation

Data source

Description of the data sources in general. Is an aggregation of specific data source types. This generic datasource is added to the model to make associations possible with generic requirements and principles

Author Bert Dingemans
Alias --
Stereotypes DataObject
Details of Data source