Data Warehouse & ETL
The mainstay of information architecture
Data warehouse systems play a key role in the information architecture of businesses. They consolidate and aggregate data from various operating systems with the aim of meeting the information needs of different user groups. The use of data warehouse systems presents a number of advantages: End-to-end analysis, as well as reports, statistics and KPIs, can be produced quickly and flexibly – providing important findings for strategic decisions.
The introduction of a data warehouse leads to the clean modelling of the data being processed. Redundancies and denormalised data are identified and correspondingly taken into account in the data model. Inconsistencies, or problems of data quality are highlighted and remedied. Only by doing this is it possible to guarantee a high level of reporting quality. Data warehouse systems act as databases with an appropriate reporting and interface layer. The additional use of data marts prevents large volumes of data resulting in performance bottlenecks.
We have been modelling and implementing data warehouse systems since 1994. Our services include analysis and concept development, development of ETL strategies/data propagation and the optimisation of existing data warehouse solutions. The technical and professional knowledge of our experts has been combined in our “Database” area of competence since 2011. This enables us to optimise in-house knowledge transfer and offer a central contact point for all employees in the event of questions and problems. Internal training courses ensure that our IT experts constantly extend their extensive expertise and familiarise themselves with the latest developments in data warehouse systems.
In data warehousing ETL systems form the data interface between operational data inventories and data warehouses or data marts. To achieve this, relevant data is extracted from a single, or multiple source systems and is transformed and uploaded into a target database. The transformation presents extremely exacting requirements: It is essential to transform data from sources structured in different ways into a single standardised data model. Other processes, including data cleaning, quality assurance, data historicisation, cube and data mart generation, as well as master data management, also run in modern ETL systems.
In our client projects we use individual developments based on PL/SQL as well as the following standard solutions:
- Informatica - Power Center
- SAP Business Objects – Business Objects Data Integrator
- Oracle – Warehouse Builder, Data Integrator
- Clover ETL
- Enterprise Application Integration (EAI)
Data modelling defines the data relevant for the creation of an information system including its structures and relationships. Collating all the possible relationships is an extremely important step. This can sometimes take quite a lot of time. For this reason, it is imperative that care is taken with data warehouses to create efficient structures as analysis is generally performed and aggregated with large volumes of data.
Incorrect or imprecise modelling can lead to significant loss of speed and thus unacceptable response times. In this case, only a redesign associated with a high volume of work can remedy the situation. This is why it is indispensable to use a technically correct data model as the starting point. Together with our clients, we structure and document technical requirements, including hierarchies, hierarchy levels and dimensions, as well as KPIs and aggregation rules. The technical data model results from the technical requirements and the data models of existing applications.
Our experts are conversant with “manual modelling” as well as with tools from the extensive Oracle or SAP suites and also MagicDraw. We also have extensive expertise in working with company-wide data models and with repository solutions for data modelling in large corporations.
We use the following standard software in our client projects:
- SAP BW