Data migration projects as part of IT system implementation

Another example may be the one of a product size and its influence on automatic transport planning or the classification of a customer and the related discounts. What clients expect is such preparation of the system that it will properly plan transport, suggest discounts or block unreliable payers.

Let us suppose that the functionality of the system was designed properly and in accordance with the expectations. However, the fully correct rules of system functioning will work only as well, as the quality of data which the system operates on will enable this.

Running a project of data migration without proper care and attention leads directly to the system which will not meet the users’ expectations. During every stage of data preparationit is necessary to assure that the prepared set of data is sufficient for proper functioning of the desired business functionalities.

Data objects and system mapping

The IT environment which awaits the change related to the planned accomplishment of business operations in a new IT system is usually more complex than a single installation of a productive ERP system.

The first step in a data migration project is an answer to the question – how a landscape of systems after the change relates to the existing one. This will allow us to determine the sources of data and their destination.

W miarę trwania fazy realizacji nowego systemu, w momencie osiągnięcia gotowości systemu testowego do przyjęcia danych, rozpoczynają się kolejne cykle etapów sprawdzenia, ładowania i testowania, z których ostatni uruchamia się przed startem produktywnym w celu finalnego załadowania danych do startującego systemu.

One of important sources might unexpectedly be the collection of spreadsheets scattered in computers used by various company employees.

The data object is both the basic data and the transactional one, which can be successfully transferred by means of an uploading programme. The example could be the material index, business partners data (customers, suppliers), purchase orders, sales orders, balance, fixed assets, BOMs (material specifications), HR data etc.

Data objects mapping, i.e. the analysis of the relation between target and source objects, is the most important element of defining the scope of a data migration. It allows distinguishing precisely the objects which are going to be converted with regard to target system requirements and to show potential sources of data or mark the lack of them.

The more detailed element of the distinguished data objects analysis is defining the correlation between them. It is crucial for planning the sequence of loading the data.

Data enrichment

The decision whether the data needs to be enriched manually is possible already at the object mapping stage. It may be required if the data must be completed to fit the desired system functionalities. If we predict that these activities are going to be quite intensive, it may be necessary to separate the so-called “data buffer” in the form of an indirect system existing between the source and target ones.

The role of the buffer, understood as the database one, is not only to facilitate connecting data coming from various sources but also enabling temporary collection of additional information (indispensable in the target system).

Going the right direction

The amount of some data objects (e.g. clients or materials) suggests the implementation of tools supporting automatic extraction, cleaning and importing of the data. We can think of reports, ABAP programmes or LSMW tools. However, excessive dependence on the automation, particularly when the data needs to be completed during the migration process, does not lead to the success of the project.

Additionally, with a small amount of the data, the cost of building extraction tools and loading may exceed the cost of work required for manual data transfer. Depending on the volume and the nature of the data itself, we use different scenarios for conversion.

During the migration of data from the origin, we can distinguish the following tasks:

• Data extraction from the source systems
• Data cleansing and enrichment
• Preparing the mechanism for converting the data into the desired format
• Checking the data
• Loading the data into the source systems ( during the loading tasks we distinguish the stage of building programmes for data conversion and the actual stage of loading the data into the target system )
• Tests, simulations and target system performance validation

In practice, checking, loading and validation are done many times as the first loading of the data reveals the full scale of challenges included in the stage of data cleansing and enrichment, the stage which continues during the whole implementation project.

Migration schedule

The aim of a data migration project is not only to deliver it to the productive environment (PRD) but also to provide the data in the development system (DEV) and the quality assurance one (QAS). For this reason, enabling the efficient run of the consecutive stages of tests is an equally important task for a data conversion team.

Due to their specific character, the activities connected with data conversion are planned independently of the main division of the entire implementation process phases. Nevertheless, they must be closely synchronized.

The tasks of extraction stages, developing the conversion mechanism and preparing programmes for loading the data are done simultaneously with the concept designing phase of the newly implemented system. During this phase also begins the involvement in the stage of cleaning and completing the data.

As the accomplishment of the system proceeds and when the test system is ready for the data input, start the next cycles of checking, loading and testing stages. The last stage is run before the go-live in order to finally load the data to the productive system.

Manual work

Cleaning and enriching the data is the most critical element of the whole conversion process. It is also the most time consuming stage requiring a lot of engagement of the end users. The most characteristic activity of the cleaning stage is removing duplicates.

The most frequent situation happens when the same business partner is repeated in the data base. It hinders correct reporting – clients may easily avoid credit controlling mechanisms. Problems arise also when sales value for a segment or a group of customers are taken into account.

A simple activity of removing excessive data records from the set evokes a number of consequences for the remaining data which is being migrated. They include: the balance or sales transactions which may have references to the removed excessive records and also need to be improved.

The next issue which requires support from end users is data enrichment. Referring to the example of a credit control, the old system could only define the credit limit value and the newly implemented ERP system will also use the functionality of defining risk categories.

Naturally, assignment of a customer to a risk category does not exist in the data extracted from the legacy system. This data will be provided by employees using their own lists or analyzing client’s history with regard to keeping payment deadlines by the customer. Enriching the data by assigning thousands of clients to risk categories makes the end users’ engagement in the migration process invaluable.

On the basis of our experience we can declare that the process of data cleansing and enrichment, even if it is supported by an IT tool, is the task which in 80% is performed manually.

The migration team

Teams which perform data migration tasks are interdisciplinary ones. For this reason it is crucial to involve both functional and technical consultants in the whole process of data migration.

Such team should consist of:

• A team leader who is a contact point and a person responsible for work coordination and its completion in time in accordance with the plan of the whole project.
• Key users, directly responsible for cleaning the data and defining mapping rules.
• Business process owners – for final validation aim and verification of mapping rules.
• Integration consultants whose time is in 100% devoted to the development of extraction and loading programmes.
• Functional consultants (at least one from every team) as basic support in defining requirements for target data collections and a single point of contact for the functional team. It is especially important due to the necessity of continuous communication to the consultants all changes in requirements which arise in consequence of constant improvement of business processes pre-pared in the SAP system.

With regard to the importance of the issue and the influence on the success of the entire implementation process, the migration team – as the group of people focused on common aims – should always be treated as a separate entity, no matter that its members might be users and consultants working at the same time in other teams.

It simply works

The effects of the migration team’s good work within the project will be most often… transparent for the end users of the system. Working in a demanding environment of present day business world and processing consecutive business steps in the system it is difficult to notice the fact that the system simply works.

A complete customer and material database does not surprise, and properly reminded deadlines for payments, delivery destinations and price conditions do not evoke enthusiasm. A usual thing is that we can see open accounts from the invoices issued in the old system, that we can accept the return of materials sold in the previous IT system.

At the start of the project the Board assures an uninterrupted continuation of business operations by choosing the right organization of the dedicated data migration project which accompanies the main project of a new IT system implementation.