Migration, collocation, optimization

More than 3,000 Żabka stores and over 500 Freshmarkets across the country, the headquarters in Poznań supervising the business, and all entangled in a complex network of systems and applications. One of the largest SAP ERP installations in Poland, the SAP HR system for managing geographically dispersed employees, central management of all stores, hundreds of virtual machines involved in processes running on a 24/7/365 basis, and additionally, management of loyalty programs and websites … – IT systems and terabytes of data make up one of the largest assets of Żabka.

Until recently, this valuable resource was fully collected and stored just under the watchful eye of the board and management of the company – in its headquarters in Poznań.

Towards collocation

Many companies, especially those that built/developed their IT infrastructure in the 90s of the twentieth century when this model was the most common solution – have their own server rooms, mainly due to the lack of a better alternative. The offer of data center services was not mature yet and boards of companies were rarely ready to entrust such valuable assets to third parties. Therefore, companies built their own server rooms, which, however, in many cases turned out to be too small with time, or did not meet increasingly higher security requirements.

For more than 16 years of its existence, Żabka was consistently developing its system landscape, meeting the expectations of business and enabling uninterrupted expansion in the market. By investing in IT systems, it in parallel invested in “hardware", maintaining a modern infrastructure of its server room. However, for reasons beyond the company’s control, not all expected standards were guaranteed, including the standards related to the power supply. Additionally – although a backup data center was created – it was located in the same building as the main one, which was not fully consistent with the expected security parameters.

There were also other reasons for making the decision about the IT collocation in Żabka. The company was relocated. At the turn of the year, the headquarters were moved to the Andersia Business Centre. It is just a few hundred meters from the previous headquarters, however this decision had far-reaching effects for the management of the corporate IT infrastructure.

In view of the present security standards and technical requirements, building a data center is a very big investment, which should be additionally multiplied by two. To meet the security requirements of Żabka, only an IT infrastructure doubled in independent data centers is an acceptable solution.

At the beginning of 2014, Żabka inquired BCC (now All for One Poland) about the service of collocation of its IT environment at All for One Data Centers.

IT Outsourcing at All for One Data Centers

BCC provides IT outsourcing, secure data processing, cloud computing and IT consulting services based on the organization and infrastructure of All for One Data Centers – a specialized competence center. All for One Data Centers are modern data centers in two different locations, which is a unique value in the market of IT outsourcing services in Poland. As part of the collocation, BCC ensures the physical security of devices, along with other services expected by the customer. The collocation service is complemented by services of IT/SAP administration, Backup & Recovery as a Service and a backup data center.

In addition to the modern technical infrastructure, All for One Data Centers are distinguished by work standards and procedures consistent with global trends:

• a separate service organization (the All for One Outsourcing Center that is a dedicated service organization, ensuring a high quality and continuity of handling of all service and outsourcing contracts of BCC customers. The Center operates according to good practices of the service organization operation, e.g. the communication with customers is based on a single point of contact, SPoC);
• a team of several dozen IT technology experts – consultants, administrators and service technicians,
• the highest work standards (including the Information Security Management System compliant with ISO 27001 and the IT Service Management System compliant with ISO 20000),
• a monitoring center where systems and networks are supervised by specialists 24 hours a day, 7 days a week,
• parameters related to performance and availability higher than 99.98%.

Collocation: basic assumptions

In the first phase – at the stage of preparing the migration of Żabka infrastructure to All for One Data Centers – the basic assumptions of the service organization and the goals to be reached by the company through the IT collocation at All for One Data Centers were defined:

• availability of services in at least two data centers physically and geographically separated;
• ensuring high availability;
• independent Internet connections;
• access to a high level of competence of IT administrators and consultants, as well as experience in the SAP technology;
• the possibility of remote management of power supply in IT hardware cabinets together with constant current and power measurement at each slot;
• independent information on each opening of cabinets, recorded in an internal log of equipment, along with customer notifications;
• physical accessibility to the equipment on a 365/24/7 basis.

Stefan Lijewski, Administrator Team Manager, Żabka Polska

Collocation – the best choice for Żabka
We have chosen BCC (now All for One Poland) to be a provider of services of collocation and migration of the hardware infrastructure mainly due to the high competence of its employees. Previous cooperation in the field of SAP Basis administration showed us that we could rely on their knowledge and experience.
Also the geographical location of both centers suits us fine: they are located a few hundred meters from each other, which provides sufficient security in case of a construction disaster, for example, while allowing for a local replication of data between server rooms. In this way, we have achieved a much higher level of availability of services, without the need to maintain expensive connections and to purchase equipment and licenses to support replication.
An important argument was also the price and scope of services. BCC’s offer was one of the best among all the submitted ones.
Earlier, we had not even considered the need for collocating our systems. However, the necessity to leave the building required us to choose between building our own data center, placing the systems in a computing cloud or renting a server space.
Having the experience in building small server rooms in logistic centers we knew that a well-designed and built data center was a very big investment fraught with a significant risk. Even in the long term the return on such investment is uncertain if one takes into account the need to maintain the facility in a proper technical condition. Cloud computing was not an option either – we did not want to take the risk of transferring a complex network of dependencies to an external infrastructure.
Hence the idea of collocation, which I think is perfect for companies like Żabka – with their own, large IT environment, which are not interested, however, in maintaining useless infrastructure and developing competence in the IT department, which will be never used probably.
The migration itself was carried out very efficiently and responsibly – starting from the creation of the migration assumptions and ending with an established plan and its final implementation.
The diligence of preparation and accuracy of performance perhaps are best illustrated by the fact that the entire process needed only two service windows, where some production systems were switched off just 4 hours in total, and most systems were moved online. No user noticed the transfer of all the systems for collocation – a month after the completion of the process we were still asked when we would finally take the servers away from the current center…
The additional, and actually primary benefits from the cooperation with BCC are their competencies in the SAP Basis administration. We began the cooperation in this field in September this year, which was just before the start of the migration. In this way, we easily correlated the transfer of the systems and their daily operation.
The administration of SAP systems carried out by BCC meets our expectations. The main objectives during the change of the outsourcing company in this area were to increase the availability of systems and to optimize the operation of the environment. A proactive approach of BCC administrators and their professionalism, which we remembered from an earlier period of cooperation, were additional advantages.
Now, after several months of cooperation with BCC in both collocation and SAP Basis, I can confidently say that it was a very good choice. We have forgotten already about everyday problems with a damaged air conditioning, unreliable fiber optic routes or the need to secure a physical access to the server room. There is no problem with estimating the cost of infrastructure either – we know an exact amount of fees in every month, and the access to power strips enables us to estimate the required expenditure for electricity.
Stefan Lijewski, Administrator Team Manager, Żabka Polska

Migration – without system downtime

The complex project of moving the entire hardware platform, with virtually no system downtime, required the preparation of a multi-level and phased plan. In the first step, the scope of space needed for Żabka in the All for One Data Centers was defined. In this case, the space meant a specific power consumption required for powering and cooling the infrastructure. Specialists from BCC and Żabka jointly determined the estimated capacity for the devices in use. Then Żabka administrators estimated the needs related to filling each cabinet, which resulted in the selection of appropriate rack cabinet models along with the equipment meeting these assumptions.

In the second place, the connection methods to ensure target optical connections (FO), WAN networks (Internet access), IPVPN/MPLS (leased lines) and Fiber Channel (data transmission between a server and a disk array) were defined.

Starting a project of migration to the All for One Data Centers, Żabka had a structure of skeletal connections between the headquarters and its logistics centers. BCC’s task was to provide an optical connection for the temporary provision of data transmission services between the server room and the All for One Data Centers during migration. In order to optimize the migration costs, the CWDM solution was used. It allowed for one fiber optic pair to be multiplied to get several parallel connections at different wavelengths. In this way, the foundations for further migration work were created.

The work started at the beginning of June. BCC completed the first stage in mid-July on schedule. The first cabinets were collocated at the All for One Data Centers SITE 1. Żabka began the process of merging the WAN infrastructure, enabling further migration. At the beginning of August, the environment was ready and performance tests of connections between Żabka and All for One Data Centers were started to ensure optimal conditions for the migration process.

At the same time, BCC (now All for One Poland) started to take over the administration of the company’s SAP systems.

After a month of testing and confirming that the conditions for transmission were met, preparations for the physical migration of hardware from Żabka server rooms were started. For mid-September, a service weekend was scheduled, during which BCC together with a logistics company transported the physically secured and insured hardware to the All for One Data Centers.

The whole process of transporting servers, disk arrays and SAN and WAN network devices to BCC, divided into stages, ended after eight hours from the time of leaving the server rooms and heading to the All for One Data Centers. The transportation, fully supervised by BCC, ran smoothly, and the hardware arrived in perfect condition. Of course, the transported IT hardware did not contain any production data.

During the hardware migration process, production systems of Żabka worked continuously on the rest of the infrastructure.

Upon completion of the first stage, Żabka administrators, supported by BCC administrators, started the migration of virtual environments between the server room of Żabka and collocated equipment in the All for One Data Centers SITE 1, using temporary fiber optic connections and VMware mechanisms (vMotion and Storage vMotion) as well as AIX (LVM Mirroring). The outcome was the transfer of production systems with full availability (online) to the All for One Data Centers at the beginning of October.

With reliable systems operating in the All for One Data Centers, Żabka started to prepare for the second stage of the migration. In mid-October, the migration of the hardware from Żabka server room to the All for One Data Centers SITE 2 was carried out jointly. Also this time the transportation took eight hours. Thus, the whole IT hardware of Żabka was located in geographically dispersed data centers of BCC.

Fiber optic connections between the IT hardware of Żabka at SITE 1 and SITE 2 were defined with two different routes to ensure full redundancy. The environment was made available outside by various operators providing connections to the Internet and MPLS network. During all the work, Żabka systems worked uninterruptedly, with full availability for internal and external resources.

Outsourcing of SAP administration

In parallel with the process of migration, the project of taking over the administration of SAP systems in Żabka by the All for One Outsourcing Center was carried out. In September, BCC took over full responsibility for the administration of all SAP systems (i.e. SAP Basis administration). In total, there are more than a dozen systems: SAP ERP, SAP BW, SAP HR, Content Server, SAP Business Connector (test, development and production environments) with a total database size of more than 50 TB.

By connecting to central monitoring systems of BCC, administrators of the All for One Outsourcing Center are proactively informed online about any incidents that might affect the security and stability of SAP systems. SAP administration services are provided by dedicated teams of consultants from the All for One Outsourcing Center. To monitor the SAP environment, the availability of databases and CPU usage, the SAP Solution Manager and Nagios solutions are used.

Ongoing operation

The physical security at the All for One Data Center is ensured by full redundancy of power supply, cooling, and connections. Data centers are also protected by certified fire detection and extinguishing systems, and safeguarded against moisture. Access zones and monitoring are another level of physical security.

The equipment is monitored 24 hours a day. IT administrators of Żabka can remotely restart collocated devices, even if a given device stops working, which significantly speeds up the repair when an emergency restart is necessary. The devices register energy consumption on an ongoing basis, so Żabka payments for BCC are based on meter readings and made per 1 kWh.

Each opening of a rack cabinet (in addition to registration in a device log) results in an automatic notification of Żabka, as expected, including detailed information about the time of opening and closing of a specific cabinet.

All these security measures do not eliminate a “human factor". Everyday, BCC (now All for One Poland) administrators make a detailed review of the infrastructure – which builds on the principle of duplication of security measures. Any irregularities or deviations from the standard are recorded and the customer is informed about them immediately.

Under the agreement, the designated administrators of Żabka are provided with a permanent access (at any time of the day or night) to both locations of All for One Data Centers. This increases the sense of security.

However, a BCC administrator does not need to go to Żabka whenever it is necessary to replace a disk in the disk array or to manually restart the server. As part the remote hands service, a BCC consultant can do minor maintenance work – as required by Żabka – and submit a detailed report on this work.

Data security, confidentiality and availability are the main reasons for which Żabka has decided to migrate its IT environment to the All for One Data Centers. The guarantee of maintaining security parameters and availability higher than 99.98% without a big investment – for monthly fees, is a compelling argument for both the IT director and the finance director.