A large-sized IT company the development of digital solutions contained vast amounts of data for the company structure, projects, customers, employees, and their education certificates of 3,000+ employees.
The existing system provided them with more functionalities than needed, leading to overpricing. The decision to change was financially based, and pricing was the reason.
The migration decreased the cost of ownership of the ETL system by at least five times compared to the original DataStage usage.
The IBM DataStage logic has been optimized and successfully migrated to Visual Flow, resulting in 3 times improved performance.
The Data Analysis team is now able to build new jobs and pipelines 8 to 10 times quicker, using the simple drag-and-drop interface/UI for the panel and technical documentation for the Visual Flow solution.
Sooner or later, many companies meet data transformation challenges during their business operation course. It was the case with one of our clients who approached the Visual Flow professional service team with the request to migrate ETL jobs from a current application to something modern and open source.
Let us start with some general information about our client. It’s a large-sized tech-driven organization focusing on the development of custom digital solutions. They invest heavily in the education of software engineers to improve their hard and soft skills.
The customer system was developed around the IBM Datastage software. It contained ETL data for the company structure, projects, customers, employees, and their education certificates. Just for reference: there was data of 3,000+ employees. It includes information for completed and passed courses, internal promotion and compensation increase lists, and data that allows for monitoring the availability of engineers for potential projects.
Our client faced a complex issue. First, the IBM Datastage provided them with more functionalities than they needed, meaning they were paying for services they did not use . Pricing was the reason for the changes.
The fast-approaching expiration date of the hosting license created a sense of urgency for the data migration.
So, they approached the Visual Flow team to help migrate away from DataStage and old servers. We were to find a solution that would meet our client’s requirements regarding the tech stack. It should be modern, cloud-native, scalable, and open-source. It also needed to be intuitive enough and come with low code and job visualization options. The latter functionalities were due because they greatly assist non-tech data analysts, and HR use them in their day-to-day ETL routines.
After a negotiating and planning phase; the Visual Flow team came up with clear goals to achieve. The team created an approved work scope.
The project was complicated by the requirement to conduct all the processes within one month without compromising the quality, reliability, or performance of ETL processes. As mentioned, deadlines are explained with the oncoming shutdown of the servers.
Business-level projects are not about meeting technical requirements, but rather about outcomes obtained. Here is a summary of the achieved results through this project implementation:
Do you have a data migration or a similar ETL project in mind?
Are you inspired by the results Visual Flow provides the clients? If so, contact us today! Our professional services team will always be happy to provide you with extra details or schedule a free strategy session.