THE NEED FOR GREATER DATA AVAILABILITY
In its transformation process, Grupo Éxito considered the need to refine data related to logistics visibility (all the movements that a distribution center has to move merchandise to warehouses), sales and inventories. This in order to offer high standards of efficiency in the allocation of resources for the more than 2,600 points of sale that the company manages in the region. Luz Marina Gallego, IT Analyst in the Business Analytical Services Department at Grupo Éxito, points out: “Before we started working with Xertica, online sales information was hosted on our Teradata Hadoop server, but it took a long time. Its update: sometimes it took more than 15 minutes. The initial challenge was to stop storing information in Hadoop and move the data to a technology that would allow us to see the information in near-time. “This is how we arrive at the GCP cloud.”
A problem that the previous technology presented was the lower availability of the data generated by the company: “Only one day's sales were shown, which forced our team to delete the day's data when a new day began. prior," says Gallego.
THE ROAD TO THE CLOUD
The project consisted of configuring near-time sales and inventory flows and logistical visibility of the various SKUs that the company provides through its points of sale and then building a data lake where they can be stored. The challenge involved being able to include the information from these flows in a data lake that enables the client to exploit said information for the benefit of their business objectives.
On the infrastructure front, an integration had to be made between Cloud Identity and the consoles used by Grupo Éxito (OpCon). Subsequently, the configuration of sales, inventory and logistics visibility flows was carried out through Matillion and then brought to the GCP data lake.
The architecture designed by Xertica is based on the following key points:
Use of Cloud Identity for user account management. Cloud Identity allows Grupo Éxito to access GCP to manage projects centrally through Cloud Resource Manager.
To securely connect Grupo Éxito's on-premise local network to GCP, Cloud VPN was used.
Matillion for the execution of ingestion and transformation processes, governed by OpCon. Matillion enabled orchestration of data extraction, transformation, and loading into Big Query, the highly available enterprise data warehouse used for super-fast queries backed by the processing power of GCP infrastructure.
Use of Microstrategy, Power BI and Google Data Studio for massive consumption of reports.
RESULTS
Currently, Grupo Éxito has at its disposal a total of 75 dashboards with analytics for decision making. More specifically, 50 daily reports, 19 trend reports and 6 comparative reports. In the past, information was only available for two distribution centers; Today, the reach is expanding to more distribution centers.
A problem that previous technology presented was the lower availability of the data generated by the company. In the past, Hadoop did not support saving data history. However, “since the project began production in February 2020, we can now consult the entire sales history day by day,” concludes Gallego.
The project led by Xertica sought to make data consumption independent according to the type of process. Everything that consumes data online is uploaded to the GCP cloud; while batch data is maintained on-premise in Teradata. Thus, it has been possible to align online data consumption with an infrastructure that could process said data in real time, with an update period of 3 minutes; that is, a reduction of up to 80% in data processing time.
