Data is pivotal for enterprises, with many emphasizing the CEO’s role in data initiatives. However, a significant number of data projects don’t reach production, with Gartner noting only 15% success in 2016. The transition of data projects into operational phases is crucial for digital transformation, building on the foundation set by DevOps. Gartner warns that by 2024, organizations without a robust data and analytics operational framework could face a two-year setback in their initiatives.

Basil Faruqui, of BMC, underscores the importance of orchestration in operationalizing data. He describes the complexities of building a data pipeline, from data ingestion to insight, emphasizing the need for automation in production. BMC’s Control-M, especially the BMC Helix Control-M launched in 2020, plays a pivotal role in this, catering to the evolving technological landscape and increasing demand for SaaS-based orchestration.

Using Hershey’s as an example, Faruqui illustrates how the company leverages Control-M for orchestration and automation in managing its intricate supply chain. Hershey’s data-driven approach involves ingesting data from various sources, processing it through AI and ML algorithms, and using analytics to make informed decisions.

However, Faruqui points out that while DataOps is integral to BMC’s strategy, it’s not the sole focus. Data pipelines interact with multiple applications, and tools like Control-M bridge these layers, differentiating them from other DataOps tools.

Faruqui will be discussing orchestration’s role in DataOps at the AI & Big Data Expo Europe. He anticipates increasing complexity in the AI and ML space, with vendors like AWS, Azure, and Google continually expanding their offerings. For orchestration in DataOps, the challenge lies in integrating these evolving tools, but it also presents an opportunity as clients seek seamless orchestration across new technologies.