Data practitioners are among those whose roles are seeing the most significant changes, as organizations expand their responsibilities. Rather than working in a siled data team, data engineers are now developing platforms and tools whose design improves data visibility and transparency for employees across the organization, including analytics engineers, data scientists, data analysts, machine learning engineers and business stakeholders.
This report explores, through a series of interviews with expert data practitioners, key changes in data engineering, evolving skills required of data practitioners, data infrastructure and tools options to support AI, as well as the data challenges and opportunities emerging alongside generative AI. . The main conclusions of the report are:
- The fundamental importance of data creates new demands for data practitioners. As the rise of AI demonstrates more clearly than ever the importance of data to business, data practitioners face new data challenges, increasing data complexity, the evolving team and emerging tools and technologies, while establishing new organizational importance.
- Data practitioners are moving closer to the business, and the business is moving closer to the data. The pressure to create value from data has led executives to invest more in data functions. Data practitioners are encouraged to broaden their knowledge of the business, engage more deeply with business units, and support the use of data in the organization, while functional teams discover they need their own internal data expertise to leverage their data.
- Data and AI strategy has become a key element of business strategy. Business leaders must invest in their data and AI strategy, including making important decisions around data team organizational structure, data platform and architecture, and data governance. because the key differentiator of every business will increasingly be its data.
- Data practitioners will shape how generative AI is deployed in the enterprise. The key considerations for deploying generative AI – producing high-quality results, avoiding bias and hallucinations, establishing governance, designing data flows, ensuring regulatory compliance – lie with data practitioners, giving them a outsized influence on how this powerful technology will be implemented. work.
This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by the MIT Technology Review editorial team.
