In the future, will data analysts find themselves unemployed due to job displacement by generative AI? Not enough. However, many roles in this field should expect major changes to their roles as companies radically rethink their approach to data analysis in response to growing demand. Generative AI revolution. How will this happen?
Context: What do data analytics teams do?
Let’s start by exploring the tasks typically performed by a data analytics team in an enterprise that has not yet adopted generative AI to improve analytics capabilities.
Traditionally, data analysts focused on integrating various data sources within a given organization and then creating queries to allow business stakeholders to answer questions based on their data. This process was often complex and lengthy, for several reasons.
One reason was the challenge of mapping disparate data sources and implementing the data transformations necessary to query them all in a consolidated way. This task required deep expertise in data integration and analysis, consuming a large portion of analysts’ time.
A second problem was that initial queries rarely fully answered a business stakeholder’s questions, due to the difficulty of determining exactly what information the stakeholder wanted. As a result, queries became iterative processes, with analysts having to refine queries and generate new reports repeatedly until they finally arrived at the desired answer.
In short, traditional data analyst roles revolve around complex data integration and querying tasks. The work tends to be tedious and time-consuming, and it becomes even more difficult as the scale and diversity of data assets within an enterprise increase.
How Generative AI Can Change Data Analytics
However, generative AI is poised to fundamentally change data analysis processes.
The main reason is that generative AI models can enable business stakeholders to ask and answer data-centric questions without relying on data analysts to write queries for them. As long as a generative AI model has access to relevant data sources, it can accept questions in natural language form and then generate appropriate data queries based on them. This is precisely one of the use cases that solutions like Amazon Q are designed for.
For the data analyst role, this approach to querying data has two profound implications. The first is that it reduces the importance of creating data inventories and maps. Instead of integrating disparate data sources in the traditional way – which involved manual effort on the part of data analysts – businesses can simply expose all of their relevant data to generative AI models and let them decide how to question.
Additionally, an AI-powered generative approach to data analysis allows queries to be iterated and refined much faster than teams could if they relied on analysts to write queries manually. Instead of a tedious process where a data analyst and a business stakeholder have a back-and-forth conversation and the data analyst writes multiple queries and reports to try to provide the answers the party is looking for stakeholder of the company, the stakeholder can interact directly. with a generative AI service to ask a question in different ways until the service produces the correct answer.
This is not to say that generative AI can interpret business needs better than human data analysts. Natural language queries are always ambiguous, both for generative AI models and humans. The advantage of generative AI in this context is its ability to iterate faster and generate new versions of an answer in seconds instead of hours.
The Future of Data Analyst Jobs
None of this is bad news for data analysts who might be worried about their jobs. Rather, while generative AI is likely to disrupt fundamental elements of the traditional data analytics function within many companies, it will make the work of data analysts more rewarding and important in other ways.
Instead of spending the majority of their time integrating and querying data, analysts at a company that adopts generative AI as a foundation for analysis will turn to work aimed at enabling Generative AI. For example, analysts will take the lead in training models. They will also play a key role in enforcing data governance and security policies, which determine what data data-generating AI models can or cannot access – or, in cases where very high levels of access granular data is needed, data analysts will help put in place controls that allow certain users to access certain data through generative AI services, which may not be available to other users of the same service.
This work will probably be more rewarding than the tedium of writing queries. This will also involve learning new data management skills. That’s why data analysts who want to get ahead of the generative AI revolution should now focus on honing skills in this area.
Conclusion: A Bright, But Different Future for Data Analysts
If you are a data analyst experiencing the generative AI revolution, now is the time to rethink your role and the value you bring to the business. Gone are the days when the ability to integrate data sources and write complex queries was the cornerstone of the data analytics function. Going forward, capabilities related to supporting and managing the data paradigms that power generative AI models will become central.
Ultimately, the work data analysts do in the brave, new, generative AI-centric world is likely to be more interesting and rewarding, and it will certainly be very different from traditional data analysis tasks.
Eamonn O’Neill is the co-founder and chief technology officer at Lemongrass.
