As AI systems become more integrated into daily life, demand for AI-related skills will rise. Historically, data scientists were crucial for developing and managing AI systems. However, with AI systems now more user-friendly and accessible, the necessity of data scientists is being questioned. Data remains vital for AI, and data scientists traditionally turn data into valuable insights. Despite their expertise, data scientists are expensive and scarce. The need for AI capabilities is outpacing the availability of skilled data scientists.
The distinction between building and using AI models is essential. Generative AI and Large Language Models (LLMs) provide easy access to AI capabilities, making them usable without data scientists. Instead, organizations will need prompt engineering skills, which rely more on soft skills like critical thinking and communication. For more specific needs, fine-tuning and Retrieval-Augmented Generation (RAG) are required, which involve basic programming and data skills rather than intensive data science.
Data engineering, focusing on data availability and quality, is becoming increasingly crucial. Data engineers manage data pipelines, ensuring data consistency and cleanliness, making them potentially more vital than data scientists for AI projects. Thus, the role of data engineers may become paramount in the next decade.
