The recent revolution in artificial intelligence has only accelerated the significant growth in data volumes we have witnessed in recent years. This explosion of data empowers us to be better informed and improves decision-making processes for businesses, governments, and private citizens alike. However, to successfully transform this raw data into relevant, actionable information, we need professionals who are highly skilled in managing, analyzing, and extracting valuable insights. This is precisely where the skills of a data scientist come into play. The global market for big data is projected to expand dramatically, with some estimates suggesting it will more than double its 2018 size by 2026. In short, big data is a massive business. Despite this surging demand, companies all over the world are reporting a significant shortage of qualified data professionals. This gap between demand and supply creates a tremendous opportunity for those who can cultivate the necessary competencies.
The ‘Unicorn’ Skill Set
One of the primary reasons for this talent shortage is the sheer difficulty companies face in finding data scientists who possess the right combination of skills. This is not surprising, as the role of a data scientist is inherently interdisciplinary. These professionals are specialists with a diverse array of skills that are not typically found in a single person, blending computer science, statistics, and business strategy. For this very reason, data scientists are often colloquially referred to as “unicorns,” a nod to the rarity of their comprehensive skill set. What are the most important skills for data scientists? This is a critical question that aspiring data scientists and current professionals who want to improve their career prospects must ask themselves. Data scientists are, by necessity, multifaceted and versatile professionals. The nature of their work demands a balanced mix of deep technical abilities and strong qualitative or leadership qualities. This article series will discuss the most in-demand skills in the data science industry and provide context for developing them.
The Role of Technical Skills
We will begin by describing some of the most important technical skills that data scientists need to be successful in the industry. These skills form the foundation upon which all analysis, modeling, and insight-generation are built. Without a strong technical grounding, it is impossible to manage and interpret the complex, large-scale datasets that define modern business challenges. These skills range from programming in specific languages to understanding the mathematical theories that power predictive algorithms.
A Popular General-Purpose Programming Language
A specific, general-purpose programming language is one of the most popular in the world, consistently ranking at the top of several key popularity indices. One of the main reasons for its worldwide popularity is its exceptional suitability for data analysis tasks. Although this language was not originally designed for data science, it has, over the years, become the undisputed king of the industry, supplanting many older, more specialized tools. This language is a central pillar in the technology stacks of countless companies, from small startups to the largest global enterprises. With powerful, pre-built libraries for data manipulation, numerical computation, and data visualization, data scientists can effortlessly work with all kinds of data. These tools streamline the entire workflow, from the initial stages of data manipulation and cleaning to the final steps of statistical analysis and creating insightful plots.
The Ecosystem of Advanced Tools
It is also worth mentioning this language’s dominance in the more advanced subfields of data science, including machine learning and deep learning. An ecosystem of popular packages and frameworks provides all the necessary tools to build and train sophisticated algorithms in these areas. This accessibility has lowered the barrier to entry for building complex predictive models, allowing more professionals to leverage these powerful techniques. Thanks to its intuitive syntax, which often mimics the English language, it is also a fantastic language for individuals who are new to programming. This gentle learning curve is a significant advantage, as it allows aspiring data scientists to become productive relatively quickly. They can start by learning the basics of the language and then progressively move on to its data-specific libraries, building a powerful toolkit for their future careers.
Developing Skills in This Language
You can begin your journey with this language by focusing on foundational online courses. An introductory course can teach the basic syntax and data structures. From there, a course specifically focused on data science applications will introduce the core libraries for data manipulation and visualization. This step-by-step approach allows you to build a solid foundation before tackling more complex topics like machine learning.
A Language for Statistical Specialists
If the first language is the king of data science, then a second, statistically-focused language is the queen. Developed in 1992, this is an open-source programming language specifically designed from the ground up for statistical and computational analysis. It has long been the standard for statisticians, academics, and researchers, and it maintains a strong, dedicated following in the data science community. This language is heavily used in scientific research and academia, as well as in industries such as finance and business where rigorous statistical analysis is paramount. It allows you to perform an extensive range of data analysis tasks. This is primarily due to its vast collection of data science packages, which are available in a comprehensive, centralized, open-source archive network.
The Statistical Language Ecosystem
Some of this language’s most popular libraries are part of a well-known, unified collection of data science tools. This collection includes powerful packages for data manipulation and transformation, as well as a world-class, declarative library for data visualization. This ecosystem is known for its elegant and consistent design, which can make the process of data analysis very efficient. The demand for programmers fluent in this language is growing rapidly. However, compared to the user base of the more general-purpose language, the number of data scientists with these specific skills is smaller. As a result, programmers proficient in this statistical language are often among the highest-paid professionals in the information technology and data science fields, as their specialized skills are highly valued.
Developing Skills in This Statistical Language
If you are new to data science, you will need to learn how to program sooner rather than later. We recommend starting with either the general-purpose language or this statistical one. An introductory course can teach you the basics of this language, and you can then take your skills a step further with an intermediate-level course. Following that, an introduction to its popular collection of tools will show you how to effectively prepare and visualize data.
The Mathematical and Statistical Foundation
You do not necessarily need a formal mathematical background to begin learning data science. However, you will not be able to get far ahead in your career if you do not take the time to familiarize yourself with some core mathematical and statistical concepts. This knowledge is not just theoretical; it is the practical foundation for everything a data scientist does. Statistical knowledge is crucial for selecting and correctly applying the various data analysis techniques that are available. It is what allows you to create robust, reliable data models and to correctly understand the nuances and limitations of the data you are dealing with. Without this foundation, it is easy to misinterpret data, choose the wrong algorithm, or generate misleading results, which can lead to flawed business decisions.
Key Areas of Mathematics for Data Science
In addition to the basic mathematics taught in school, you should invest time in learning the fundamentals of several key areas. Calculus, particularly differential calculus, is important for understanding how machine learning models are optimized. Probability theory is essential for understanding uncertainty and is the basis for many algorithms. Statistics, both descriptive and inferential, is the grammar of data science itself. Linear algebra is another critical component. In modern data science, data is almost always represented as vectors and matrices. Linear algebra provides the tools to manipulate and analyze this data. Furthermore, Bayesian theory is extremely beneficial if you are interested in more advanced artificial intelligence and machine learning techniques, as it provides a framework for reasoning under uncertainty.
Building Your Math and Statistics Knowledge
You can start by taking a code-free introductory statistics course to grasp the core concepts before moving on to more advanced topics. Many educational platforms offer a wide range of statistics and probability courses where you can choose your preferred technology to work with. These resources allow you to refresh your understanding of statistical techniques and learn how to apply them programmatically.
The Centrality of Data Management
While programming languages and mathematics form the theoretical and computational foundation of data science, the practical, day-to-day work often begins with data. Before any analysis can be performed or any model can be built, a data scientist must be able to acquire, store, and manage the data. This skill set is frequently overlooked by beginners but is a non-negotiable requirement in any professional setting. Data does not arrive in a clean, simple file; it lives in complex, operational database systems. The world’s data is stored in a variety of formats, but the most common by far, especially for business operations, is in a relational database. These systems are the backbones of modern enterprise, storing everything from sales transactions and customer records to financial ledgers and inventory logs. Therefore, the ability to communicate with these databases is not just a “nice to have” skill; it is an essential, fundamental competency for any data scientist.
SQL: The Language of Relational Databases
Although the Structured Query Language, commonly known as SQL, has been around since the 1970s, it remains an essential and vital skill for data scientists. SQL is the industry standard tool, the universal language, for managing and communicating with relational databases. These databases are designed to store structured data in a highly organized and efficient manner. Relational databases work by storing data in tables, which are analogous to spreadsheets with rows and columns. These tables are linked to one another by a few common columns or “keys.” This relational model is incredibly powerful. It reduces data redundancy and ensures data integrity, meaning the data remains accurate and consistent as it is modified. A large portion of the world’s data, and especially the most critical business data, is stored in these relational databases.
Why SQL is Essential for Data Scientists
Given the prevalence of relational databases, SQL is an absolutely essential skill for any data scientist. Before any data analysis can happen, the data scientist must first get the data. This “data extraction” step is almost always accomplished by writing a SQL query. The data scientist must be able to write queries to select specific columns, filter for relevant rows, and, most importantly, join multiple tables together to create the final, unified dataset they need for their analysis. For example, an analyst might need to investigate customer churn. The customer information might be in one table, their order history in a second, and their support tickets in a third. The data scientist must use SQL to write a query that joins these three tables, creating a single, comprehensive dataset where each row represents one customer and their complete history. Fortunately, compared to a full-fledged programming language like the ones discussed in Part 1, SQL is a declarative language that is relatively straightforward and easy to learn.
Developing Your SQL Skills
You can start your journey of learning relational database queries with an introductory course on SQL. These courses teach the basic syntax, including how to select, filter, sort, and aggregate data. From there, you can learn how to create your own databases and tables in a course on relational databases. More advanced topics include mastering complex joins, writing subqueries, and using window functions, which are all critical for sophisticated data extraction and analysis.
The Challenge of Unstructured Data
While SQL is the perfect tool for handling neatly structured data stored in tables, things can get much more complicated with unstructured data. A vast majority of the data being generated today is unstructured. This includes text data from emails, social media, and documents; audio and video data; satellite images; web server logs; and data from Internet of Things (IoT) sensors. This type of data is difficult, and often impossible, to store and process using the traditional relational model. The rigid, predefined schemas of relational databases are not a good fit for the variety and velocity of this new data. A relational database requires you to define your table structure before you can insert data, but unstructured data often lacks a consistent format. This has led to the development of a new class of databases designed specifically to handle these challenges.
The Rise of NoSQL Databases
To handle the different types of complex, unstructured data, other types of databases have been developed. So-called NoSQL databases, which typically stands for “Not Only SQL,” are capable of processing and storing large amounts of complex, unstructured, and semi-structured data. These databases are designed for high-speed, scalable performance, often sacrificing the strict consistency of relational models for greater flexibility and horizontal scalability. This means that instead of relying on one large, powerful server, NoSQL databases are often designed to run on clusters of many smaller, commodity servers. This makes them ideal for the large-scale data ingestion and processing required by modern web applications and big data systems. There are several different categories of NoSQL databases, each designed for a specific type of data and access pattern.
Key Types of NoSQL Databases
There are four main types of NoSQL databases that a data scientist might encounter. Document databases are very popular. They store data in a flexible, semi-structured document format, often similar to a JSON file. This is great for storing data where each item may have a different set of attributes, such as user profiles or product catalog items. Key-value stores are the simplest type, storing data as a collection of unique keys, each paired with a value. They are incredibly fast and are often used for caching data. Column-family stores are designed for very large datasets and high-write throughput, organizing data by columns instead of rows. Finally, graph databases are designed specifically to store and navigate data with complex relationships, such as social networks, fraud detection rings, or recommendation engines.
Developing Your NoSQL Knowledge
NoSQL databases are at the forefront of innovation in data science and big data. A data scientist does not need to be a database administrator, but they must understand these different database types and know how to query them. This allows them to access the full spectrum of an organization’s data, not just the structured data in relational systems. An introductory course on NoSQL concepts can help you get started with this in-demand technology, explaining the differences between the database types and when to use each one.
The Data Scientist’s Role in Data Management
In some larger organizations, there may be a dedicated “data engineering” team responsible for managing databases and building data pipelines. In these cases, the data scientist’s job is to collaborate with the engineers, specifying the data they need. However, in many smaller companies or on leaner teams, the data scientist is often expected to handle these tasks themselves. This means a truly versatile data scientist should be comfortable with both SQL and NoSQL. They should be able to connect to a relational database to pull transactional data, and just as easily connect to a document database to extract and parse text data from web applications. This full-stack data access capability makes a data scientist far more effective, independent, and valuable to their team.
The Critical Last Mile: Communicating Results
A key part of a data scientist’s work, and one that is often under-appreciated by newcomers, is communicating the results of their data analysis. An analysis that is technically brilliant but impossible to understand is useless. Only when decision-makers, stakeholders, and non-technical colleagues understand the conclusions of the data analysis can that data be translated into meaningful, real-world action. If people do not understand the results, your work as a scientist, no matter how sophisticated, provides no value to the company. This “last mile” of data science is where insights become impact. It requires a completely different set of skills from the technical, code-first abilities of programming and modeling. This is where soft skills become paramount. The ability to present and communicate the insights gained from data analysis is just as important, if not more important, than the analysis itself. Data scientists must be able to effectively communicate their findings and, just as importantly, tell compelling stories about the data.
The Power of Data Visualization
One of the most effective techniques for achieving this goal is data visualization. Data visualization is the subfield that involves representing data graphically, for example, in charts, tables, graphs, and maps. These representations are not just pretty pictures; they are a powerful tool for cognition. The human brain is wired to process visual information far more quickly and effectively than tables of numbers or lines of text. Data visualization enables data scientists to summarize thousands, or even millions, of rows and columns of complex data and present them in an understandable, accessible, and condensed format. A single, well-designed line chart can reveal a trend that would be impossible to spot in a spreadsheet. A scatter plot can show a relationship between two variables in an instant. This makes visualization the primary tool for both exploratory data analysis (helping the scientist understand the data) and explanatory analysis (helping others understand the findings).
The Rapidly Developing Field of Visualization
The subfield of data visualization is developing rapidly, with important contributions from disciplines such as cognitive psychology and neuroscience. This research helps data scientists understand the best and most effective ways to convey information through images. It provides a set of principles for choosing the right chart type, using color effectively, and reducing “clutter” to ensure the key message is clear and not misleading. A good data scientist does not just randomly pick a chart. They understand that a bar chart is best for comparing categories, a line chart is for showing trends over time, and a histogram is for understanding a distribution. They know how to use color to highlight key information, rather than just for decoration. This deliberate, psychology-based approach to visualization is what separates a professional from an amateur and is a critical skill for effective communication.
Tools for Data Visualization
There are many tools a data scientist can use to create meaningful visualizations. These tools fall into two main categories: programmatic libraries and business intelligence software. Programmatic libraries are code-based tools, often part of the programming ecosystems we discussed in Part 1. For the popular statistical language, a well-known declarative library is the gold standard for creating complex, publication-quality graphics. For the popular general-purpose language, a foundational library provides low-level control, while a higher-level library built on top of it is excellent for statistical plots. The second category is popular business intelligence (BI) software. These are often proprietary, graphical, drag-and-drop tools that allow analysts to create interactive dashboards without writing any code. These platforms are incredibly powerful for business reporting and allowing non-technical users to explore data themselves. A data scientist should be familiar with both approaches. They will use the programmatic libraries in their notebooks for exploration, and they will often use the BI tools to build the final, polished dashboards for stakeholders.
Developing Your Data Visualization Skills
You can start by taking a code-free introduction to data visualization to learn the core principles of chart types and effective communication. From there, you can explore the full range of courses on data visualization. Whether you want to learn programmatic libraries or master the popular business intelligence tools, you can find courses that cater to your preferred technology and help you build this critical skill.
Beyond the Technical: Business Acumen
Data is nothing more than recorded information. As humans, our bodies constantly gather information through our senses. But to use this information effectively, we must understand its meaning and its implications for our actions. The exact same principle applies to analyzing large datasets. To extract truly meaningful, actionable insights from data, we must first understand the context of the data we are dealing with. This is where business acumen, or domain knowledge, comes in. In addition to the deep technical skills we have already mentioned, data scientists must also have a solid understanding of the business sector or industry in which they work. This domain-specific knowledge is crucial for using data effectively and conducting better analyses. A data scientist who understands the “why” behind the data will always produce more valuable insights than one who only understands the “how” of the analysis.
Why Domain Knowledge is a Skill Multiplier
Domain knowledge acts as a “skill multiplier.” It enhances every other skill a data scientist possesses. For example, when a data scientist with healthcare domain knowledge is analyzing patient data, they will have an intuitive understanding of medical terminology, billing codes, and patient workflows. This will help them identify potential errors in the data, craft more relevant features for their models (e.t., create a “comorbidity index”), and ask more insightful questions that a non-expert would never think of. Similarly, a data scientist working in finance who understands market principles and regulatory requirements will be able to build more realistic and compliant models. This knowledge is crucial for guiding the analysis. Without it, a data scientist is just a technician, a “hammer in search of a nail.” With it, they become a strategic partner, a problem-solver who can identify the most valuable business questions and then use their technical skills to answer them.
The Art of Data Storytelling
The final and most advanced communication skill is data storytelling. This is the ability to weave data and narrative together to create a compelling, persuasive, and memorable story. It is the pinnacle of data-driven communication. A typical data presentation might be a dry recitation of facts: “Our sales in the western region are down by 15 percent, and our customer churn in that region is up by 8 percent.” This is accurate, but it is not engaging and does not inspire action. Data storytelling, in contrast, frames these facts within a larger narrative. A data scientist might start by introducing the “protagonist” (the western region), identifying the “conflict” (a new competitor entered the market), and then using the data as the “plot,” showing how the competitor’s actions are directly correlated with the drop in sales and rise in churn. This narrative structure makes the insights more engaging, easier to understand, and far more persuasive. It concludes with a clear “call to action” based on the data, giving stakeholders a clear path forward.
Developing Communication and Business Skills
Unlike technical skills, which can be learned from a textbook, these softer skills are often harder to develop. Business acumen is best learned through experience, by working closely with domain experts, asking endless questions, and reading industry news. Take your marketing manager to lunch and ask them to explain their job. Sit with a sales representative and listen to their calls. This on-the-ground learning is invaluable. Communication skills can be developed through practice. Join a public speaking club or volunteer to present at team meetings. When you write a report, give it to a non-technical friend or family member and ask them if they understand it. Practice framing your analyses as stories, with a clear beginning, middle, and end. Learning to be an effective communicator is a lifelong process, but it is the single skill that will most differentiate you in your career.
From Analysis to Prediction
The skills we have discussed so far—programming, mathematics, data management, and visualization—are the core components of a data analyst’s toolkit. They allow a professional to look at historical data and understand what happened in the past and why it happened. This is a powerful and valuable skill. However, the data scientist’s role often goes one step further. Businesses do not just want to know what happened; they want to know what will happen next. This is the leap from “descriptive” and “diagnostic” analytics to “predictive” and “prescriptive” analytics. This is the domain of machine learning and artificial intelligence. These are among the most exciting and in-demand topics in data science. Machine learning is a subfield of artificial intelligence that deals with the development of algorithms that can learn to perform tasks without being explicitly programmed to do so. These systems learn directly from data, identifying patterns and building models that can make predictions about new, unseen data.
The Ubiquity of Machine Learning
From personalized movie recommendations and social media filters to email spam detectors and credit card fraud alerts, machine learning is deeply embedded in your everyday life. The increasing use of these machine learning systems by businesses to optimize processes, personalize customer experiences, and create new products is leading to a surging demand for data scientists with machine learning expertise. This demand far outstrips the supply. Statistics from 2020 showed that 82 percent of companies reported a need for employees with machine learning skills, while only 12 percent of those companies reported having a sufficient supply of machine learning professionals. This massive skills gap makes machine learning one of the most valuable and marketable competencies a data scientist can acquire. It is a key differentiator that moves a professional from a “data analyst” role to a “data scientist” role.
Understanding Supervised Learning
The most common type of machine learning is “supervised learning.” In this paradigm, the algorithm learns from a dataset that is already “labeled” with the correct answers. The goal is to learn a mapping function that, given a new, unseen input, can predict the correct output label. For example, you might have a dataset of emails, where each one is labeled as either “spam” or “not spam.” The algorithm “learns” the patterns associated with spam, and can then predict whether a new email is spam. Supervised learning tasks are typically broken into two categories. “Classification” tasks involve predicting a discrete category, like “spam/not spam,” “fraud/not fraud,” or “cat/dog/mouse.” “Regression” tasks involve predicting a continuous numerical value, such as the price of a house, the temperature tomorrow, or the number of sales a store will have next month. A data scientist must be proficient in the core algorithms for both classification and regression.
Understanding Unsupervised Learning
The second major paradigm is “unsupervised learning.” In this approach, the algorithm is given a dataset that has no pre-existing labels or correct answers. The goal is to find hidden structures, patterns, or groupings within the data. This is often more exploratory than supervised learning but is incredibly powerful for discovering new insights. The most common unsupervised task is “clustering.” This is where the algorithm automatically groups similar data points together into “clusters,” without being told what the groups are. This is widely used in marketing for “customer segmentation,” where a company can discover different groups of customers with similar behaviors and then target them with different marketing strategies. Another common unsupervised task is “dimensionality reduction,” a technique used to simplify complex, high-dimensional datasets, making them easier to visualize and analyze.
The Machine Learning Workflow
A data scientist’s work in machine learning is not just about choosing an algorithm. It is about following a systematic, end-to-end workflow. This workflow begins with “feature engineering,” which is the art of selecting and transforming the raw data variables into a set of “features” that the algorithm can best learn from. This is often the most critical step and requires a blend of domain knowledge and technical skill. Once the features are prepared, the data scientist will split their data into “training” and “testing” sets. The model learns the patterns from the training set. Then, its performance is evaluated on the testing set, which it has never seen before. This evaluation is crucial to ensure the model “generalizes” well to new data and is not just “memorizing” the training data, a problem known as “overfitting.” The data scientist will then “tune” the model’s parameters to find the best-performing version.
Key Libraries for Machine Learning
This entire workflow is made possible by powerful, open-source libraries. In the most popular general-purpose programming language for data science, one library has become the gold standard for “classical” (non-deep-learning) machine learning. This library provides a simple, unified interface for a vast number of algorithms, including those for classification, regression, and clustering. It also provides a comprehensive set of “helper” tools for the entire machine learning workflow. It includes modules for splitting data, for preprocessing and scaling features, and for model evaluation, allowing you to calculate metrics like accuracy, precision, and recall. This consistent and powerful toolkit makes it incredibly efficient to prototype, build, and evaluate many different models to find the one that best solves your problem.
Developing Your Machine Learning and AI Skills
For those new to the field, you can gain a high-level overview of the basics with a course on understanding machine learning. This will introduce the core concepts, such as the difference between supervised and unsupervised learning, without requiring deep technical knowledge. From there, you can explore how this technology is used to improve business outcomes in a more applied, business-focused course. For those interested in the broader field of artificial intelligence, you can explore a full learning path on AI basics. These resources can help you build a conceptual foundation, and separate articles and guides can provide a roadmap on how to learn AI from scratch, starting with the mathematical and programming prerequisites and building up to more advanced topics.
Machine Learning for Business
It is important to note that using machine learning in a business context is not just about building the most accurate model. It is about building a model that provides real, tangible value and is integrated into a business process. A model that predicts customer churn with 99 percent accuracy is useless if the company has no process in place to contact those customers and try to retain them. A successful data scientist understands this. They work with stakeholders to define the problem, understand how the model’s predictions will be used, and set clear metrics for success. They are concerned not only with the model’s accuracy but also with its “interpretability”—can they explain why the model made a certain prediction? This is often a critical requirement for businesses that need to justify their decisions.
A Further Step for Machine Learning Practitioners
A significant next step for machine learning practitioners is to move into the subfield of “deep learning.” Deep learning is a more advanced subset of machine learning that focuses on a powerful class of algorithms known as “artificial neural networks.” These algorithms are computationally intensive but are inspired by the structure and function of the human brain, with many interconnected layers of “neurons” that learn to detect patterns. Most of the high-profile advances in artificial intelligence in recent years have been achieved through deep learning. These neural networks are the technology that powers some of the most groundbreaking and impressive applications, including autonomous cars, virtual assistants, real-time language translation, and advanced image recognition. They are particularly effective at handling unstructured data like images, audio, and text, where traditional machine learning algorithms often struggle.
The Value and Challenge of Deep Learning
Knowledge of neural network theory and practice is rapidly becoming a crucial factor in the hiring and promotion of data scientists. However, it is fair to say that deep learning is a complex discipline. It requires a more advanced level of mathematics and programming skill than classical machine learning. Understanding how to design network architectures, select loss functions, and manage the complex training process is a significant undertaking. Because of this high barrier to entry, data professionals who have expertise in deep learning are among the highest-paid and most sought-after in the data science industry. They are the ones who can build the state-of-the-art models that unlock new product capabilities and competitive advantages for their companies.
Developing Your Deep Learning Skills
The journey into deep learning begins by learning how to build and train neural networks using one of the popular deep learning frameworks. These are open-source libraries that handle the complex, low-level mathematics, allowing developers to define their models at a higher level of abstraction. You can start with courses that introduce deep learning using one of the most popular high-level frameworks, or learn how to use a different, but equally powerful, low-level framework.
Handling Text: Natural Language Processing Skills
Humans communicate primarily through speech and text. It is therefore not surprising that a large portion of the data we collect is in this unstructured, natural language format. Natural Language Processing, or NLP, is a specialized subfield of artificial intelligence that focuses on giving computers the ability to understand, process, and extract meaningful information from human language. NLP is gaining ground rapidly in the data industry. NLP techniques, which are heavily based on machine learning and, more recently, deep learning, power some of the most ubiquitous applications we use every day. These include search engines, customer service chatbots, spam filters, language translation apps, and recommendation systems that analyze product reviews. A data scientist with NLP skills can unlock insights from vast quantities of text data, such as customer feedback, social media posts, or legal documents.
Developing Your NLP Skills
You can discover how to gain insights from text data by following a dedicated learning path on natural language processing. These paths will teach you the entire workflow, from cleaning and “tokenizing” text to building sophisticated machine learning and deep learning models for tasks like sentiment analysis and text classification, often using the most popular general-purpose programming language.
The Challenge of Scale: Big Data Skills
When it comes to processing truly massive amounts of complex data at high speed, relying solely on the data manipulation libraries of a single programming language is no longer sufficient. These libraries, while powerful, are “in-memory” tools, meaning they require the entire dataset to fit into the RAM of a single computer. When a dataset is terabytes or petabytes in size, this is simply not possible. This is the domain of “big data.” The big data ecosystem encompasses a rapidly growing set of tools and technologies that enable faster, more scalable, and more reliable analytics on distributed data. These tasks range from “ETL” (Extract, Transform, Load) processes and database management to real-time data analysis and automated task scheduling. These tools work by distributing both the data and the computation across a “cluster” of many computers, allowing for horizontal scalability.
Developing Your Big Data Skills
Learning the basics of distributed data management and processing is a key skill for senior data scientists and data engineers. You can start with a learning path on big data that uses a popular open-source framework for distributed processing. This will teach you how to use a familiar, high-level API to manipulate data that is spread across a large cluster. You can also learn how to plan and schedule these complex data workflows using a data orchestration tool.
The Role of Cloud Computing
In parallel with the development of the big data ecosystem, cloud-based services are becoming the first choice for many companies that want to get the most out of their data infrastructure. Building and maintaining a large, on-premise cluster of computers for big data processing is incredibly complex, expensive, and time-consuming. Cloud computing platforms solve this problem by offering these resources on demand. The cloud computing landscape is dominated by a few large technology companies. These providers offer tailored solutions based on customer requirements and a vast, integrated ecosystem of data tools. These platforms allow us to perform our entire data science workflow, from data storage and processing to model training and deployment, without ever having to manage a physical server. A data scientist can spin up a powerful, GPU-equipped machine for a few hours to train a deep learning model, and then shut it down, paying only for what they used.
Developing Your Cloud Computing Skills
A modern data scientist must be “cloud-fluent.” You can deepen your understanding of the fundamentals with no-code courses that explain the basic concepts of cloud computing or provide an overview of a specific major provider’s services. From there, you can learn how to programmatically interact with these cloud services to optimize your workflows, for example, by learning to use a provider’s software development kit to store data or manage machine learning models.
The Importance of Soft Skills
Although technical skills are a critical and non-negotiable part of a data scientist’s toolkit, they are only half of the equation. There are also less tangible, “soft” skills that you need to truly succeed and make an impact in the industry. An analyst who can build a technically perfect model but cannot explain it to a business leader or understand its real-world implications will ultimately have a limited career. These qualitative skills are what separate a good data scientist from a great one.
The Foundational Soft Skill: Business Acumen
We have briefly touched on this, but it bears repeating. Data is nothing more than information. To extract meaningful, actionable insights from it, we must first understand the context. Data scientists must have a solid understanding of the business sector or industry in which they work, whether it is finance, healthcare, marketing, or any other field. This domain-specific knowledge is crucial for using data effectively and conducting better analyses. A data scientist with business acumen can ask better questions. They can identify new opportunities where data can provide value. They can more effectively “feature engineer” by knowing which variables are likely to be important. And they can better communicate their findings in a way that resonates with stakeholders, speaking their language and focusing on the business metrics that matter. This skill is often developed not through courses, but through experience, curiosity, and collaboration with domain experts.
The Critical Role of Communication Skills
Data science is not just about mathematics and programming; it is also about presenting and communicating the insights gained from data analysis. If people do not understand the results of an analysis, your work as a scientist, however complex, is not valuable to a company. To transform data into decisions, data scientists must be able to communicate their findings effectively. This goes beyond just making a pretty chart. It involves explaining complex technical concepts in simple, accessible terms. It means understanding your audience and tailoring your message to their needs. A presentation to a team of engineers will be very different from a presentation to the executive board. Furthermore, data scientists should know how to tell compelling stories about data. Innovative approaches and frameworks for communication, such as data storytelling, can make a significant difference in this regard, making your insights more memorable and persuasive.
A New Imperative: Data Ethics Skills
Technology itself is neutral. But its use is not. In recent years, some data-driven companies and applications have come under scrutiny for developing practices and applications that have the potential to negatively impact people and society. These issues have, in some cases, undermined the credibility and trust of citizens in these companies and in technology more generally. To ensure that data leads to a positive impact, data scientists must develop a strong ethical awareness. This is no longer a “nice to have” skill; it is a core responsibility. This includes familiarizing themselves with key concepts such as data privacy, algorithmic bias, and feedback loops. Data scientists have a professional obligation to work toward the development of algorithms that are fair, transparent, and accountable for their decisions.
Understanding Data Privacy and Algorithmic Bias
Data privacy is a key ethical concern. Data scientists often work with sensitive, personal information. They must understand and adhere to legal frameworks that govern how this data can be collected, stored, and used. They must also employ techniques that protect user anonymity wherever possible. Algorithmic bias is another critical area. A machine learning model trained on historical data can inadvertently learn and even amplify existing human biases. For example, a loan application model trained on biased historical data might learn to unfairly discriminate against certain groups. A responsible data scientist knows how to audit their models for bias and uses techniques to mitigate it, ensuring their models are fair and equitable. You should also familiarize yourself with the broader field of AI ethics, as this is becoming a major topic of legal and societal discussion.
A Growing Concern: Environmental Awareness
The world is in the midst of an unprecedented climate crisis. Climate change and the rapid loss of biodiversity threaten the conditions that make human life possible on our planet. Although it is often overlooked, the digital industry, including the field of data science, must consider its significant environmental impact. The work of a data scientist is not “free” from an energy perspective. Storing and processing massive amounts of data in large data centers and, in particular, training complex machine learning algorithms, requires a tremendous amountof energy. This energy consumption leads to additional carbon dioxide emissions in the atmosphere. For example, in 2019, it was estimated that training a single, large, deep-learning language model could emit more than 626,000 pounds of carbon dioxide equivalent. This is almost five times the lifetime emissions of an average American car, including the emissions from its manufacture.
The Growing Digital Infrastructure Crisis
The digital revolution has fundamentally transformed how humanity stores, processes, and accesses information. Behind every online search, streaming video, social media post, and cloud-stored document lies a vast network of data centers that consume enormous quantities of energy and resources. These facilities, often described as the factories of the digital age, house thousands of servers working continuously to power our increasingly connected world. As society becomes more dependent on digital services, the environmental footprint of these data centers has grown into a significant contributor to global carbon emissions and resource depletion. Understanding the scale and nature of this environmental challenge represents the first step toward developing more sustainable digital infrastructure.
Data centers now account for a substantial and growing percentage of global electricity consumption. Recent estimates suggest that data centers consume between one and two percent of worldwide electricity use, a figure that continues to climb as digital services expand and data generation accelerates exponentially. To put this in perspective, the global data center industry uses more electricity annually than entire countries like the United Kingdom or Germany. This massive energy consumption translates directly into carbon emissions when that electricity comes from fossil fuel sources, making data centers significant contributors to climate change. The environmental impact extends beyond just energy use to include water consumption for cooling, land use for facility construction, and the complex supply chains required to manufacture and maintain the hardware that powers these digital engines.
The rapid growth of artificial intelligence, machine learning, streaming services, and cloud computing has accelerated data center expansion at an unprecedented rate. Training large language models or sophisticated AI systems requires computational resources that dwarf previous computing demands. A single training run for an advanced AI model can consume as much electricity as several hundred homes use in a year. Streaming high-definition video content to billions of users worldwide demands continuous server capacity and bandwidth. Cloud storage services that promise unlimited space encourage users to save vast quantities of data that must be stored indefinitely across multiple redundant systems. Each of these services, while providing genuine value and convenience, contributes to the growing environmental burden of digital infrastructure that society has only recently begun to acknowledge and address.
Energy Consumption Patterns in Modern Data Centers
The energy demands of data centers arise from multiple sources, with computing workloads and cooling systems representing the two largest consumers. Servers performing computational tasks require substantial electricity to power processors, memory, storage devices, and networking equipment. This computational energy use has grown dramatically as processing demands have intensified with the rise of data analytics, artificial intelligence, and real-time applications that require continuous server availability. The always-on nature of cloud services means that data center servers typically operate at or near full capacity around the clock, consuming electricity continuously rather than only during peak usage periods. This baseline consumption remains relatively constant regardless of actual demand, as servers must be kept operational and ready to respond to requests instantaneously.
Cooling systems represent the other major energy consumer in data centers, often accounting for nearly as much electricity use as the computing equipment itself. Servers generate tremendous heat during operation, and this heat must be continuously removed to prevent equipment failure and maintain optimal performance. Traditional air conditioning systems blow cool air through data center facilities to absorb heat from servers, then expel that heat outside the building. This cooling process requires enormous amounts of electricity to power fans, compressors, and pumps that circulate coolants. The challenge of cooling becomes more acute as servers become more powerful and generate more heat per unit of space, and as data centers are built in locations with warm climates where cooling demands are particularly high.
The efficiency with which data centers use energy varies dramatically across facilities, with older centers often using twice as much total energy per unit of computational work compared to modern, optimized facilities. Power Usage Effectiveness, a standard metric for data center efficiency, measures the ratio of total facility energy use to the energy used just for computing. Less efficient data centers might have Power Usage Effectiveness values of two or higher, meaning that for every unit of energy used for computation, an additional unit is consumed by cooling and other overhead. Leading-edge facilities have achieved Power Usage Effectiveness values approaching one point one, indicating that only ten percent additional energy is used for non-computing purposes. This wide variation in efficiency demonstrates both the problem with legacy infrastructure and the potential for significant improvements through better design and management.
The Carbon Footprint of Digital Services
The carbon emissions associated with data centers depend not just on how much electricity they consume but crucially on where that electricity comes from. Data centers powered by electricity from coal or natural gas plants have dramatically higher carbon footprints than those powered by renewable energy sources like wind, solar, or hydroelectric power. The global distribution of data centers means they draw electricity from diverse energy grids with vastly different carbon intensities. A data center in a region with a coal-heavy electricity grid might produce several times more carbon emissions per unit of computation than an identical facility in a location with abundant renewable energy. This geographic variation in carbon intensity makes location decisions critical for the environmental impact of digital infrastructure.
The concept of embodied carbon adds another dimension to understanding data center environmental impact. Beyond the operational emissions from electricity consumption, significant carbon emissions occur during the manufacturing of servers, networking equipment, and facility infrastructure. Mining raw materials, fabricating computer chips, assembling servers, and constructing data center buildings all require energy and generate emissions. When a server is manufactured in one country using materials sourced globally, then shipped to a data center in another region, the cumulative carbon footprint of that hardware includes emissions from this entire supply chain. Embodied carbon can represent a significant portion of the total lifetime emissions of data center equipment, particularly for hardware that is replaced frequently as technology advances.
Understanding the carbon footprint of specific digital services helps quantify the climate impact of everyday online activities. Streaming an hour of high-definition video generates carbon emissions equivalent to driving a car a certain distance, though the exact figure varies based on numerous factors including network efficiency, device type, and electricity sources. Training large artificial intelligence models can produce carbon emissions equivalent to the lifetime emissions of several cars. Sending a single email has a minimal carbon footprint, but when multiplied across billions of messages sent daily worldwide, the cumulative impact becomes substantial. These comparisons, while imperfect and context-dependent, help make abstract carbon emissions tangible by relating them to familiar activities and highlighting that seemingly weightless digital services carry real environmental costs.
Water Consumption and Local Environmental Stress
Water use represents a less visible but equally important environmental challenge for data centers, particularly those using water-based cooling systems. Many data centers employ evaporative cooling, which uses water to absorb heat from servers more efficiently than air-based systems alone. In this process, water evaporates to carry away heat, meaning the water is consumed rather than simply circulated and returned. Large data centers can consume millions of gallons of water daily, placing significant strain on local water supplies. This water demand competes with agricultural, municipal, and industrial uses, and can be particularly problematic in regions already facing water scarcity or drought conditions.
The quality of water required for data center cooling varies by system design, but many facilities need relatively clean water to prevent mineral buildup and equipment damage. This requirement means data centers often compete directly with drinking water supplies rather than being able to use lower-quality water sources that might suffice for other industrial purposes. The treatment and transportation of water to data centers requires additional energy and resources, adding to the overall environmental footprint. In some locations, data center water consumption has sparked community opposition, with local residents questioning whether tech companies should be allowed to use scarce water resources for commercial purposes while communities face usage restrictions during droughts.
Alternative cooling technologies can reduce water consumption but often involve trade-offs with energy efficiency or operational complexity. Air cooling systems eliminate direct water use but typically require more electricity to achieve equivalent cooling capacity. Closed-loop liquid cooling systems that recirculate coolant without evaporation can reduce water consumption but involve different technical challenges and costs. The geographic location of data centers significantly affects both water and energy requirements for cooling, with facilities in cool climates able to use outside air for cooling during much of the year, dramatically reducing both water and energy demands. These climate-dependent efficiency variations have driven some data center operators to build facilities in Nordic countries and other cool regions where natural cooling reduces environmental impact.
Electronic Waste and Hardware Lifecycle Challenges
The rapid pace of technological advancement creates a continuous cycle of hardware upgrades that generates enormous quantities of electronic waste. Data center servers typically have operational lifespans of three to five years before being replaced with newer, more efficient models. This replacement cycle is driven by the economic benefits of newer technology, which can provide more computing power per unit of energy consumed, and by the increasing demands of modern applications that require more powerful hardware. When servers are decommissioned, they become electronic waste that must be properly managed to prevent environmental contamination and recover valuable materials. The sheer volume of equipment cycling through data centers means the industry generates substantial e-waste streams that require responsible handling.
Electronic waste from data centers contains both valuable materials worth recovering and hazardous substances that pose environmental risks if improperly disposed. Circuit boards, processors, and memory modules contain precious metals like gold, silver, and platinum, as well as valuable base metals like copper and aluminum. Recovering these materials through recycling reduces the need for virgin resource extraction and its associated environmental impacts. However, electronic components also contain hazardous materials including lead, mercury, and various toxic chemicals that can contaminate soil and water if e-waste ends up in landfills or is processed unsafely. Proper e-waste management requires specialized facilities and processes to safely extract valuable materials while containing hazardous substances.
The environmental impact of manufacturing replacement hardware extends far beyond the facilities where equipment is assembled. Producing computer chips requires rare earth elements and other materials that must be mined, often in environmentally sensitive regions and sometimes under conditions that cause significant ecological damage. Semiconductor fabrication involves complex chemical processes that consume large amounts of water and energy while generating hazardous waste. The global supply chains that produce data center hardware span multiple continents, with raw materials, components, and finished products transported thousands of miles. This distributed manufacturing system creates carbon emissions from transportation and makes the full environmental impact of hardware difficult to track and quantify accurately.
The Intersection of Digital Growth and Environmental Limits
Global data generation continues to grow exponentially, driven by increasing internet penetration, the proliferation of connected devices, the rise of high-bandwidth applications, and the expansion of data-intensive technologies like artificial intelligence and Internet of Things systems. This data explosion directly translates into increased demand for data center capacity and the associated environmental impacts. Every new streaming service, social media platform, cloud application, and smart device adds to the collective demand for computing and storage resources. The environmental implications of this growth trajectory become concerning when projected forward, as continuing current trends could see data centers consuming a much larger share of global electricity and resources within coming decades.
The challenge of balancing digital innovation with environmental sustainability raises fundamental questions about how society should approach technology development and deployment. Digital technologies provide genuine benefits including improved communications, access to information, business efficiency, scientific advancement, and quality of life improvements. However, these benefits come with real environmental costs that have historically been externalized or ignored. As awareness of climate change and resource constraints grows, the technology industry faces increasing pressure to account for and reduce the environmental impact of digital infrastructure. This pressure comes from regulators implementing environmental requirements, investors considering environmental risks, customers preferring sustainable providers, and employees expecting their employers to act responsibly.
The potential for continued exponential growth in computing demand to collide with planetary boundaries represents a looming challenge that the technology industry and society must address. If data center energy consumption continues growing at recent rates while global efforts to combat climate change require dramatic emissions reductions, something must change fundamentally in how digital infrastructure operates. This tension between digital expansion and environmental sustainability drives urgent innovation in energy efficiency, renewable energy adoption, and more sustainable computing practices. The choices made now about data center design, location, operation, and regulation will significantly influence whether the digital future remains compatible with a stable climate and sustainable resource use.
Concluding Remarks on the Data Scientist’s Skills
This article series has covered the 15 most in-demand skills for data scientists. Learning all of them can be a challenging, even overwhelming, prospect, especially if you are just starting out on your data science journey. But there is no need to stress. Very few, if any, data scientists possess such a comprehensive toolkit at the expert level. The “unicorn” is a myth; data science is a team sport. You should start by acquiring some basic, foundational knowledge. This typically includes proficiency in a programming language like the general-purpose one or the statistical one, a strong command of the query language for relational databases, and a solid understanding of the basics of statistics. From there, you can gradually work your way into other subjects.
Conclusion
But what skills should you learn next? There is no single correct answer. Most likely, your learning journey will depend on the specific requirements of your job or the industry you wish to enter. For example, if you end up working at a company that is heavily invested in a specific cloud provider, you will probably need to acquire deep knowledge of that provider’s cloud computing services. On the other hand, if your company’s primary focus is on developing new AI-powered products, you already know that you need to focus on machine learning and deep learning to get promoted. And if you are simply learning on your own to improve your skills, our advice is quite simple: learn the skills that interest you the most. Your passion and curiosity will be the strongest motivators on this exciting and lifelong career path.