Pandas Introduction: Series, DataFrame and Data Analysis

Pandas is one of the most important Python libraries for anyone who wants to learn data analysis, data science, machine learning, or artificial intelligence. When you start working with real-world datasets, you quickly realize that handling data is not only about performing mathematical calculations. Real datasets can contain thousands or millions of rows, different types of information, missing values, duplicate records, text, dates, categories, and many other problems that need to be handled before meaningful analysis can begin.

This is where Pandas becomes extremely useful.

Pandas provides powerful data structures and tools that make it easier to work with labeled and tabular data. The official Pandas documentation describes it as a Python package designed for fast, flexible, and expressive data structures and data analysis tools, with Series and DataFrame serving as its two primary data structures.

In this Day 37 lesson, we will build a strong foundation in Pandas. We will understand what Pandas is, why it is important in data science, how it differs from NumPy, how to install and import it, what Series and DataFrame mean, how to create our first Pandas objects, where Pandas is used in the real world, how a typical Pandas workflow works, and which basic terms every beginner should understand.

What Is Pandas in Python?

Pandas is an open-source Python library used primarily for working with structured and labeled data. It provides convenient tools for loading, organizing, inspecting, cleaning, transforming, analyzing, and preparing data.

The name Pandas is commonly associated with the phrase “Panel Data,” although today the library is much broader than that original concept.

If you have worked with Excel spreadsheets before, you can think of a Pandas DataFrame as a programmable table that you can control using Python.

For example, imagine that a company has the following student-like dataset.

A human can easily look at this table and understand it, but imagine having 100,000 rows of data. Manually checking the information would become extremely difficult.

Pandas allows Python to work with this type of data programmatically.

Instead of manually checking every row, you can write Python code to find the highest marks, calculate the average, filter students above a certain score, identify missing information, group students by categories, or combine this dataset with another dataset.

That is the real power of Pandas.

Why Is Pandas Important in Data Science?

Data science is not only about machine learning algorithms. A large portion of practical data science involves preparing and understanding data before a model is ever created.

A typical data science project may begin with raw data collected from a CSV file, Excel spreadsheet, database, website, API, application, or another system.

The raw data may not be ready for analysis.

It may contain missing values, duplicate records, inconsistent text, incorrect data types, unnecessary columns, or values that need to be transformed.

Pandas provides tools for handling many of these situations.

A simplified data analysis process can look like this.

Pandas is particularly valuable because it can support many of these stages within a single Python-based workflow.

This makes Pandas an important skill for beginners who want to move from Python programming toward practical data science.

Pandas vs NumPy

Before learning Pandas, it is important to understand its relationship with NumPy.

You have already completed NumPy, so this is a good point to connect the two libraries.

NumPy is primarily designed for numerical computing and multidimensional arrays. Pandas is designed more specifically for data analysis and labeled data.

The difference becomes clearer when we compare their basic structures.

For example, NumPy can create an array like this:

import numpy as np

arr = np.array([10, 20, 30, 40])

print(arr)

Pandas can create a Series like this:

import pandas as pd

marks = pd.Series([10, 20, 30, 40])

print(marks)

Both structures can store numerical values, but the Pandas Series provides an index and is designed as a labeled data structure.

Pandas also works closely with NumPy. Understanding NumPy first therefore gives you a useful foundation for learning Pandas.

Installing Pandas

Before using Pandas, you need to make sure that it is installed in your Python environment.

If you are using pip, you can install Pandas using the following command.

pip install pandas

If you are using Conda or a Conda-based environment, Pandas can also be installed through Conda.

conda install pandas

The current official Pandas installation documentation also provides Conda and pip installation options and recommends using an appropriate Python environment when working with Pandas.

In many Python environments, especially common data-science distributions, Pandas may already be installed.

If you run the installation command and see a message indicating that the requirement is already satisfied, it generally means that Pandas is already available in that environment.

Importing Pandas

Installing Pandas and importing Pandas are two different things.

Installation makes the library available in your Python environment.

Importing makes the library available to the current Python program.

The most common way to import Pandas is:

import pandas as pd

Here, pandas is the library name and pd is the alias we use to refer to Pandas.

The alias is not mandatory, but pd is the standard convention used in most Pandas examples and documentation.

For example, without an alias you could write:

import pandas

data = pandas.DataFrame()

With the commonly used alias, you can write:

import pandas as pd

data = pd.DataFrame()

The second approach is shorter and easier to read.

It is important to understand that pd is only a variable name referring to the imported Pandas module. It is not a separate library.

Understanding Pandas Series

One of the two fundamental data structures in Pandas is called a Series.

A Series is a one-dimensional labeled data structure.

The easiest way to understand a Series is to imagine a single column of data with an index.

For example:

This can be created using:

import pandas as pd

marks = pd.Series([85, 90, 78, 92])

print(marks)

The output will look similar to:

0    85
1    90
2    78
3    92
dtype: int64

Notice that Pandas automatically created the index values 0, 1, 2, and 3.

The values are 85, 90, 78, and 92.

The final line tells us the data type of the values.

A Series can contain integers, floating-point numbers, strings, dates, and other Python-compatible objects. The official Pandas documentation defines a Series as a one-dimensional labeled array capable of holding different data types.

A useful beginner-friendly way to remember it is:

Series = one-dimensional labeled data.

You can also think of it as something similar to one column of an Excel table.

Understanding Pandas DataFrame

The second major Pandas data structure is the DataFrame.

A DataFrame is a two-dimensional labeled data structure containing rows and columns.

You can think of it as a complete table.

For example:

This can be represented in Python using:

import pandas as pd

data = {
    "Name": ["Rahul", "Priya", "Aman"],
    "Age": [21, 22, 20],
    "Marks": [85, 90, 78]
}

df = pd.DataFrame(data)

print(df)

The resulting DataFrame looks like:

    Name  Age  Marks
0  Rahul   21     85
1  Priya   22     90
2   Aman   20     78

The DataFrame has multiple columns, and each row represents a record.

The official Pandas documentation describes a DataFrame as a two-dimensional, size-mutable and potentially heterogeneous tabular data structure with labeled rows and columns.

A simple way to remember it is:

DataFrame = two-dimensional labeled table.

Series vs DataFrame

The difference between Series and DataFrame is one of the first concepts you should become comfortable with.

Consider the following DataFrame:

    Name  Age  Marks
0  Rahul   21     85
1  Priya   22     90
2   Aman   20     78

The Marks column can be represented as a Series.

df["Marks"]

The entire table is a DataFrame.

df

This relationship becomes extremely important when you start selecting and manipulating data in later Pandas lessons.

Creating Your First Pandas Series

Let us create a Series step by step.

import pandas as pd

marks = pd.Series([85, 90, 78, 92])

print(marks)

The output is:

0    85
1    90
2    78
3    92
dtype: int64

The first column represents the index.

The second column represents the values.

Pandas automatically starts the default index from zero.

You can also provide your own index.

import pandas as pd

marks = pd.Series(
    [85, 90, 78],
    index=["Rahul", "Priya", "Aman"]
)

print(marks)

The output becomes:

Rahul    85
Priya    90
Aman     78
dtype: int64

Now the index contains meaningful labels instead of automatically generated numbers.

This idea of labeled data is one of the most important differences between basic Python lists and Pandas data structures.

Creating Your First DataFrame

Now let us create a DataFrame.

import pandas as pd

data = {
    "Name": ["Rahul", "Priya", "Aman"],
    "Age": [21, 22, 20],
    "Marks": [85, 90, 78]
}

df = pd.DataFrame(data)

print(df)

The result is:

    Name  Age  Marks
0  Rahul   21     85
1  Priya   22     90
2   Aman   20     78

The dictionary keys become column names.

The list values become the data inside those columns.

Pandas automatically creates a row index because we did not provide one.

This is one of the most common ways beginners create DataFrames.

Why Are Indexes Important?

An index identifies rows in a Pandas object.

For the following DataFrame:

    Name  Age  Marks
0  Rahul   21     85
1  Priya   22     90
2   Aman   20     78

the values 0, 1, and 2 are the row index.

The index helps Pandas identify and work with individual rows.

Indexes can also contain meaningful labels.

For example:

data = {
    "Marks": [85, 90, 78]
}

df = pd.DataFrame(
    data,
    index=["Rahul", "Priya", "Aman"]
)

print(df)

The result becomes:

       Marks
Rahul     85
Priya     90
Aman      78

Later, when we learn loc[] and iloc[], understanding indexes will become especially important.

Understanding Rows and Columns

A row represents one record or observation.

A column generally represents one particular attribute or category of information.

Consider this dataset.

The Name column contains names.

The Age column contains ages.

The Marks column contains marks.

The first row represents Rahul's complete record.

The second row represents Priya's complete record.

The third row represents Aman's complete record.

This row-and-column structure is what makes DataFrames particularly useful for real-world tabular datasets.

Real-World Uses of Pandas

Pandas is used across many industries because structured data exists almost everywhere.

In business and sales analysis, Pandas can be used to study revenue, sales performance, customer purchases, product performance, and regional performance.

In finance, Pandas can be used to work with stock prices, financial transactions, expenses, budgets, and historical market data.

In marketing, Pandas can help analyze campaign performance, customer segments, advertising data, website traffic, and lead information.

In education, Pandas can be used to analyze student marks, attendance, examination results, course performance, and enrollment information.

In healthcare-related analytics, Pandas can be used to organize and analyze datasets containing appointments, measurements, records, and other structured information, subject to the appropriate privacy and data-handling requirements.

In website and application analytics, Pandas can be used to study traffic, user activity, clicks, conversions, and engagement.

The official Pandas overview specifically highlights tabular data such as SQL tables and Excel spreadsheets, time-series data, matrix-like data, and observational or statistical datasets as suitable use cases.

The important idea is that Pandas is not limited to one industry.

If your work involves structured data, Pandas can often become a useful part of the workflow.

Pandas Data Analysis Workflow

Learning individual Pandas functions is useful, but understanding the overall workflow is even more important.

A typical workflow begins with raw data.

The data may come from a CSV file, Excel spreadsheet, database, API, or another source.

The next step is to load that data into a Pandas DataFrame.

After loading it, you should inspect the dataset.

You may want to check the first few rows, the number of rows and columns, the column names, the data types, and summary statistics.

Once you understand the data, you can begin cleaning it.

Cleaning may involve handling missing values, removing duplicates, correcting incorrect values, standardizing text, or converting data types.

After cleaning, you can filter and select the information you need.

You can then transform the data, create new columns, group records, calculate statistics, and combine datasets.

Finally, you can analyze the results and use visualization libraries to communicate your findings.

The workflow can be represented as:

You should not think of these steps as completely separate tasks.

They are connected parts of the data analysis process.

Basic Pandas Terminology

Before moving deeper into Pandas, you need to understand some basic terminology.

For example, consider this DataFrame.

       Name  Age  Marks
0      Rahul  21     85
1      Priya  22     90
2       Aman  20     78

Here, Name, Age, and Marks are columns.

The values 0, 1, and 2 form the default index.

Each horizontal entry is a row.

The values such as Rahul, 21, and 85 are the actual data.

The data type tells us what kind of values a column contains.

The shape tells us how many rows and columns the DataFrame contains.

Understanding Shape

The shape attribute tells us the number of rows and columns.

For example:

print(df.shape)

If the DataFrame contains three rows and three columns, the output is:

(3, 3)

The first value represents the number of rows.

The second value represents the number of columns.

This follows the format:

(rows, columns)

Understanding this is important because later you will frequently use shape while inspecting datasets.

Understanding dtype

Data type tells us what kind of data is stored.

For example:

print(df.dtypes)

You may see output similar to:

Name     object
Age       int64
Marks     int64
dtype: object

The exact dtype representation can vary with the Pandas and Python environment.

The important idea is that Age and Marks contain integer values, while Name contains text.

Understanding data types becomes extremely important when cleaning and transforming datasets.

Pandas Is More Than Just Tables

A common beginner mistake is to think that Pandas is simply a tool for creating tables.

It is much more than that.

Pandas gives you a programming interface for interacting with data.

You can load files.

You can inspect datasets.

You can select columns.

You can filter rows.

You can clean missing values.

You can remove duplicates.

You can sort records.

You can calculate statistics.

You can group data.

You can combine datasets.

You can work with dates and text.

You can prepare data for visualization and machine learning.

This is why learning Pandas is such an important step after learning Python and NumPy.

A Small Complete Example

Let us combine several concepts from today's lesson.

import pandas as pd

data = {
    "Name": ["Rahul", "Priya", "Aman", "Neha"],
    "Age": [21, 22, 20, 23],
    "Marks": [85, 90, 78, 92]
}

df = pd.DataFrame(data)

print("Student Data:")
print(df)

print("\nShape:")
print(df.shape)

print("\nColumns:")
print(df.columns)

print("\nData Types:")
print(df.dtypes)

This small program creates a DataFrame and then examines some of its basic properties.

The important point is not to memorize every command immediately.

Instead, understand what each command is trying to tell you about your dataset.

What You Should Remember From Day 37

Pandas is a Python library designed for practical data analysis and manipulation.

Its two primary data structures are Series and DataFrame.

A Series is one-dimensional and labeled.

A DataFrame is two-dimensional and organized into rows and columns.

Pandas uses indexes to identify rows.

DataFrame columns represent different fields or attributes.

Pandas works especially well with tabular and labeled data.

Pandas and NumPy are closely related, but they focus on somewhat different problems.

NumPy is strongly focused on numerical computing and arrays, while Pandas provides higher-level tools for practical data analysis.

The standard import convention is:

import pandas as pd

A typical Pandas workflow can be summarized as:

Read → Inspect → Clean → Select → Filter → Transform → Analyze → Visualize → Report

The goal is not simply to learn Pandas syntax.

The real goal is to learn how to use Python to turn raw data into useful information.

Frequently Asked Questions About Pandas

Is Pandas a programming language?

No. Pandas is a Python library. Python is the programming language, while Pandas provides specialized data structures and functions for data analysis.

Is Pandas difficult for beginners?

Pandas can look confusing initially because it introduces concepts such as Series, DataFrame, indexes, labels, and vectorized operations. However, if you already understand Python lists, dictionaries, functions, and NumPy arrays, you already have a useful foundation for learning Pandas.

What is the difference between Series and DataFrame?

A Series is one-dimensional labeled data, while a DataFrame is two-dimensional labeled tabular data containing rows and columns.

Why is Pandas used in data science?

Pandas makes it easier to load, inspect, clean, transform, filter, combine, and analyze structured datasets before and during the data science process.

Is Pandas better than NumPy?

It is not useful to describe one as simply better than the other. They solve different problems. NumPy is fundamental for numerical and array-based computing, while Pandas provides higher-level tools for labeled and tabular data analysis.

Can Pandas read Excel files?

Yes. Pandas provides functions for working with several common data formats, including CSV, Excel, JSON, SQL-related sources, and others.

Do I need NumPy before learning Pandas?

You can learn Pandas without deep NumPy knowledge, but understanding NumPy is very helpful because Pandas is closely connected to the Python numerical-computing ecosystem.

Day 37 Practice Questions

The following questions are designed to test today's concepts without immediately giving away the answers.