Coming soon · Technical Reference
Python for Data Analysis
A practical, hands-on guide to manipulating, processing, cleaning, and analyzing structured data in Python using pandas, NumPy, and the Jupyter/IPython ecosystem.
This is a hands-on technical/engineering book — we're building it into Stepcode (tool-by-tool how-to), with a profile to follow.
What it’s about
Written by the creator of pandas, this book teaches the foundational programming skills and library workflows needed to become an effective data analyst in Python. Rather than focusing on statistical methodology, it concentrates on the data-oriented Python toolset—NumPy arrays for fast numerical computing, pandas Series and DataFrames for tabular data wrangling, matplotlib and seaborn for visualization, and IPython/Jupyter for interactive development. Through detailed, reproducible examples and real-world datasets (Bitly links, MovieLens ratings, US baby names, USDA food data, FEC contributions), readers learn to load, clean, transform, merge, reshape, group, aggregate, and visualize data, and to handle time series and feed cleaned data into modeling libraries like statsmodels and scikit-learn. It is ideal both for analysts new to Python and for Python programmers new to data work, serving as a durable foundation for moving on to more advanced data science and machine learning resources.
The through-line
- Who it’s for
- An analyst or programmer who wants to effectively manipulate, clean, and analyze data in Python.
- The problem
- Raw, messy data is hard to load, clean, transform, and analyze, and the Python data tooling is large and confusing to navigate. They feel overwhelmed by the breadth of libraries and options and unsure they're using the right, efficient approach.
- The plan
- Set up a Python environment with the essential data libraries.
- Learn the Python language basics and the IPython/Jupyter interactive workflow.
- Master NumPy arrays and vectorized computation.
- Learn pandas Series and DataFrame for tabular data manipulation.
- Practice loading data from many file formats and sources.
- The payoff
- The reader can confidently load, clean, and prepare messy real-world data. · They use vectorized pandas/NumPy operations to efficiently transform and aggregate data. · They can visualize results and handle time series competently.
See our guide
Related profiles we’ve built
- A Practical Guide To Conjoint Analysis →
- Analytics at Work: Smarter Decisions, Better Results →
- Big Data: A Revolution That Will Transform How We Live, Work, and Think →
- Big Data_ A Very Short Introduction (Very Short Introductions) →
- Business Adventures →
- Business Intelligence Guidebook: From Data Integration to Analytics →
- Data Mining for Business Analytics: Concepts, Techniques, and Applications →
- Data Warehouse and Data Mining →
Additional reading
- Competing on Analytics: The New Science of Winning · Thomas H. Davenport and Jeanne G. Harris
The authors' previous book, which provides the strategic context by describing the earliest and most aggressive adopters of analytics. This book builds on it by providing a 'how-to' guide for all organizations.
- Sources of Power: How People Make Decisions · Gary Klein
Discusses decision-making in high-pressure situations where there is no time for systematic data gathering, providing a contrast to the analytical approach and showing when intuition is necessary.
- The Black Swan: The Impact of the Highly Improbable · Nassim Nicholas Taleb
Argues that statistical analysis is limited because it cannot predict rare, high-impact 'black swan' events, serving as a cautionary note on the limits of analytics.
- Moneyball: The Art of Winning an Unfair Game · Michael Lewis
A popular case study of how the Oakland A's baseball team used an analytical approach to player selection to compete with richer teams, illustrating the power of competing on analytics.
- Why Great Leaders Don't Take Yes for an Answer · Michael Roberto
Describes how to foster a culture of constructive conflict and debate in decision-making processes, which is essential for an analytical culture where assumptions are tested and merit triumphs over politics.
- The Visual Display of Quantitative Information · Edward Tufte
A foundational work on how to create clear visual representations of data, a key skill for communicating analytical findings effectively.
- Super Crunchers: Why Thinking-By-Numbers Is the New Way to Be Smart · Ian Ayres
The book discusses how statistical analyses are replacing human intuition and expert judgment in decision-making, a core theme related to the discussion of 'Moneyball' and the demise of the expert.
- Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed · James Scott
Documents how governments' fetish for quantification and data can lead to misguided and harmful policies, providing a deep historical context for the book's warnings about the 'dictatorship of data'.
- The War Managers · Douglas Kinnard
A survey of U.S. generals' views on the Vietnam War, revealing that the 'body count' metric was seen as a worthless and inflated measure of progress, illustrating the dangers of relying on flawed data.
- Thinking, Fast and Slow · Daniel Kahneman
Explains the cognitive biases that lead humans to see illusory causal links, which the author's argue big data correlations can challenge and disprove.