Working with data

Class ressources

• Slides: html, pdf
• Jupyter notebook: intro to pandas

Pandas basic

Introduction to pandas, a Python library used for working with data sets.

• Exercises 1-12 from the intro to pandas notebook

Pandas advanced

Data munging (wrangling) is the process of transforming raw data to a set of data tables that can be used for a variety of downstream purposes such as analytics.

• Exercises 13-31 from the intro to pandas notebook

References

• Gabor & Békés 1.1, 1.5, 1.7
• 10 minutes to pandas
• Video introduction to pandas (8 minutes)
• Video on pandas dataframe (10 minutes)
• Pandas Basics [15 minutes]

Learning objectives

Theory

• Data should not be thought out of its context
• Important questions to ask when starting to collect / work with data
• Importance of the sampling
• Understand types of variables and observations

Practice basic

• create pandas Series
• create pandas DataFrames from Series, dictionaries, lists
• access data in a DataFrame with loc and iloc
• reset index
• rename columns
• access metadata of DataFrames

Practice advanced

• Add new manipulated variables
• Separate char to new variables
• Convert variables to numeric or factor
• Some string manipulations
• Rename variables
• Filter out different observations conditional selection tabulate frequency of a var missing values replace values duplicates
• (Using pipes)
• Sorting data