Analyze NYC Property Sales Data

Clean and analyze 845K+ NYC property sales to find investment opportunities.

Introduction

30 Second Summary

Every day, thousands of properties change hands in New York City. Behind each sale is a data point that reveals where money is flowing, which neighborhoods are heating up, and where smart investors are placing their bets.

In this project, you will analyze 845,000+ real NYC property sales records to answer a business question: where should a real estate investment firm focus its next round of residential acquisitions? You will clean messy government data, build professional visualizations, and write a data-driven recommendation.

What You'll Build

A complete data analysis project using Python, pandas, and seaborn that transforms raw NYC Department of Finance data into actionable investment insights.

By the end of this project, you'll have:

  • A cleaned dataset of 500,000+ valid property sales extracted from messy raw government data.
  • Professional data visualizations comparing borough pricing, neighborhood value, and market patterns.
  • A business recommendation memo and portfolio-ready GitHub repository that hiring managers can evaluate in under 3 minutes.
  • Secret Mission: A year-over-year price trend analysis revealing the fastest-appreciating neighborhoods in NYC.

Are there any prerequisites?

You should have basic Python familiarity and a general understanding of how tabular data works (spreadsheets or dataframes). No prior experience with pandas or data visualization is required.

Set Up Your Analysis Environment

Before you can analyze NYC property sales, your notebook needs the right tools loaded and the data accessible. Think of this step as laying out your workbench.

You'll import three Python libraries that handle different parts of the analysis workflow. Then you'll load the full 845,000+ record dataset into memory so it's ready for exploration.

In this step, get ready to:
  • Import pandas, matplotlib, and seaborn into your notebook.
  • Configure a consistent visualization style for all your charts.
  • Load the NYC property sales CSV into a DataFrame and confirm it loaded correctly.
Import your libraries and configure visualization style

Each library handles a specific part of your workflow. pandas handles data manipulation, matplotlib creates the actual chart graphics, and seaborn adds statistical visualization capabilities on top of matplotlib.

  • In your nyc_property_analysis.ipynb notebook, click into the first empty cell and paste the following code:
# Cell 1: Setup and Imports
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# Set visualization style
sns.set_style("whitegrid")
plt.rcParams["figure.figsize"] = (12, 6)

What does this code do?

  • import pandas as pd loads pandas and gives it the short alias pd so you can type less throughout the notebook.
  • sns.set_style("whitegrid") applies a clean grid background to every chart you create. This makes data points easier to read.
  • plt.rcParams["figure.figsize"] = (12, 6) sets all charts to 12 inches wide by 6 inches tall by default, so you don't have to specify dimensions every time.
  • Run the cell by pressing Shift+Enter.
  • You should see no output and no errors. The cell number to the left updates from [ ] to [1], confirming it ran successfully.

Seeing a ModuleNotFoundError?

This means the library isn't installed in the Python environment your notebook is using. Open a terminal and run pip install pandas matplotlib seaborn, then restart your notebook kernel.

Help me fix a ModuleNotFoundError in my Jupyter Notebook

Load the dataset into a DataFrame

Your nyc_sales.csv file contains 845,000+ property sale records from the NYC Department of Finance. Loading it into a pandas DataFrame lets you filter, group, and analyze the data using Python.

  • In the next empty cell (Cell 2), paste the following code to load and preview the data:
# Cell 2: Load the Data
df = pd.read_csv("nyc_sales.csv")
print(f"Dataset shape: {df.shape}")
print(f"\nColumn names:\n{df.columns.tolist()}")
df.head()

What does this code do?

  • pd.read_csv("nyc_sales.csv") reads the entire CSV file into a DataFrame called df. This might take a few seconds given the file size.
  • df.shape prints the number of rows and columns so you can confirm the full dataset loaded.
  • df.head() displays the first 5 rows as a formatted table, giving you a quick look at what the data contains.
  • Run the cell by pressing Shift+Enter.
  • You should see output showing a dataset shape of approximately (845845, 30), a list of all column names, and a table showing the first 5 rows of property sale records.

What should I see?

The shape output tells you how many records you're working with. 845,000+ rows means you have nearly a decade of NYC property sales data.

The column list includes fields like BOROUGH, NEIGHBORHOOD, BUILDING CLASS CATEGORY, SALE PRICE, SALE DATE, LAND SQUARE FEET, and GROSS SQUARE FEET. You'll use these throughout the project.

Seeing a FileNotFoundError?

This means pandas can't find nyc_sales.csv in the same folder as your notebook. Make sure the CSV file is saved in your nyc-property-analysis folder alongside nyc_property_analysis.ipynb.

Also check that the filename matches exactly. It should be nyc_sales.csv with underscores, not hyphens or spaces.

Help me fix a FileNotFoundError when loading my CSV in Jupyter

✔️ Awesome, I've got everything!

Great. Make sure you've saved your notebook before moving on.

ⓧ I'd like to double check the full code

Here's what your notebook should contain so far across both cells:

# Cell 1: Setup and Imports
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# Set visualization style
sns.set_style("whitegrid")
plt.rcParams["figure.figsize"] = (12, 6)

# Cell 2: Load the Data
df = pd.read_csv("nyc_sales.csv")
print(f"Dataset shape: {df.shape}")
print(f"\nColumn names:\n{df.columns.tolist()}")
df.head()

Your analysis environment is ready and the full dataset is loaded. Next up, you'll dig into the raw data to understand its structure and uncover quality issues that need fixing before any real analysis can happen.

Explore and Understand the Raw Data

In the last step, you loaded 845,000+ rows of NYC property sales data into a DataFrame. Before you can analyze anything meaningful, you need to understand what you're working with.

Real-world data is messy. Government datasets often have missing values, encoding inconsistencies, and records that don't represent what you think they do. In this step, you'll profile the dataset to uncover these issues so you know exactly what to fix before running any analysis.

In this step, get ready to:
  • Inspect the dataset's structure, data types, and null values.
  • Identify data quality issues like $0 sale prices and mistyped columns.
  • Document your findings in markdown cells for a clean notebook narrative.
Inspect the dataset structure

You already saw the shape and column names in the previous step. Now you need to dig deeper into data types and missing values to understand what pandas is actually storing.

  • Create a new code cell below Cell 2 and add the following code to examine data types, null counts, and sale price statistics:
# Cell 3: Initial Exploration
print(f"Data types:\n{df.dtypes}")
print(f"\nNull values:\n{df.isnull().sum()}")
print(f"\nSale Price statistics:\n{df['SALE PRICE'].describe()}")

What does this code do?

This cell gives you three critical pieces of information about your raw data.

  • df.dtypes shows how pandas interpreted each column. Columns that should be numbers but show as object indicate text data that needs conversion.
  • df.isnull().sum() counts missing values per column. Columns with high null counts need special handling.
  • df['SALE PRICE'].describe() shows the statistical summary of sale prices, which reveals outliers and suspicious values.
  • Run the cell by pressing Shift+Enter.
  • You should see a long list of column data types, null value counts per column, and a statistical summary showing the sale price min, max, mean, and quartiles.

Seeing a NameError for df?

This means Cell 2 hasn't been executed yet. Jupyter cells must be run in order. Go back to Cell 1 and Cell 2 and run them first with Shift+Enter, then run Cell 3 again.

help me fix a NameError in my Jupyter Notebook

Identify data quality issues

The describe output likely shows a minimum sale price of $0. That seems wrong for a property sale. You need to quantify how many records have suspicious prices and check whether those square footage columns are actually stored as text.

  • Create a new code cell below Cell 3 and add the following code to quantify the data quality problems:
# Cell 4: Identify Data Quality Issues
print(f"Records with $0 sale price: {(df['SALE PRICE'] == 0).sum()}")
print(f"Records with sale price <= $10: {(df['SALE PRICE'] <= 10).sum()}")
print(f"\nLAND SQUARE FEET dtype: {df['LAND SQUARE FEET'].dtype}")
print(f"GROSS SQUARE FEET dtype: {df['GROSS SQUARE FEET'].dtype}")
print(f"\nSample LAND SQUARE FEET values: {df['LAND SQUARE FEET'].head(10).tolist()}")

What does this code do?

This cell investigates three specific data quality issues you'll need to fix before analyzing.

  • The $0 and $10 checks count records where the sale price indicates an ownership transfer (like between family members) rather than a real market sale. You should see around 262,000 records with $0 prices.
  • The dtype checks confirm whether LAND SQUARE FEET and GROSS SQUARE FEET are stored as object (text) instead of numbers. This happens because the raw data has comma formatting like "2,000".
  • The sample values let you see the actual text format so you know what to clean.
  • Run the cell with Shift+Enter.
  • You should see counts of $0 and low-price records, both square footage columns showing object as their dtype, and sample values with comma-formatted strings.

Why are $0 sales a problem?

In NYC property records, a $0 or $10 sale price usually means the property was transferred between related parties, given as a gift, or changed ownership through legal proceedings. These are not market transactions.

If you include them in your analysis, they'll drag down every average and distort your price comparisons. You'll remove them in the next step.

Seeing a KeyError for SALE PRICE?

Column names in this dataset use ALL CAPS with spaces. Make sure you typed SALE PRICE exactly as shown (not Sale Price or sale_price). You can verify column names by running df.columns.tolist() in a new cell.

help me fix a KeyError in pandas

Document your findings in markdown cells

A professional notebook tells a story. Anyone reading your notebook (including hiring managers) should understand what you found without re-running the code. Markdown cells between your code cells explain your observations and reasoning.

  • Create a new Markdown cell above Cell 3. In Jupyter, click the cell type dropdown (it says "Code" by default) and change it to Markdown.
  • Add a heading and your observations about the dataset structure. For example:
## Data Exploration

This dataset contains 845,000+ NYC property sale records. Key observations:

- Multiple columns have significant null values (especially square footage fields with ~244,000 nulls)
- SALE PRICE has a minimum of $0, indicating non-market transactions are included
- LAND SQUARE FEET and GROSS SQUARE FEET are stored as text (object type) instead of numbers

Write what YOU observe

The markdown above is a starting point. Replace the specific numbers with what your output actually shows. The goal is to practice documenting your thought process as you explore data, not to copy text verbatim.

  • Create another Markdown cell after Cell 4 to document the data quality issues you found. Include the count of $0 records and note the text-typed columns.
  • Run both markdown cells with Shift+Enter to render them. You should see formatted headings and bullet points in your notebook.

✔️ Awesome, I've got everything!

Great. Make sure you've saved your notebook before continuing.

ⓧ I'd like to double check the full code

Here is the full notebook code through Step 2:

# Cell 1: Setup and Imports
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# Set visualization style
sns.set_style("whitegrid")
plt.rcParams["figure.figsize"] = (12, 6)

# Cell 2: Load the Data
df = pd.read_csv("nyc_sales.csv")
print(f"Dataset shape: {df.shape}")
print(f"\nColumn names:\n{df.columns.tolist()}")
df.head()

# Cell 3: Initial Exploration
print(f"Data types:\n{df.dtypes}")
print(f"\nNull values:\n{df.isnull().sum()}")
print(f"\nSale Price statistics:\n{df['SALE PRICE'].describe()}")

# Cell 4: Identify Data Quality Issues
print(f"Records with $0 sale price: {(df['SALE PRICE'] == 0).sum()}")
print(f"Records with sale price <= $10: {(df['SALE PRICE'] <= 10).sum()}")
print(f"\nLAND SQUARE FEET dtype: {df['LAND SQUARE FEET'].dtype}")
print(f"GROSS SQUARE FEET dtype: {df['GROSS SQUARE FEET'].dtype}")
print(f"\nSample LAND SQUARE FEET values: {df['LAND SQUARE FEET'].head(10).tolist()}")

You now have a clear picture of what's wrong with the raw data: hundreds of thousands of non-market transactions, text-encoded numbers, and significant null values. Next up, you'll clean all of this so the dataset is ready for real analysis.

Clean the Data for Analysis

In the previous step, you explored the raw dataset and identified some serious data quality problems. Over 262,000 records have $0 sale prices, square footage columns are stored as text, and the dataset contains commercial properties you don't need.

Now it's time to fix those issues. Data cleaning is where you transform messy raw data into something you can actually analyze. By the end of this step, you'll have a focused dataset of valid residential property sales ready for borough-level comparisons.

In this step, get ready to:
  • Remove non-sale records that skew your analysis.
  • Convert text columns to proper numeric types.
  • Filter the dataset to residential properties and map borough names.
Remove non-sale records

Many records in this dataset aren't actual market sales. A $0 or $10 sale price usually means an ownership transfer between family members or entities. These records would drag your median prices down and misrepresent the real market.

  • Create a new cell in your notebook (Cell 5) and add the following code to filter out non-market transactions:
# Cell 5: Data Cleaning - Remove Non-Sales
# $0 and very low prices indicate ownership transfers, not market sales
df_clean = df[df["SALE PRICE"] > 10000].copy()
print(f"Records removed: {len(df) - len(df_clean)}")
print(f"Records remaining: {len(df_clean)}")

What does this code do?

The filter df["SALE PRICE"] > 10000 keeps only records where someone paid more than $10,000. This removes ownership transfers ($0), nominal sales ($10), and other non-market transactions.

The .copy() creates an independent DataFrame so changes to df_clean won't affect the original df.

  • Run the cell by pressing Shift+Enter.
  • You should see output showing approximately 270,000+ records removed and the remaining count of valid sales.

Seeing a KeyError?

Make sure the column name is exactly SALE PRICE with a space (not underscore). Check your Cell 2 output to confirm the column names.

Also confirm you ran all previous cells first. Cells 1 and 2 must run before this cell can access df.

Help me debug my data cleaning cell

Convert text columns to numeric types

In Step 2, you discovered that LAND SQUARE FEET and GROSS SQUARE FEET are stored as text with comma formatting (like "2,000"). You can't do math on text, so you need to convert these to numbers.

  • Create a new cell (Cell 6) and add the following code to convert the columns:
# Cell 6: Data Cleaning - Convert Numeric Columns
df_clean["LAND SQUARE FEET"] = pd.to_numeric(df_clean["LAND SQUARE FEET"].str.replace(",", ""), errors="coerce")
df_clean["GROSS SQUARE FEET"] = pd.to_numeric(df_clean["GROSS SQUARE FEET"].str.replace(",", ""), errors="coerce")
df_clean["SALE DATE"] = pd.to_datetime(df_clean["SALE DATE"])

print(f"Null LAND SQUARE FEET after conversion: {df_clean['LAND SQUARE FEET'].isnull().sum()}")
print(f"Null GROSS SQUARE FEET after conversion: {df_clean['GROSS SQUARE FEET'].isnull().sum()}")

What does this code do?

  • str.replace(",", "") strips the comma formatting from values like "2,000" so they become "2000".
  • pd.to_numeric() converts the cleaned text to actual numbers. The errors="coerce" parameter turns unconvertible values into NaN instead of crashing.
  • pd.to_datetime() converts the SALE DATE text column to proper date objects, which you'll use later to filter by year.
  • Run the cell by pressing Shift+Enter.
  • You should see null counts for both square footage columns. These represent records where the original data was blank or couldn't be converted to a number.

Why keep the nulls instead of dropping them?

Not every analysis needs square footage. Dropping rows with null square footage would remove valid sale price data you still need for borough-level comparisons. You'll filter to non-null square footage later, only when you calculate price per square foot.

Seeing an AttributeError on str.replace?

This error means the column is already numeric (not text). Check df_clean["LAND SQUARE FEET"].dtype. If it says float64 or int64, you can skip the str.replace() step for that column.

Help me fix my column type conversion

Filter to residential properties and map boroughs

The dataset contains commercial buildings, parking garages, industrial lots, and other property types that aren't relevant to your residential investment question. Filtering to residential categories keeps your analysis focused on the properties your hypothetical firm would actually buy.

  • Create a new cell (Cell 7) and add the following code to filter to residential properties:
# Cell 7: Filter to Residential Properties
residential_categories = [
    "01 ONE FAMILY DWELLINGS",
    "02 TWO FAMILY DWELLINGS",
    "03 THREE FAMILY DWELLINGS",
    "07 RENTALS - WALKUP APARTMENTS",
    "09 COOPS - WALKUP APARTMENTS",
    "10 COOPS - ELEVATOR APARTMENTS",
    "12 CONDOS - WALKUP APARTMENTS",
    "13 CONDOS - ELEVATOR APARTMENTS",
    "15 CONDOS - 2-10 UNIT RESIDENTIAL",
]

df_residential = df_clean[df_clean["BUILDING CLASS CATEGORY"].isin(residential_categories)].copy()
print(f"Residential sales: {len(df_residential)}")
print(f"\nBreakdown by category:")
print(df_residential["BUILDING CLASS CATEGORY"].value_counts())

What does this code do?

The residential_categories list defines exactly which building class categories count as residential. These 9 categories cover single-family homes, multi-family dwellings, co-ops, condos, and rental apartments.

The .isin() method checks whether each row's building class matches any value in the list. The result is a filtered DataFrame containing only residential sales.

  • Run the cell by pressing Shift+Enter.
  • You should see a count of residential sales and a breakdown showing how many transactions exist in each category. One-family dwellings and co-op/condo elevator apartments typically have the highest counts.

Seeing 0 residential sales?

The category strings must match the dataset exactly, including spacing and capitalization. Run df_clean["BUILDING CLASS CATEGORY"].unique() in a temporary cell to see the actual values in your data and compare them against your list.

Help me debug my residential filter

The dataset uses numeric borough codes (1 through 5) instead of names. Adding a readable borough name column makes your charts and analysis much clearer.

  • Create a new cell (Cell 8) and add the following code to map borough codes to names:
# Cell 8: Borough Mapping
borough_map = {
    "1": "Manhattan",
    "2": "Bronx",
    "3": "Brooklyn",
    "4": "Queens",
    "5": "Staten Island"
}
df_residential["BOROUGH_NAME"] = df_residential["BOROUGH"].astype(str).map(borough_map)

What does this code do?

  • borough_map is a dictionary that translates each numeric code to its borough name.
  • .astype(str) converts the BOROUGH column to strings so the dictionary keys ("1", "2", etc.) match correctly.
  • .map(borough_map) replaces each code with its corresponding name, creating a new BOROUGH_NAME column.
  • Run the cell by pressing Shift+Enter.

You won't see any output from this cell. The new column will be used in the next step when you create charts comparing boroughs.

✔️ Awesome, I've got everything!

Great. Double check you've saved your notebook before moving on.

ⓧ I'd like to double check the full code

Here's the complete notebook up through Cell 8. Compare your cells against this:

# Cell 1: Setup and Imports
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# Set visualization style
sns.set_style("whitegrid")
plt.rcParams["figure.figsize"] = (12, 6)

# Cell 2: Load the Data
df = pd.read_csv("nyc_sales.csv")
print(f"Dataset shape: {df.shape}")
print(f"\nColumn names:\n{df.columns.tolist()}")
df.head()

# Cell 3: Initial Exploration
print(f"Data types:\n{df.dtypes}")
print(f"\nNull values:\n{df.isnull().sum()}")
print(f"\nSale Price statistics:\n{df['SALE PRICE'].describe()}")

# Cell 4: Identify Data Quality Issues
print(f"Records with $0 sale price: {(df['SALE PRICE'] == 0).sum()}")
print(f"Records with sale price <= $10: {(df['SALE PRICE'] <= 10).sum()}")
print(f"\nLAND SQUARE FEET dtype: {df['LAND SQUARE FEET'].dtype}")
print(f"GROSS SQUARE FEET dtype: {df['GROSS SQUARE FEET'].dtype}")
print(f"\nSample LAND SQUARE FEET values: {df['LAND SQUARE FEET'].head(10).tolist()}")

# Cell 5: Data Cleaning - Remove Non-Sales
# $0 and very low prices indicate ownership transfers, not market sales
df_clean = df[df["SALE PRICE"] > 10000].copy()
print(f"Records removed: {len(df) - len(df_clean)}")
print(f"Records remaining: {len(df_clean)}")

# Cell 6: Data Cleaning - Convert Numeric Columns
df_clean["LAND SQUARE FEET"] = pd.to_numeric(df_clean["LAND SQUARE FEET"].str.replace(",", ""), errors="coerce")
df_clean["GROSS SQUARE FEET"] = pd.to_numeric(df_clean["GROSS SQUARE FEET"].str.replace(",", ""), errors="coerce")
df_clean["SALE DATE"] = pd.to_datetime(df_clean["SALE DATE"])

print(f"Null LAND SQUARE FEET after conversion: {df_clean['LAND SQUARE FEET'].isnull().sum()}")
print(f"Null GROSS SQUARE FEET after conversion: {df_clean['GROSS SQUARE FEET'].isnull().sum()}")

# Cell 7: Filter to Residential Properties
residential_categories = [
    "01 ONE FAMILY DWELLINGS",
    "02 TWO FAMILY DWELLINGS",
    "03 THREE FAMILY DWELLINGS",
    "07 RENTALS - WALKUP APARTMENTS",
    "09 COOPS - WALKUP APARTMENTS",
    "10 COOPS - ELEVATOR APARTMENTS",
    "12 CONDOS - WALKUP APARTMENTS",
    "13 CONDOS - ELEVATOR APARTMENTS",
    "15 CONDOS - 2-10 UNIT RESIDENTIAL",
]

df_residential = df_clean[df_clean["BUILDING CLASS CATEGORY"].isin(residential_categories)].copy()
print(f"Residential sales: {len(df_residential)}")
print(f"\nBreakdown by category:")
print(df_residential["BUILDING CLASS CATEGORY"].value_counts())

# Cell 8: Borough Mapping
borough_map = {
    "1": "Manhattan",
    "2": "Bronx",
    "3": "Brooklyn",
    "4": "Queens",
    "5": "Staten Island"
}
df_residential["BOROUGH_NAME"] = df_residential["BOROUGH"].astype(str).map(borough_map)

Your dataset is now clean, focused, and ready for real analysis. Next up, you'll use this prepared data to calculate borough-level pricing patterns and create your first professional visualizations.

Analyze Borough-Level Pricing Patterns

Your cleaned dataset of residential property sales is ready for analysis. You have hundreds of thousands of valid market transactions filtered down to the property types that matter for a residential investment strategy.

The first question any investor asks is: which boroughs command the highest prices, and how much variation exists within each? This macro-level view helps the firm understand where premium markets sit versus where value opportunities might hide.

In this step, get ready to:
  • Calculate summary statistics grouped by borough.
  • Create a bar chart comparing median sale prices across all five boroughs.
  • Build a box plot showing price distributions to reveal outliers and spread.
Calculate summary statistics by borough

Before building any charts, you need to quantify the pricing landscape. Grouping sales by borough and computing median, mean, and count gives you a numerical foundation for the visualizations that follow.

  • Create a new cell in your notebook (Cell 9) and add the following code:
# Cell 9: Borough-Level Summary Statistics
borough_stats = df_residential.groupby("BOROUGH_NAME")["SALE PRICE"].agg(
    median_price="median",
    mean_price="mean",
    total_sales="count"
)
print(borough_stats.sort_values("median_price", ascending=False))

What does this code do?

The .groupby("BOROUGH_NAME") splits the DataFrame into five groups, one per borough. The .agg() method then calculates three statistics on each group's sale prices.

Sorting by median_price descending puts the most expensive borough at the top so you can immediately see the pricing hierarchy.

Why median instead of mean?

Real estate prices have extreme outliers. A single $50 million penthouse sale in Manhattan would pull the mean way up, making it misleading. The median gives you the "typical" sale price that better represents what most buyers actually pay.

  • Run the cell by pressing Shift+Enter.
  • You should see a table with five rows (one per borough) showing median price, mean price, and total sales count. Manhattan should appear at the top with the highest median.

Seeing a KeyError or empty DataFrame?

If you see KeyError: 'BOROUGH_NAME', make sure you ran Cell 8 first. That cell creates the BOROUGH_NAME column by mapping borough codes to names.

If the table shows zero rows, check that Cell 7 (residential filtering) ran successfully and that df_residential is not empty.

Help me debug my borough statistics cell.

Create a bar chart of median prices by borough

Numbers in a table are precise, but a bar chart lets you instantly see relative differences between boroughs. This is the kind of visualization that belongs in an investment memo because it communicates the pricing hierarchy at a glance.

  • Create a new cell (Cell 10) and add the following code:
# Cell 10: Bar Chart - Median Sale Price by Borough
fig, ax = plt.subplots(figsize=(10, 6))
borough_order = borough_stats.sort_values("median_price", ascending=False).index
sns.barplot(
    data=df_residential,
    x="BOROUGH_NAME",
    y="SALE PRICE",
    estimator="median",
    order=borough_order,
    palette="viridis",
    ax=ax
)
ax.set_title("Median Residential Sale Price by Borough", fontsize=14, fontweight="bold")
ax.set_xlabel("Borough", fontsize=12)
ax.set_ylabel("Median Sale Price ($)", fontsize=12)
ax.yaxis.set_major_formatter(plt.FuncFormatter(lambda x, p: f"${x:,.0f}"))
plt.tight_layout()
plt.savefig("borough_median_prices.png", dpi=150, bbox_inches="tight")
plt.show()

What does this code do?

  • borough_order sorts the boroughs from highest to lowest median price so the chart reads left-to-right from most to least expensive.
  • sns.barplot() with estimator="median" tells seaborn to calculate the median of each group rather than the default mean.
  • plt.FuncFormatter() formats the y-axis labels as dollar amounts with commas (e.g., $750,000) instead of raw numbers.
  • plt.savefig() saves the chart as a PNG file in your project folder. This is important for your GitHub repository since viewers can see the image without running the notebook.
  • Run the cell by pressing Shift+Enter.
  • You should see a bar chart with five colored bars, ordered from tallest (Manhattan) to shortest. The y-axis displays dollar-formatted labels.

Chart not displaying or looks wrong?

If no chart appears, make sure you have %matplotlib inline enabled in your notebook or that plt.show() is the last line. Jupyter Notebook displays plots inline by default, but some configurations need the magic command.

If you see a NameError: name 'borough_stats' is not defined, run Cell 9 first. The bar chart uses borough_stats to determine the ordering.

Help me fix my borough bar chart.

Build a box plot to reveal price distributions

A bar chart shows you the "typical" price, but it hides how spread out the data is. A box plot reveals the full distribution: where most sales cluster, how wide the range is, and where outliers sit. This tells the investment firm whether a borough has predictable pricing or wild variation.

  • Create a new cell (Cell 11) and add the following code:
# Cell 11: Box Plot - Price Distribution by Borough
fig, ax = plt.subplots(figsize=(12, 6))
sns.boxplot(
    data=df_residential[df_residential["SALE PRICE"] <= 5_000_000],
    x="BOROUGH_NAME",
    y="SALE PRICE",
    order=borough_order,
    palette="viridis",
    ax=ax
)
ax.set_title("Residential Sale Price Distribution by Borough (Up to $5M)", fontsize=14, fontweight="bold")
ax.set_xlabel("Borough", fontsize=12)
ax.set_ylabel("Sale Price ($)", fontsize=12)
ax.yaxis.set_major_formatter(plt.FuncFormatter(lambda x, p: f"${x:,.0f}"))
plt.tight_layout()
plt.savefig("borough_price_distribution.png", dpi=150, bbox_inches="tight")
plt.show()

What does this code do?

  • The filter df_residential["SALE PRICE"] <= 5_000_000 caps the data at $5 million. Without this, a few ultra-luxury sales (some over $100 million) would compress the box plot so much that you could not read the differences between boroughs.
  • sns.boxplot() draws a box for each borough where the box edges mark the 25th and 75th percentiles, the line inside marks the median, and the whiskers extend to 1.5x the interquartile range.
  • Using the same borough_order and palette keeps your visualizations visually consistent across the project.

What is a box plot?

A box plot (also called a box-and-whisker plot) summarizes a distribution in five numbers: minimum, 25th percentile, median, 75th percentile, and maximum. Points beyond the whiskers are individual outliers.

A tall box means prices vary widely within that borough. A short box means most sales cluster in a tight range. For an investor, a tall box signals more risk but also more potential for finding underpriced properties.

  • Run the cell by pressing Shift+Enter.
  • You should see a box plot with five box-and-whisker groups. Manhattan's box should be noticeably taller (wider price spread) than the other boroughs, with many outlier dots above the whiskers.

Box plot looks squished or empty?

If all boxes appear compressed at the bottom, check that the $5 million filter is applied correctly. Without the filter, extreme outliers stretch the y-axis and make the boxes invisible.

If you see a NameError: name 'borough_order' is not defined, run Cell 10 first. That cell defines borough_order which this box plot reuses.

Help me fix my box plot visualization.

✔️ Awesome, I've got everything!

Great. Double check you've saved your notebook file.

ⓧ I'd like to double check the full code

Here is the complete notebook code through Cell 11:

# Cell 1: Setup and Imports
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# Set visualization style
sns.set_style("whitegrid")
plt.rcParams["figure.figsize"] = (12, 6)

# Cell 2: Load the Data
df = pd.read_csv("nyc_sales.csv")
print(f"Dataset shape: {df.shape}")
print(f"\nColumn names:\n{df.columns.tolist()}")
df.head()

# Cell 3: Initial Exploration
print(f"Data types:\n{df.dtypes}")
print(f"\nNull values:\n{df.isnull().sum()}")
print(f"\nSale Price statistics:\n{df['SALE PRICE'].describe()}")

# Cell 4: Identify Data Quality Issues
print(f"Records with $0 sale price: {(df['SALE PRICE'] == 0).sum()}")
print(f"Records with sale price <= $10: {(df['SALE PRICE'] <= 10).sum()}")
print(f"\nLAND SQUARE FEET dtype: {df['LAND SQUARE FEET'].dtype}")
print(f"GROSS SQUARE FEET dtype: {df['GROSS SQUARE FEET'].dtype}")
print(f"\nSample LAND SQUARE FEET values: {df['LAND SQUARE FEET'].head(10).tolist()}")

# Cell 5: Data Cleaning - Remove Non-Sales
df_clean = df[df["SALE PRICE"] > 10000].copy()
print(f"Records removed: {len(df) - len(df_clean)}")
print(f"Records remaining: {len(df_clean)}")

# Cell 6: Data Cleaning - Convert Numeric Columns
df_clean["LAND SQUARE FEET"] = pd.to_numeric(df_clean["LAND SQUARE FEET"].str.replace(",", ""), errors="coerce")
df_clean["GROSS SQUARE FEET"] = pd.to_numeric(df_clean["GROSS SQUARE FEET"].str.replace(",", ""), errors="coerce")
df_clean["SALE DATE"] = pd.to_datetime(df_clean["SALE DATE"])

print(f"Null LAND SQUARE FEET after conversion: {df_clean['LAND SQUARE FEET'].isnull().sum()}")
print(f"Null GROSS SQUARE FEET after conversion: {df_clean['GROSS SQUARE FEET'].isnull().sum()}")

# Cell 7: Filter to Residential Properties
residential_categories = [
    "01 ONE FAMILY DWELLINGS",
    "02 TWO FAMILY DWELLINGS",
    "03 THREE FAMILY DWELLINGS",
    "07 RENTALS - WALKUP APARTMENTS",
    "09 COOPS - WALKUP APARTMENTS",
    "10 COOPS - ELEVATOR APARTMENTS",
    "12 CONDOS - WALKUP APARTMENTS",
    "13 CONDOS - ELEVATOR APARTMENTS",
    "15 CONDOS - 2-10 UNIT RESIDENTIAL",
]

df_residential = df_clean[df_clean["BUILDING CLASS CATEGORY"].isin(residential_categories)].copy()
print(f"Residential sales: {len(df_residential)}")
print(f"\nBreakdown by category:")
print(df_residential["BUILDING CLASS CATEGORY"].value_counts())

# Cell 8: Borough Mapping
borough_map = {
    "1": "Manhattan",
    "2": "Bronx",
    "3": "Brooklyn",
    "4": "Queens",
    "5": "Staten Island"
}
df_residential["BOROUGH_NAME"] = df_residential["BOROUGH"].astype(str).map(borough_map)

# Cell 9: Borough-Level Summary Statistics
borough_stats = df_residential.groupby("BOROUGH_NAME")["SALE PRICE"].agg(
    median_price="median",
    mean_price="mean",
    total_sales="count"
)
print(borough_stats.sort_values("median_price", ascending=False))

# Cell 10: Bar Chart - Median Sale Price by Borough
fig, ax = plt.subplots(figsize=(10, 6))
borough_order = borough_stats.sort_values("median_price", ascending=False).index
sns.barplot(
    data=df_residential,
    x="BOROUGH_NAME",
    y="SALE PRICE",
    estimator="median",
    order=borough_order,
    palette="viridis",
    ax=ax
)
ax.set_title("Median Residential Sale Price by Borough", fontsize=14, fontweight="bold")
ax.set_xlabel("Borough", fontsize=12)
ax.set_ylabel("Median Sale Price ($)", fontsize=12)
ax.yaxis.set_major_formatter(plt.FuncFormatter(lambda x, p: f"${x:,.0f}"))
plt.tight_layout()
plt.savefig("borough_median_prices.png", dpi=150, bbox_inches="tight")
plt.show()

# Cell 11: Box Plot - Price Distribution by Borough
fig, ax = plt.subplots(figsize=(12, 6))
sns.boxplot(
    data=df_residential[df_residential["SALE PRICE"] <= 5_000_000],
    x="BOROUGH_NAME",
    y="SALE PRICE",
    order=borough_order,
    palette="viridis",
    ax=ax
)
ax.set_title("Residential Sale Price Distribution by Borough (Up to $5M)", fontsize=14, fontweight="bold")
ax.set_xlabel("Borough", fontsize=12)
ax.set_ylabel("Sale Price ($)", fontsize=12)
ax.yaxis.set_major_formatter(plt.FuncFormatter(lambda x, p: f"${x:,.0f}"))
plt.tight_layout()
plt.savefig("borough_price_distribution.png", dpi=150, bbox_inches="tight")
plt.show()

You now have a clear macro-level picture of NYC's residential pricing landscape. Next, you will zoom in from boroughs to individual neighborhoods and calculate price-per-square-foot to identify where the real value opportunities hide.

Identify Undervalued Neighborhoods

Your borough-level analysis revealed the big picture: Manhattan is the most expensive, and outer boroughs offer lower entry points.

But an investment firm needs more precision than "buy in Brooklyn." Within each borough, some neighborhoods cost three times more per square foot than others. In this step, you will calculate price-per-square-foot for every neighborhood and rank them to surface the most undervalued areas.

In this step, get ready to:
  • Calculate median price per square foot by neighborhood for the most recent year.
  • Rank neighborhoods to find relative bargains within each borough.
  • Create horizontal bar charts comparing the 10 most and least expensive neighborhoods.
Calculate price per square foot by neighborhood

Comparing raw sale prices across neighborhoods is misleading because a $1M studio apartment and a $1M townhouse are very different investments. Price per square foot normalizes for property size and gives you a true apples-to-apples comparison.

  • Create a new cell in your notebook (Cell 12) and add the following code to calculate price per square foot and rank neighborhoods:
# Cell 12: Calculate Price Per Square Foot by Neighborhood
df_sqft = df_residential[
    (df_residential["GROSS SQUARE FEET"] > 0) &
    (df_residential["GROSS SQUARE FEET"].notna())
].copy()
df_sqft["PRICE_PER_SQFT"] = df_sqft["SALE PRICE"] / df_sqft["GROSS SQUARE FEET"]

# Filter to most recent year
most_recent_year = df_sqft["SALE DATE"].dt.year.max()
df_recent = df_sqft[df_sqft["SALE DATE"].dt.year == most_recent_year]

neighborhood_stats = df_recent.groupby(["BOROUGH_NAME", "NEIGHBORHOOD"]).agg(
    median_price_per_sqft=("PRICE_PER_SQFT", "median"),
    num_sales=("PRICE_PER_SQFT", "count")
).reset_index()

# Filter to neighborhoods with at least 20 sales for statistical reliability
neighborhood_stats = neighborhood_stats[neighborhood_stats["num_sales"] >= 20]
print(f"Analyzing {len(neighborhood_stats)} neighborhoods with 20+ sales in {most_recent_year}")

What does this code do?

First, it filters out rows with missing or zero square footage. You cannot calculate price-per-sqft without valid square footage data.

Then it narrows the data to the most recent year only. This ensures your neighborhood rankings reflect current market conditions, not historical averages.

Finally, the 20-sale minimum removes neighborhoods with too few transactions to draw reliable conclusions. A neighborhood with only 2 sales could have a misleading median.

  • Run the cell by pressing Shift+Enter.
  • You should see output telling you how many neighborhoods qualified for analysis (e.g., "Analyzing 85 neighborhoods with 20+ sales in 2025").

Why use median instead of mean?

Real estate data is full of outliers. A single $20M penthouse sale in a neighborhood would pull the mean way above what a typical property costs there. The median gives you the middle-of-the-road price that better represents what an investor would actually pay.

Seeing 0 neighborhoods in the output?

Check that you ran Cell 6 (which converts GROSS SQUARE FEET to numeric) and Cell 8 (which adds the BOROUGH_NAME column). Both are required for this cell to produce results.

help me debug why my neighborhood stats DataFrame is empty

Create neighborhood comparison charts

Numbers in a table are hard to scan quickly. A horizontal bar chart lets you instantly see which neighborhoods stand out at both ends of the price spectrum.

  • Create a new cell (Cell 13) and add this code to visualize the top 10 most and least expensive neighborhoods:
# Cell 13: Top 10 Most and Least Expensive Neighborhoods
top_10_expensive = neighborhood_stats.nlargest(10, "median_price_per_sqft")
top_10_affordable = neighborhood_stats.nsmallest(10, "median_price_per_sqft")

fig, axes = plt.subplots(1, 2, figsize=(16, 8))

# Most expensive
sns.barplot(
    data=top_10_expensive,
    y="NEIGHBORHOOD",
    x="median_price_per_sqft",
    palette="Reds_r",
    ax=axes[0]
)
axes[0].set_title(f"Top 10 Most Expensive Neighborhoods ({most_recent_year})", fontsize=12, fontweight="bold")
axes[0].set_xlabel("Median Price per Sq Ft ($)", fontsize=10)
axes[0].set_ylabel("")

# Most affordable
sns.barplot(
    data=top_10_affordable,
    y="NEIGHBORHOOD",
    x="median_price_per_sqft",
    palette="Greens_r",
    ax=axes[1]
)
axes[1].set_title(f"Top 10 Most Affordable Neighborhoods ({most_recent_year})", fontsize=12, fontweight="bold")
axes[1].set_xlabel("Median Price per Sq Ft ($)", fontsize=10)
axes[1].set_ylabel("")

plt.tight_layout()
plt.savefig("neighborhood_comparison.png", dpi=150, bbox_inches="tight")
plt.show()

What does this code do?

  • nlargest(10, "median_price_per_sqft") pulls the 10 neighborhoods with the highest price per square foot. nsmallest does the opposite.
  • plt.subplots(1, 2) creates two side-by-side charts so you can compare expensive vs. affordable at a glance.
  • Horizontal bars (y="NEIGHBORHOOD") make long neighborhood names easy to read compared to angled text on a vertical axis.
  • Run the cell by pressing Shift+Enter.
  • You should see two horizontal bar charts side by side. The left chart (red tones) shows the priciest neighborhoods, and the right chart (green tones) shows the most affordable.
  • Confirm that neighborhood_comparison.png was saved in your project folder.

Chart looks empty or only shows a few bars?

If fewer than 10 bars appear, your neighborhood_stats DataFrame may have fewer than 10 qualifying neighborhoods. Check that Cell 12 ran successfully and printed a reasonable count (you should see dozens of neighborhoods).

help me figure out why my neighborhood comparison chart is not rendering correctly

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook before moving on.

ⓧ I'd like to double check the full code

Here is the complete notebook up to this point. Compare your cells against this:

# Cell 1: Setup and Imports
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# Set visualization style
sns.set_style("whitegrid")
plt.rcParams["figure.figsize"] = (12, 6)

# Cell 2: Load the Data
df = pd.read_csv("nyc_sales.csv")
print(f"Dataset shape: {df.shape}")
print(f"\nColumn names:\n{df.columns.tolist()}")
df.head()

# Cell 3: Initial Exploration
print(f"Data types:\n{df.dtypes}")
print(f"\nNull values:\n{df.isnull().sum()}")
print(f"\nSale Price statistics:\n{df['SALE PRICE'].describe()}")

# Cell 4: Identify Data Quality Issues
print(f"Records with $0 sale price: {(df['SALE PRICE'] == 0).sum()}")
print(f"Records with sale price <= $10: {(df['SALE PRICE'] <= 10).sum()}")
print(f"\nLAND SQUARE FEET dtype: {df['LAND SQUARE FEET'].dtype}")
print(f"GROSS SQUARE FEET dtype: {df['GROSS SQUARE FEET'].dtype}")
print(f"\nSample LAND SQUARE FEET values: {df['LAND SQUARE FEET'].head(10).tolist()}")

# Cell 5: Data Cleaning - Remove Non-Sales
df_clean = df[df["SALE PRICE"] > 10000].copy()
print(f"Records removed: {len(df) - len(df_clean)}")
print(f"Records remaining: {len(df_clean)}")

# Cell 6: Data Cleaning - Convert Numeric Columns
df_clean["LAND SQUARE FEET"] = pd.to_numeric(df_clean["LAND SQUARE FEET"].str.replace(",", ""), errors="coerce")
df_clean["GROSS SQUARE FEET"] = pd.to_numeric(df_clean["GROSS SQUARE FEET"].str.replace(",", ""), errors="coerce")
df_clean["SALE DATE"] = pd.to_datetime(df_clean["SALE DATE"])

print(f"Null LAND SQUARE FEET after conversion: {df_clean['LAND SQUARE FEET'].isnull().sum()}")
print(f"Null GROSS SQUARE FEET after conversion: {df_clean['GROSS SQUARE FEET'].isnull().sum()}")

# Cell 7: Filter to Residential Properties
residential_categories = [
    "01 ONE FAMILY DWELLINGS",
    "02 TWO FAMILY DWELLINGS",
    "03 THREE FAMILY DWELLINGS",
    "07 RENTALS - WALKUP APARTMENTS",
    "09 COOPS - WALKUP APARTMENTS",
    "10 COOPS - ELEVATOR APARTMENTS",
    "12 CONDOS - WALKUP APARTMENTS",
    "13 CONDOS - ELEVATOR APARTMENTS",
    "15 CONDOS - 2-10 UNIT RESIDENTIAL",
]

df_residential = df_clean[df_clean["BUILDING CLASS CATEGORY"].isin(residential_categories)].copy()
print(f"Residential sales: {len(df_residential)}")
print(f"\nBreakdown by category:")
print(df_residential["BUILDING CLASS CATEGORY"].value_counts())

# Cell 8: Borough Mapping
borough_map = {
    "1": "Manhattan",
    "2": "Bronx",
    "3": "Brooklyn",
    "4": "Queens",
    "5": "Staten Island"
}
df_residential["BOROUGH_NAME"] = df_residential["BOROUGH"].astype(str).map(borough_map)

# Cell 9: Borough-Level Summary Statistics
borough_stats = df_residential.groupby("BOROUGH_NAME")["SALE PRICE"].agg(
    median_price="median",
    mean_price="mean",
    total_sales="count"
)
print(borough_stats.sort_values("median_price", ascending=False))

# Cell 10: Bar Chart - Median Sale Price by Borough
fig, ax = plt.subplots(figsize=(10, 6))
borough_order = borough_stats.sort_values("median_price", ascending=False).index
sns.barplot(
    data=df_residential,
    x="BOROUGH_NAME",
    y="SALE PRICE",
    estimator="median",
    order=borough_order,
    palette="viridis",
    ax=ax
)
ax.set_title("Median Residential Sale Price by Borough", fontsize=14, fontweight="bold")
ax.set_xlabel("Borough", fontsize=12)
ax.set_ylabel("Median Sale Price ($)", fontsize=12)
ax.yaxis.set_major_formatter(plt.FuncFormatter(lambda x, p: f"${x:,.0f}"))
plt.tight_layout()
plt.savefig("borough_median_prices.png", dpi=150, bbox_inches="tight")
plt.show()

# Cell 11: Box Plot - Price Distribution by Borough
fig, ax = plt.subplots(figsize=(12, 6))
sns.boxplot(
    data=df_residential[df_residential["SALE PRICE"] <= 5_000_000],
    x="BOROUGH_NAME",
    y="SALE PRICE",
    order=borough_order,
    palette="viridis",
    ax=ax
)
ax.set_title("Residential Sale Price Distribution by Borough (Up to $5M)", fontsize=14, fontweight="bold")
ax.set_xlabel("Borough", fontsize=12)
ax.set_ylabel("Sale Price ($)", fontsize=12)
ax.yaxis.set_major_formatter(plt.FuncFormatter(lambda x, p: f"${x:,.0f}"))
plt.tight_layout()
plt.savefig("borough_price_distribution.png", dpi=150, bbox_inches="tight")
plt.show()

# Cell 12: Calculate Price Per Square Foot by Neighborhood
df_sqft = df_residential[
    (df_residential["GROSS SQUARE FEET"] > 0) &
    (df_residential["GROSS SQUARE FEET"].notna())
].copy()
df_sqft["PRICE_PER_SQFT"] = df_sqft["SALE PRICE"] / df_sqft["GROSS SQUARE FEET"]

# Filter to most recent year
most_recent_year = df_sqft["SALE DATE"].dt.year.max()
df_recent = df_sqft[df_sqft["SALE DATE"].dt.year == most_recent_year]

neighborhood_stats = df_recent.groupby(["BOROUGH_NAME", "NEIGHBORHOOD"]).agg(
    median_price_per_sqft=("PRICE_PER_SQFT", "median"),
    num_sales=("PRICE_PER_SQFT", "count")
).reset_index()

# Filter to neighborhoods with at least 20 sales for statistical reliability
neighborhood_stats = neighborhood_stats[neighborhood_stats["num_sales"] >= 20]
print(f"Analyzing {len(neighborhood_stats)} neighborhoods with 20+ sales in {most_recent_year}")

# Cell 13: Top 10 Most and Least Expensive Neighborhoods
top_10_expensive = neighborhood_stats.nlargest(10, "median_price_per_sqft")
top_10_affordable = neighborhood_stats.nsmallest(10, "median_price_per_sqft")

fig, axes = plt.subplots(1, 2, figsize=(16, 8))

# Most expensive
sns.barplot(
    data=top_10_expensive,
    y="NEIGHBORHOOD",
    x="median_price_per_sqft",
    palette="Reds_r",
    ax=axes[0]
)
axes[0].set_title(f"Top 10 Most Expensive Neighborhoods ({most_recent_year})", fontsize=12, fontweight="bold")
axes[0].set_xlabel("Median Price per Sq Ft ($)", fontsize=10)
axes[0].set_ylabel("")

# Most affordable
sns.barplot(
    data=top_10_affordable,
    y="NEIGHBORHOOD",
    x="median_price_per_sqft",
    palette="Greens_r",
    ax=axes[1]
)
axes[1].set_title(f"Top 10 Most Affordable Neighborhoods ({most_recent_year})", fontsize=12, fontweight="bold")
axes[1].set_xlabel("Median Price per Sq Ft ($)", fontsize=10)
axes[1].set_ylabel("")

plt.tight_layout()
plt.savefig("neighborhood_comparison.png", dpi=150, bbox_inches="tight")
plt.show()

You now have a clear picture of which neighborhoods are the most and least expensive per square foot. Next, you will turn these insights into a business recommendation that a stakeholder can act on.

Write Your Business Recommendation

You have clean data, summary statistics, and professional visualizations. But charts alone don't get investment decisions made.

Hiring managers want to see that you can translate data into a business recommendation. In this step, you'll write a stakeholder memo summarizing your findings, create a portfolio-ready README, and push everything to GitHub.

In this step, get ready to:
  • Write an investment recommendation memo as a markdown cell in your notebook.
  • Create a README.md following the Problem, Data, Decision framework.
  • Push your completed project to GitHub.
Write the investment recommendation memo

A data analyst's job doesn't end with visualizations. The final deliverable is a recommendation that stakeholders can act on without needing to read your code.

  • In your Jupyter Notebook, click the + button in the toolbar to add a new cell below Cell 13.
  • Change the cell type from Code to Markdown using the dropdown menu in the toolbar (it defaults to "Code").
  • Paste the following memo content into the markdown cell:
## Investment Recommendation Memo

**To:** Real Estate Investment Committee
**From:** Data Analyst
**Re:** NYC Residential Property Acquisition Strategy

### Key Findings

1. **Manhattan commands the highest prices** but also shows the widest spread,
   indicating both premium and relatively undervalued pockets exist.

2. **Brooklyn and Queens offer the best value-to-growth balance** for residential
   acquisitions, with median prices significantly below Manhattan but strong
   transaction volume indicating healthy demand.

3. **Affordable neighborhoods with high transaction volume** represent the best
   acquisition targets. High volume signals liquidity (ability to exit the
   investment), while lower price-per-sqft signals room for appreciation.

### Recommendation

Focus acquisitions on the top 5 affordable neighborhoods (by price-per-sqft)
that also rank in the top 25% by transaction volume. These neighborhoods offer:
- Below-market entry prices
- Proven buyer demand (liquidity for exit)
- Room for value appreciation

### Next Steps
- Deep-dive analysis on the recommended neighborhoods (permit activity, transit access)
- Year-over-year trend analysis to confirm appreciation trajectory
- Comparable analysis against recent institutional purchases in target areas

Why a memo format instead of bullet points?

The memo format (To/From/Re, Key Findings, Recommendation, Next Steps) mirrors how real analysts communicate with stakeholders. It shows hiring managers you can package insights for decision-makers, not just produce charts.

  • Run the cell by pressing Shift+Enter.

You should see the memo rendered as formatted text with bold headings, numbered findings, and bullet points.

Seeing raw markdown instead of formatted text?

Make sure you changed the cell type to Markdown before running it. If the cell type is still set to "Code", Jupyter treats the content as Python and throws a syntax error.

Double-click the cell to edit it, change the dropdown to Markdown, then press Shift+Enter again.

Help me fix my markdown cell in Jupyter Notebook

  • Save your notebook by pressing Cmd+S (macOS) or Ctrl+S (Windows).
Create the README.md

A README is the first thing a hiring manager sees when they open your GitHub repository. It needs to answer three questions in under 3 minutes: what problem did you solve, what data did you use, and what did you decide.

  • Create a new file called README.md in your nyc-property-analysis folder by running this command in your terminal:
touch README.md
  • Open README.md in a text editor and paste the following content:
# NYC Property Sales Analysis: Identifying Investment Opportunities

## Business Problem

A real estate investment firm needs to identify which NYC neighborhoods offer the best
residential acquisition opportunities based on pricing patterns, transaction volume,
and relative value. With hundreds of neighborhoods across five boroughs, the firm needs
data-driven guidance on where to focus its capital for maximum return potential.

## Dataset

- **Source:** NYC Department of Finance, Citywide Annualized Calendar Sales Update
- **Records:** 845,000+ property sales (2016-2025)
- **Access:** [NYC Open Data](https://data.cityofnewyork.us/d/w2pb-icbu)
- **Fields:** Borough, neighborhood, building class, sale price, square footage, sale date, and more

## Approach

1. **Data Cleaning:** Removed 270,000+ non-market transactions ($0 and nominal-price
   ownership transfers), converted text fields to numeric, filtered to residential properties.

2. **Borough Analysis:** Compared median prices and distributions across all five boroughs
   to understand the macro pricing landscape.

3. **Neighborhood Ranking:** Calculated price-per-square-foot for each neighborhood to
   identify relative value within each borough.

4. **Investment Recommendation:** Combined pricing data with transaction volume to identify
   neighborhoods offering both value and liquidity.

## Key Findings

- Manhattan median residential sale price is 3-4x higher than outer boroughs
- Brooklyn and Queens show the highest transaction volumes, indicating strong demand
- Top affordable neighborhoods with high volume represent the best acquisition targets
- Price-per-sqft analysis reveals hidden value pockets even within expensive boroughs

## Tools Used

- Python 3.x
- pandas 3.0.2 (data manipulation and cleaning)
- matplotlib 3.7.1 (visualization)
- seaborn 0.13.2 (statistical visualization)
- Jupyter Notebook (analysis environment)

## How to Run

```bash
pip install pandas matplotlib seaborn jupyter
jupyter notebook nyc_property_analysis.ipynb
```

## Visualizations

- Borough median price comparison (bar chart)
- Price distribution by borough (box plot)
- Top 10 most/least expensive neighborhoods (horizontal bar charts)

What is the Problem, Data, Decision framework?

This framework structures your README so reviewers understand your project quickly. Business Problem explains what question you're answering. Dataset and Approach show how you worked with the data. Key Findings delivers the decision output.

A hiring manager scanning your GitHub can evaluate your analytical thinking in under 3 minutes without opening the notebook.

  • Save README.md.
  • Confirm both files exist in your project folder by running:
ls

You should see README.md, nyc_property_analysis.ipynb, nyc_sales.csv, and your three .png image files listed in the output.

Don't see README.md?

Make sure your terminal is inside the nyc-property-analysis folder. Run pwd to check your current directory. If you're in the wrong folder, use cd to navigate to the correct location and try the touch command again.

Help me find my project folder in the terminal

Push your project to GitHub

Your analysis is complete. Now you need to make it publicly accessible so hiring managers and collaborators can view it.

  • Go to github.com in your browser and sign in to your account.
  • Click the + icon in the top-right corner and select New repository.
  • Enter nyc-property-analysis as the repository name.
  • Leave the repository set to Public.
  • Do not check any of the initialization boxes (no README, no .gitignore, no license).
  • Click Create repository.

GitHub shows you a set of commands to push an existing repository. Back in your terminal, initialize Git and push your files by running these commands:

git init
git add nyc_property_analysis.ipynb README.md borough_median_prices.png borough_price_distribution.png neighborhood_comparison.png
git commit -m "Complete NYC property sales analysis with business recommendation"
git branch -M main
git remote add origin [[YOUR_REPO_URL="https://github.com/YOUR_USERNAME/nyc-property-analysis.git"]]
git push -u origin main

Why not add nyc_sales.csv?

The raw dataset is ~150MB, which exceeds GitHub's file size limit. Your README includes a link to the NYC Open Data source so anyone can download it themselves. This is standard practice for data projects with large source files.

  • Refresh your GitHub repository page in the browser.

You should see your README.md rendered as a formatted page with your notebook file and three .png visualization files listed above it.

Getting a permission or authentication error?

If you see "Permission denied" or "Authentication failed", you may need to set up a personal access token or SSH key for GitHub. GitHub no longer accepts passwords for command-line pushes.

Check that the remote URL matches your repository exactly. Run git remote -v to verify it.

Help me authenticate with GitHub from the command line

✔️ Awesome, I've got everything!

Great. Double check you've saved your notebook and confirmed all files appear on GitHub.

ⓧ I'd like to double check the full code

Here is the complete notebook and README for comparison.

# Cell 1: Setup and Imports
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

# Set visualization style
sns.set_style("whitegrid")
plt.rcParams["figure.figsize"] = (12, 6)

# Cell 2: Load the Data
df = pd.read_csv("nyc_sales.csv")
print(f"Dataset shape: {df.shape}")
print(f"\nColumn names:\n{df.columns.tolist()}")
df.head()

# Cell 3: Initial Exploration
print(f"Data types:\n{df.dtypes}")
print(f"\nNull values:\n{df.isnull().sum()}")
print(f"\nSale Price statistics:\n{df['SALE PRICE'].describe()}")

# Cell 4: Identify Data Quality Issues
print(f"Records with $0 sale price: {(df['SALE PRICE'] == 0).sum()}")
print(f"Records with sale price <= $10: {(df['SALE PRICE'] <= 10).sum()}")
print(f"\nLAND SQUARE FEET dtype: {df['LAND SQUARE FEET'].dtype}")
print(f"GROSS SQUARE FEET dtype: {df['GROSS SQUARE FEET'].dtype}")
print(f"\nSample LAND SQUARE FEET values: {df['LAND SQUARE FEET'].head(10).tolist()}")

# Cell 5: Data Cleaning - Remove Non-Sales
# $0 and very low prices indicate ownership transfers, not market sales
df_clean = df[df["SALE PRICE"] > 10000].copy()
print(f"Records removed: {len(df) - len(df_clean)}")
print(f"Records remaining: {len(df_clean)}")

# Cell 6: Data Cleaning - Convert Numeric Columns
df_clean["LAND SQUARE FEET"] = pd.to_numeric(df_clean["LAND SQUARE FEET"].str.replace(",", ""), errors="coerce")
df_clean["GROSS SQUARE FEET"] = pd.to_numeric(df_clean["GROSS SQUARE FEET"].str.replace(",", ""), errors="coerce")
df_clean["SALE DATE"] = pd.to_datetime(df_clean["SALE DATE"])

print(f"Null LAND SQUARE FEET after conversion: {df_clean['LAND SQUARE FEET'].isnull().sum()}")
print(f"Null GROSS SQUARE FEET after conversion: {df_clean['GROSS SQUARE FEET'].isnull().sum()}")

# Cell 7: Filter to Residential Properties
residential_categories = [
    "01 ONE FAMILY DWELLINGS",
    "02 TWO FAMILY DWELLINGS",
    "03 THREE FAMILY DWELLINGS",
    "07 RENTALS - WALKUP APARTMENTS",
    "09 COOPS - WALKUP APARTMENTS",
    "10 COOPS - ELEVATOR APARTMENTS",
    "12 CONDOS - WALKUP APARTMENTS",
    "13 CONDOS - ELEVATOR APARTMENTS",
    "15 CONDOS - 2-10 UNIT RESIDENTIAL",
]

df_residential = df_clean[df_clean["BUILDING CLASS CATEGORY"].isin(residential_categories)].copy()
print(f"Residential sales: {len(df_residential)}")
print(f"\nBreakdown by category:")
print(df_residential["BUILDING CLASS CATEGORY"].value_counts())

# Cell 8: Borough Mapping
borough_map = {
    "1": "Manhattan",
    "2": "Bronx",
    "3": "Brooklyn",
    "4": "Queens",
    "5": "Staten Island"
}
df_residential["BOROUGH_NAME"] = df_residential["BOROUGH"].astype(str).map(borough_map)

# Cell 9: Borough-Level Summary Statistics
borough_stats = df_residential.groupby("BOROUGH_NAME")["SALE PRICE"].agg(
    median_price="median",
    mean_price="mean",
    total_sales="count"
)
print(borough_stats.sort_values("median_price", ascending=False))

# Cell 10: Bar Chart - Median Sale Price by Borough
fig, ax = plt.subplots(figsize=(10, 6))
borough_order = borough_stats.sort_values("median_price", ascending=False).index
sns.barplot(
    data=df_residential,
    x="BOROUGH_NAME",
    y="SALE PRICE",
    estimator="median",
    order=borough_order,
    palette="viridis",
    ax=ax
)
ax.set_title("Median Residential Sale Price by Borough", fontsize=14, fontweight="bold")
ax.set_xlabel("Borough", fontsize=12)
ax.set_ylabel("Median Sale Price ($)", fontsize=12)
ax.yaxis.set_major_formatter(plt.FuncFormatter(lambda x, p: f"${x:,.0f}"))
plt.tight_layout()
plt.savefig("borough_median_prices.png", dpi=150, bbox_inches="tight")
plt.show()

# Cell 11: Box Plot - Price Distribution by Borough
fig, ax = plt.subplots(figsize=(12, 6))
sns.boxplot(
    data=df_residential[df_residential["SALE PRICE"] <= 5_000_000],
    x="BOROUGH_NAME",
    y="SALE PRICE",
    order=borough_order,
    palette="viridis",
    ax=ax
)
ax.set_title("Residential Sale Price Distribution by Borough (Up to $5M)", fontsize=14, fontweight="bold")
ax.set_xlabel("Borough", fontsize=12)
ax.set_ylabel("Sale Price ($)", fontsize=12)
ax.yaxis.set_major_formatter(plt.FuncFormatter(lambda x, p: f"${x:,.0f}"))
plt.tight_layout()
plt.savefig("borough_price_distribution.png", dpi=150, bbox_inches="tight")
plt.show()

# Cell 12: Calculate Price Per Square Foot by Neighborhood
df_sqft = df_residential[
    (df_residential["GROSS SQUARE FEET"] > 0) &
    (df_residential["GROSS SQUARE FEET"].notna())
].copy()
df_sqft["PRICE_PER_SQFT"] = df_sqft["SALE PRICE"] / df_sqft["GROSS SQUARE FEET"]

# Filter to most recent year
most_recent_year = df_sqft["SALE DATE"].dt.year.max()
df_recent = df_sqft[df_sqft["SALE DATE"].dt.year == most_recent_year]

neighborhood_stats = df_recent.groupby(["BOROUGH_NAME", "NEIGHBORHOOD"]).agg(
    median_price_per_sqft=("PRICE_PER_SQFT", "median"),
    num_sales=("PRICE_PER_SQFT", "count")
).reset_index()

# Filter to neighborhoods with at least 20 sales for statistical reliability
neighborhood_stats = neighborhood_stats[neighborhood_stats["num_sales"] >= 20]
print(f"Analyzing {len(neighborhood_stats)} neighborhoods with 20+ sales in {most_recent_year}")

# Cell 13: Top 10 Most and Least Expensive Neighborhoods
top_10_expensive = neighborhood_stats.nlargest(10, "median_price_per_sqft")
top_10_affordable = neighborhood_stats.nsmallest(10, "median_price_per_sqft")

fig, axes = plt.subplots(1, 2, figsize=(16, 8))

# Most expensive
sns.barplot(
    data=top_10_expensive,
    y="NEIGHBORHOOD",
    x="median_price_per_sqft",
    palette="Reds_r",
    ax=axes[0]
)
axes[0].set_title(f"Top 10 Most Expensive Neighborhoods ({most_recent_year})", fontsize=12, fontweight="bold")
axes[0].set_xlabel("Median Price per Sq Ft ($)", fontsize=10)
axes[0].set_ylabel("")

# Most affordable
sns.barplot(
    data=top_10_affordable,
    y="NEIGHBORHOOD",
    x="median_price_per_sqft",
    palette="Greens_r",
    ax=axes[1]
)
axes[1].set_title(f"Top 10 Most Affordable Neighborhoods ({most_recent_year})", fontsize=12, fontweight="bold")
axes[1].set_xlabel("Median Price per Sq Ft ($)", fontsize=10)
axes[1].set_ylabel("")

plt.tight_layout()
plt.savefig("neighborhood_comparison.png", dpi=150, bbox_inches="tight")
plt.show()

# Cell 14: Business Recommendation (Markdown Cell)
# This is a MARKDOWN cell, not a code cell. Change the cell type to Markdown.
"""
## Investment Recommendation Memo

**To:** Real Estate Investment Committee
**From:** Data Analyst
**Re:** NYC Residential Property Acquisition Strategy

### Key Findings

1. **Manhattan commands the highest prices** but also shows the widest spread,
   indicating both premium and relatively undervalued pockets exist.

2. **Brooklyn and Queens offer the best value-to-growth balance** for residential
   acquisitions, with median prices significantly below Manhattan but strong
   transaction volume indicating healthy demand.

3. **Affordable neighborhoods with high transaction volume** represent the best
   acquisition targets. High volume signals liquidity (ability to exit the
   investment), while lower price-per-sqft signals room for appreciation.

### Recommendation

Focus acquisitions on the top 5 affordable neighborhoods (by price-per-sqft)
that also rank in the top 25% by transaction volume. These neighborhoods offer:
- Below-market entry prices
- Proven buyer demand (liquidity for exit)
- Room for value appreciation

### Next Steps
- Deep-dive analysis on the recommended neighborhoods (permit activity, transit access)
- Year-over-year trend analysis to confirm appreciation trajectory
- Comparable analysis against recent institutional purchases in target areas
"""
# NYC Property Sales Analysis: Identifying Investment Opportunities

## Business Problem

A real estate investment firm needs to identify which NYC neighborhoods offer the best
residential acquisition opportunities based on pricing patterns, transaction volume,
and relative value. With hundreds of neighborhoods across five boroughs, the firm needs
data-driven guidance on where to focus its capital for maximum return potential.

## Dataset

- **Source:** NYC Department of Finance, Citywide Annualized Calendar Sales Update
- **Records:** 845,000+ property sales (2016-2025)
- **Access:** [NYC Open Data](https://data.cityofnewyork.us/d/w2pb-icbu)
- **Fields:** Borough, neighborhood, building class, sale price, square footage, sale date, and more

## Approach

1. **Data Cleaning:** Removed 270,000+ non-market transactions ($0 and nominal-price
   ownership transfers), converted text fields to numeric, filtered to residential properties.

2. **Borough Analysis:** Compared median prices and distributions across all five boroughs
   to understand the macro pricing landscape.

3. **Neighborhood Ranking:** Calculated price-per-square-foot for each neighborhood to
   identify relative value within each borough.

4. **Investment Recommendation:** Combined pricing data with transaction volume to identify
   neighborhoods offering both value and liquidity.

## Key Findings

- Manhattan median residential sale price is 3-4x higher than outer boroughs
- Brooklyn and Queens show the highest transaction volumes, indicating strong demand
- Top affordable neighborhoods with high volume represent the best acquisition targets
- Price-per-sqft analysis reveals hidden value pockets even within expensive boroughs

## Tools Used

- Python 3.x
- pandas 3.0.2 (data manipulation and cleaning)
- matplotlib 3.7.1 (visualization)
- seaborn 0.13.2 (statistical visualization)
- Jupyter Notebook (analysis environment)

## How to Run

```bash
pip install pandas matplotlib seaborn jupyter
jupyter notebook nyc_property_analysis.ipynb
```

## Visualizations

- Borough median price comparison (bar chart)
- Price distribution by borough (box plot)
- Top 10 most/least expensive neighborhoods (horizontal bar charts)

Your analysis is live on GitHub. You now have a portfolio-ready data project that demonstrates the full analyst workflow: messy data in, business recommendation out. Next up, you can extend this project with a year-over-year trend analysis in the Secret Mission.

Secret mission

Add Year-Over-Year Price Trend Analysis

Your borough-level analysis tells investors where to look. But smart investors want to know where prices are heading, not just where they are today. Can you identify the neighborhoods with the fastest-rising prices over the last decade?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally, so there are no ongoing costs.

Resources you used:

  • Project folder containing nyc_property_analysis.ipynb.
  • Downloaded dataset nyc_sales.csv (~150MB).
  • Generated visualization images (PNG files).
  • README.md file.
  • GitHub repository.
  • Installed Python packages (pandas, matplotlib, seaborn).

Keep everything running

No action needed. Choose this if you plan to extend your analysis or revisit the dataset.

  • Your project folder, notebook, and dataset stay on your Desktop.
  • Your GitHub repository remains public for hiring managers to view.
  • Your installed packages remain available for future Python projects.

Pause - I'll come back to this later

Free up disk space by removing the large dataset file while keeping your analysis work intact.

  • Delete nyc_sales.csv from your project folder to reclaim ~150MB of disk space.
  • Keep your notebook and README. You can re-download the dataset from NYC Open Data when you return.
  • Your GitHub repository stays live. No changes needed there.

Delete - I don't want to use this again

Remove all project resources and start fresh.

  • Delete your project folder from the Desktop (this removes the notebook, dataset, and visualization files).
  • Delete the GitHub repository by going to your repository's Settings page, scrolling to the bottom, and clicking Delete this repository.
  • Optionally uninstall the Python packages by running pip uninstall pandas matplotlib seaborn in your terminal.

Nice Work!

Nice Work!

You just built a complete data analysis project using real NYC government data with 845,000+ property sales records. From messy raw data to business recommendations, you've demonstrated the full data analyst workflow.

Here's what you accomplished:

  • Cleaned and filtered 500,000+ valid property sales from a messy government dataset by removing non-market transactions and converting text columns to numeric types.
  • Created professional data visualizations comparing borough-level pricing, price distributions, and neighborhood value using pandas, matplotlib, and seaborn.
  • Published a portfolio-ready GitHub repository with a business recommendation memo and a README following the Problem, Data, Decision framework.
  • Secret Mission: Built a year-over-year price trend analysis identifying the top 5 fastest-appreciating neighborhoods in NYC.

Ready to quiz yourself?