Nepal Stock Market Analyzer with ML

Build a Random Forest model to predict NEPSE stock gainers using live API data.

Introduction

30 Second Summary

Stock markets produce a flood of numbers every single day. The patterns hiding inside all that data are invisible to the naked eye.

In this project, you will pull live stock data from Nepal's NEPSE market into a Google Colab notebook. You will explore the data visually and train a machine learning model to classify stocks as daily gainers or losers.

What You'll Build

You will open your notebook, run all cells, and watch it pull live data for hundreds of Nepali stocks, chart sector trends, and predict which ones are today's gainers or losers.

By the end of this project, you'll have:

  • A live data pipeline that pulls fresh trading data for ~300 Nepal stocks from a free API every time you hit run.
  • Visual charts showing which sectors dominate the Nepal stock market and how metrics like volume and turnover relate to price movement.
  • A trained Random Forest model that predicts daily stock gainers vs losers, with a feature importance chart ranking which trading metrics matter most.
  • Secret Mission: Pull IPO announcement data and cross-reference it with current stock prices to identify which recently listed companies outperform the market.

Do I need any coding experience?

No prior experience is required. You will need a free Google account to use Google Colab, which runs Python in your browser with all libraries pre-installed.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters. This step is your moment to commit to the project goal before any hands-on work begins.

Set Up Google Colab and Load Live NEPSE Data

Every data science project starts with data. Before you can visualize trends or train a model, you need a real dataset to work with.

In this step, you'll set up a free Google Colab notebook and pull live stock market data from Nepal's NEPSE exchange using a public API. By the end, you'll have hundreds of stock records loaded and ready for analysis.

In this step, get ready to:
  • Create a new Google Colab notebook.
  • Fetch live NEPSE stock data from a public API.
  • Inspect the dataset's structure, types, and missing values.
Open Google Colab and create a new notebook

Google Colab is a free, browser-based Python environment from Google. It comes with popular data science libraries pre-installed, so you can start coding immediately without any local setup.

  • Open your browser and navigate to colab.research.google.com.
  • Sign in with your Google account if prompted.
  • Click File in the top menu bar, then select New notebook.
  • Click the notebook title at the top left (it defaults to something like Untitled0.ipynb).
  • Rename it to NEPSE_Market_Analyzer.

You now have a blank Colab notebook with an empty code cell ready for input.

What is Google Colab?

Google Colab is a hosted Jupyter notebook service. It gives you a Python environment in your browser with free access to computing resources. Libraries like pandas, scikit-learn, matplotlib, and seaborn are all pre-installed.

Your notebooks are saved automatically to your Google Drive, so you can close your browser and come back anytime.

Import libraries and fetch live stock data

Each cell in a Colab notebook is a self-contained block of code you can run independently. You'll build your first cell now to import libraries and pull live data from the NEPSE market.

  • Import all the libraries your project needs by pasting the following code into the first cell:
# Cell 1: Import Libraries and Fetch Live NEPSE Data
import requests
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report

What do these libraries do?

  • requests fetches data from web APIs.
  • pandas loads and manipulates tabular data using DataFrames.
  • numpy handles numerical operations like division-by-zero protection.
  • matplotlib and seaborn create charts and visualizations.
  • sklearn provides the Random Forest classifier and evaluation tools you'll use later to predict stock movement.
  • Run the cell by pressing Shift+Enter (or clicking the play button to the left of the cell).

A small number like [1] appears to the left of the cell, confirming the imports loaded successfully. No text output is expected here.

Seeing an import error?

  • Make sure you copied all the import lines exactly as shown, including the from sklearn... lines.
  • Check that there are no extra spaces or missing letters in the library names.
  • All of these libraries come pre-installed in Colab. If you see a ModuleNotFoundError, double check the spelling of the library name.

Help me debug my Colab import error.

  • Below the imports in the same cell, add the code that fetches live stock data and inspects it:
# Fetch live stock data
url = "https://shubhamnpk.github.io/yonepse/data/nepse_data.json"
response = requests.get(url)
data = response.json()
df = pd.DataFrame(data)
print(f"Loaded {len(df)} stocks from NEPSE")
print(df.head())
print("\nColumn types:")
print(df.info())
print("\nMissing values:")
print(df.isnull().sum())

What does this code do?

  • requests.get(url) sends a request to the yonepse API, which serves live NEPSE stock data as JSON.
  • response.json() converts the raw JSON into a Python list of dictionaries.
  • pd.DataFrame(data) loads that list into a DataFrame, a table-like structure that pandas uses for data analysis.
  • df.head() prints the first 5 rows so you can see the data at a glance.
  • df.info() shows each column's name and data type, helping you spot columns that need type conversion.
  • df.isnull().sum() counts how many missing values each column has, so you know what needs cleaning later.

Before you run this, what do you think the output will look like? How many stocks do you expect to see?

  • Run the cell again with Shift+Enter.

You see a message like Loaded 285 stocks from NEPSE followed by the first 5 rows of the DataFrame. Below that, column types and missing value counts are printed.

What should I see?

The exact stock count depends on the current trading day. You'll typically see around 280 to 300 stocks. Each row represents one company listed on NEPSE, with columns like symbol, ltp (last traded price), volume, turnover, and percent_change.

Notice that some columns have missing values. The volume and turnover columns often show null values for open-ended mutual funds, which do not trade like regular stocks. You'll clean these out in the next step.

Seeing a connection error?

  • Check that the URL in your code matches exactly: https://shubhamnpk.github.io/yonepse/data/nepse_data.json.
  • Make sure your browser has internet access. Colab runs in the cloud and needs a network connection to fetch external data.
  • If you see JSONDecodeError, the API might be temporarily unavailable. Wait a moment and try running the cell again.

Help me fix my NEPSE data fetch error in Colab.

✔️ Awesome, I've got everything!

Great. Make sure your cell has both the imports and the fetch code together, then move on.

ⓧ I'd like to double check the full code

Here is the complete code for Cell 1. Make sure your cell matches this exactly:

# Cell 1: Import Libraries and Fetch Live NEPSE Data
import requests
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report

# Fetch live stock data
url = "https://shubhamnpk.github.io/yonepse/data/nepse_data.json"
response = requests.get(url)
data = response.json()
df = pd.DataFrame(data)
print(f"Loaded {len(df)} stocks from NEPSE")
print(df.head())
print("\nColumn types:")
print(df.info())
print("\nMissing values:")
print(df.isnull().sum())

Your notebook is live and pulling real NEPSE data. Next up, you'll clean this dataset and start visualizing what Nepal's stock market actually looks like.

Clean the Data and Explore NEPSE Visually

Your notebook now has live NEPSE data sitting in a DataFrame. That is a great start, but the raw data has some problems. Some rows belong to open-ended mutual funds that report no trading volume. Others have mixed data types that would silently break your charts and ML model.

In this step, you will clean out the bad rows, add sector labels to each stock, and build your first visualizations. By the end, you will see exactly which sectors dominate NEPSE and how trading metrics relate to each other.

In this step, get ready to:
  • Clean the raw data by filtering out invalid rows and converting column types.
  • Fetch sector data from the yonepse API and label each stock with its sector.
  • Create a sector distribution bar chart and a correlation heatmap of trading metrics.
Clean the raw data

When you ran df.isnull().sum() in the previous step, you saw that some columns have missing values. Those missing-value rows are open-ended mutual funds that do not trade like regular stocks. Leaving them in would skew every calculation from here on.

  • Click the + Code button in the toolbar to create a new code cell below Cell 1.
  • Paste the following cleaning code into the new cell:
# Cell 2: Clean Data and Add Sector Information
# Filter out open-ended mutual funds (they have null volume)
df_clean = df[df['volume'].notna()].copy()
# Drop rows with no trading activity
df_clean = df_clean[df_clean['trades'].notna()].copy()
# Keep only relevant columns
cols_to_keep = ['symbol', 'name', 'ltp', 'previous_close', 'change',
                'percent_change', 'high', 'low', 'volume', 'turnover', 'trades']
df_clean = df_clean[cols_to_keep].copy()

# Convert numeric columns
numeric_cols = ['ltp', 'previous_close', 'change', 'percent_change',
                'high', 'low', 'volume', 'turnover', 'trades']
for col in numeric_cols:
    df_clean[col] = pd.to_numeric(df_clean[col], errors='coerce')

What does this code do?

  • df[df['volume'].notna()] keeps only rows that have a volume value, which filters out open-ended mutual funds.
  • cols_to_keep narrows the DataFrame to only the columns useful for analysis, dropping extra metadata.
  • pd.to_numeric(df_clean[col], errors='coerce') converts each column to a numeric type. Any value that cannot be converted becomes NaN instead of crashing the notebook.

Do not run this cell yet. You will add sector labels to it first, then run everything together.

Add sector labels to each stock

NEPSE stocks belong to sectors like Commercial Banks, Hydro Power, and Insurance. The yonepse API provides a separate endpoint that maps each stock symbol to its sector. Adding this label will let you group and compare stocks by industry.

  • Add the following code below the cleaning code in the same Cell 2:
# Fetch sector data
sector_url = "https://shubhamnpk.github.io/yonepse/data/other/sector_codes.json"
sector_response = requests.get(sector_url)
sector_data = sector_response.json()

# Create symbol-to-sector mapping
symbol_to_sector = {}
for sector_name, stocks in sector_data.items():
    for stock in stocks:
        symbol_to_sector[stock['symbol']] = sector_name

df_clean['sector'] = df_clean['symbol'].map(symbol_to_sector)
df_clean['sector'] = df_clean['sector'].fillna('Unknown')

print(f"\nCleaned dataset: {len(df_clean)} stocks")
print(f"Sectors found: {df_clean['sector'].nunique()}")

What does this code do?

  • It fetches a second JSON file from the yonepse API that maps sector names to arrays of stock symbols.
  • The for loop builds a dictionary (symbol_to_sector) where each stock symbol points to its sector name.
  • .map(symbol_to_sector) uses that dictionary to add a sector column to the DataFrame. Stocks with no matching sector get labelled Unknown.
  • Run Cell 2 by clicking the play button to the left of the cell, or by pressing Shift+Enter.

You should see output like Cleaned dataset: 240 stocks (the exact number depends on the live data) and a count of how many sectors were found.

Seeing a KeyError or NameError?

  • A NameError: name 'df' is not defined means Cell 1 was not run first. Go back to Cell 1 and run it before running Cell 2.
  • A KeyError on a column name means a typo in cols_to_keep or numeric_cols. Double check that every column name matches exactly.
  • help me debug my Cell 2 cleaning code
Visualize sector distribution and trading correlations

Now that the data is clean and labelled, you can start exploratory data analysis (EDA). EDA is the stage where you visualize your data to spot patterns before building any model. Two charts will give you a strong overview: a sector distribution bar chart and a correlation heatmap.

  • Click + Code to create a new cell below Cell 2.
  • Paste the following visualization code into the new cell:
# Cell 3: Exploratory Data Analysis
fig, axes = plt.subplots(1, 2, figsize=(16, 6))

# Sector distribution
sector_counts = df_clean['sector'].value_counts()
sector_counts.plot(kind='barh', ax=axes[0], color='steelblue')
axes[0].set_title('Number of Stocks by Sector (NEPSE)')
axes[0].set_xlabel('Count')

# Correlation heatmap
corr_cols = ['ltp', 'change', 'percent_change', 'volume', 'turnover', 'trades']
corr_matrix = df_clean[corr_cols].corr()
sns.heatmap(corr_matrix, annot=True, cmap='coolwarm', center=0, ax=axes[1], fmt='.2f')
axes[1].set_title('Correlation Between Trading Metrics')

plt.tight_layout()
plt.show()

What does this code do?

  • plt.subplots(1, 2) creates a figure with two charts side by side.
  • The left chart (axes[0]) is a horizontal bar chart counting how many stocks belong to each sector.
  • The right chart (axes[1]) is a correlation heatmap. Each cell shows how strongly two trading metrics move together. Values near 1 mean they rise and fall together. Values near -1 mean they move in opposite directions.

Before you run this cell, take a guess. Which sector do you think has the most listed stocks on NEPSE?

  • Run Cell 3 by pressing Shift+Enter.

You should see two charts displayed below the cell. The left chart is a horizontal bar chart showing the number of stocks in each NEPSE sector. The right chart is a heatmap with colored squares and numbers showing correlations between trading metrics like ltp, volume, and turnover.

What should I notice in the heatmap?

Look for bright red or blue squares off the diagonal. You will likely see a strong positive correlation between volume and turnover. That makes sense because turnover is essentially volume multiplied by price.

Meanwhile, percent_change will likely have weak correlations with most other columns. That is a hint that predicting stock movement is not as simple as looking at one metric alone.

Charts not showing up?

  • Make sure you ran Cell 2 before Cell 3. Cell 3 depends on df_clean which is created in Cell 2.
  • If you see a blank output area with no error, try clicking Runtime in the menu bar, then Run all to re-execute all cells in order.
  • help me debug my NEPSE visualization cell

✔️ Awesome, I've got everything!

Great. Make sure your notebook is saved. Colab auto-saves, but you can also press Cmd+S (macOS) or Ctrl+S (Windows) to save manually.

ⓧ I'd like to double check the full code

Your notebook should have three cells at this point. Scroll up and confirm each one is present, in order:

  • Cell 1: Imports all libraries, fetches live data from nepse_data.json, loads into df, and prints shape, column types, and missing values.
  • Cell 2: Filters null volume and trades rows, keeps relevant columns, converts to numeric types, fetches sector data from sector_codes.json, and maps each stock to its sector in df_clean.
  • Cell 3: Creates a side-by-side figure with a horizontal bar chart of sector counts and a correlation heatmap of trading metrics.

Your data is clean, labelled by sector, and you have already spotted your first patterns. Next up, you will engineer new features from this data and build the target variable that your machine learning model will try to predict.

Engineer Features and Build the ML Target

In the last step, you cleaned your raw NEPSE data and explored it visually with a sector distribution chart and a correlation heatmap. You now have a solid picture of what the data looks like.

Raw trading columns like price and volume give you a starting point, but a machine learning model needs more. In this step, you will engineer new features that capture hidden patterns in the data and create a binary target variable that defines exactly what you are trying to predict: whether a stock is a daily gainer or loser.

In this step, get ready to:
  • Engineer three new trading features from existing columns.
  • Create a binary target variable to classify stocks as gainers or losers.
  • Select the final feature set and prepare a model-ready DataFrame.
Engineer new features from existing columns

Feature engineering is the process of creating new columns from existing data that help a model spot patterns it would otherwise miss. A single volume column tells you how many shares traded, but dividing volume by the number of trades reveals the average deal size per transaction.

  • Create a new code cell below your Cell 3 (Exploratory Data Analysis) by clicking the + Code button in the toolbar.
  • Add the feature engineering code by pasting the following into the new cell:
# Cell 4: Feature Engineering
df_features = df_clean.copy()

# Engineer new features
df_features['volume_per_trade'] = np.where(
    df_features['trades'] > 0,
    df_features['volume'] / df_features['trades'],
    0
)
df_features['turnover_per_volume'] = np.where(
    df_features['volume'] > 0,
    df_features['turnover'] / df_features['volume'],
    0
)
df_features['price_range'] = df_features['high'] - df_features['low']

What does this code do?

df_features = df_clean.copy() creates a fresh copy of your cleaned DataFrame so the original stays untouched.

  • volume_per_trade divides total volume by number of trades, revealing the average deal size. np.where checks if trades is greater than zero first, preventing division-by-zero errors.
  • turnover_per_volume divides turnover by volume, giving you the average price per share traded.
  • price_range subtracts the day's low from the high, capturing how volatile the stock was during the trading session.
  • Run the cell by pressing the play button (or Shift+Enter).
  • You should see a green checkmark and a runtime duration next to the cell. The cell produces no printed output yet, but the checkmark confirms the three new columns were created successfully.

Seeing a NameError?

  • If you see an error mentioning df_clean or np, make sure you ran all previous cells (Cells 1, 2, and 3) first. Colab needs each cell to run in order.
  • Check that the cell header says # Cell 4: Feature Engineering and that it is below your Cell 3 code, not accidentally inside it.

Help me debug my feature engineering cell in Colab.

Create the target variable

Every classification model needs a target variable that defines what it is trying to predict. Your target is simple: did the stock go up today, or not?

  • In the same Cell 4, add the following code below the price_range line:
# Create target variable
df_features['is_gainer'] = (df_features['percent_change'] > 0).astype(int)

print("Class distribution:")
print(df_features['is_gainer'].value_counts())
print(f"\nGainers: {df_features['is_gainer'].sum()} ({df_features['is_gainer'].mean()*100:.1f}%)")
print(f"Losers: {(df_features['is_gainer'] == 0).sum()} ({(1 - df_features['is_gainer'].mean())*100:.1f}%)")

What does this code do?

  • is_gainer is set to 1 if the stock's percent_change is greater than zero (the stock went up), and 0 otherwise (it stayed flat or went down).
  • The print statements show how many stocks fall into each class. This split matters because a heavily lopsided distribution can confuse a model later on.

Before you run this cell, take a guess: do you think more NEPSE stocks went up today or went down?

  • Run Cell 4 by pressing Shift+Enter.

You should see the class distribution printed, showing the count of gainers (1) and losers (0) along with their percentages. The split will likely lean toward one class, which is completely normal for any single trading day.

Why does the class split matter?

If 70% of stocks are losers and only 30% are gainers, a model could just predict "loser" every time and be 70% accurate without learning anything real. This is called class imbalance. You will address this in a later step.

Not seeing any output?

  • Make sure you added the new code below the price_range line inside the same Cell 4, not in a separate cell.
  • Double check that the print statements are not indented inside the np.where block. They should be at the top level of the cell, not nested.

Help me debug my target variable code in Colab.

Select the feature columns for the model

Your DataFrame now has trading columns, engineered features, and the target. Not all columns belong in the model though. Text columns like symbol and sector would confuse a numeric classifier. You need to explicitly pick the numeric features and drop any rows with missing values.

  • In Cell 4, add the following code below the class distribution print statements:
# Select features and target
feature_cols = ['ltp', 'volume', 'turnover', 'trades',
                'volume_per_trade', 'turnover_per_volume', 'price_range']
target_col = 'is_gainer'

# Drop rows with NaN in feature or target columns
df_model = df_features[feature_cols + [target_col]].dropna()
print(f"\nModel dataset: {len(df_model)} stocks with {len(feature_cols)} features")

What does this code do?

  • feature_cols lists the seven numeric columns your model will use as inputs: the four original trading metrics plus the three features you just engineered.
  • target_col is is_gainer, the binary label the model will try to predict.
  • .dropna() removes any rows where a feature or target value is missing, since the Random Forest classifier cannot handle NaN values.
  • Run Cell 4 by pressing Shift+Enter.

You should see the class distribution output from before, followed by a new line showing something like Model dataset: 240 stocks with 7 features. The exact stock count depends on today's live data.

Model dataset shows 0 stocks?

  • This means .dropna() removed all rows because a feature column had NaN everywhere. Scroll up and check that your Cell 2 converted all numeric columns with pd.to_numeric() and that df_clean filtered out rows with null volume and trades.
  • Verify the column names match exactly. A typo in feature_cols like volume_per_trades instead of volume_per_trade will cause a KeyError.

Help me debug my feature selection code.

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook by pressing Cmd+S (macOS) or Ctrl+S (Windows).

ⓧ I'd like to double check the full code

Here is the complete Cell 4. Make sure your cell matches this exactly:

# Cell 4: Feature Engineering
df_features = df_clean.copy()

df_features['volume_per_trade'] = np.where(
    df_features['trades'] > 0,
    df_features['volume'] / df_features['trades'],
    0
)
df_features['turnover_per_volume'] = np.where(
    df_features['volume'] > 0,
    df_features['turnover'] / df_features['volume'],
    0
)
df_features['price_range'] = df_features['high'] - df_features['low']

df_features['is_gainer'] = (df_features['percent_change'] > 0).astype(int)

print("Class distribution:")
print(df_features['is_gainer'].value_counts())
print(f"\nGainers: {df_features['is_gainer'].sum()} ({df_features['is_gainer'].mean()*100:.1f}%)")
print(f"Losers: {(df_features['is_gainer'] == 0).sum()} ({(1 - df_features['is_gainer'].mean())*100:.1f}%)")

feature_cols = ['ltp', 'volume', 'turnover', 'trades',
                'volume_per_trade', 'turnover_per_volume', 'price_range']
target_col = 'is_gainer'

df_model = df_features[feature_cols + [target_col]].dropna()
print(f"\nModel dataset: {len(df_model)} stocks with {len(feature_cols)} features")

Your data is now shaped and ready for a machine learning model. You have seven numeric features, a clean binary target, and no missing values. Next up, you will train a Random Forest classifier on this data and see how well it can predict which NEPSE stocks are daily gainers.

Train a Random Forest and Evaluate It

In the previous step, you engineered trading features and created a target variable that labels every stock as a gainer or a loser. Your data is model-ready.

Now it is time to train a machine learning model on this data. You will use a Random Forest classifier, which builds many decision trees and combines their votes to make a prediction. The big question is whether it can predict both gainers and losers accurately.

In this step, get ready to:
  • Split the data into training and test sets.
  • Train a Random Forest classifier and check its accuracy.
  • Evaluate the model with a confusion matrix and classification report.
Split the data and train the model

A model needs to be tested on data it has never seen before. If you test on the same data you trained with, the model could just memorize answers instead of learning real patterns.

What is a train/test split?

You divide your dataset into two parts. The training set (70% of the data) is what the model learns from. The test set (the remaining 30%) is held back so you can check how well the model performs on data it has never encountered.

  • Create a new code cell at the bottom of your notebook by clicking the + Code button.
  • Split the data, train a Random Forest, and print the accuracy by pasting this code into the new cell:
# Cell 5: Train Random Forest (First Attempt)
X = df_model[feature_cols]
y = df_model[target_col]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42
)

rf_model = RandomForestClassifier(n_estimators=100, random_state=42)
rf_model.fit(X_train, y_train)
y_pred = rf_model.predict(X_test)

accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy (first attempt): {accuracy:.2f}")

What does this code do?

  • X holds all seven feature columns, and y holds the target (is_gainer).
  • train_test_split randomly shuffles the data and reserves 30% for testing. random_state=42 makes the split reproducible so you get the same results each time.
  • RandomForestClassifier(n_estimators=100) creates a forest of 100 decision trees. Each tree votes on whether a stock is a gainer or loser, and the majority wins.

Before you run this cell, what accuracy do you think a model can achieve when predicting daily stock movement? Take a guess.

  • Run the cell by clicking the play button or pressing Shift+Enter.

You should see an accuracy score printed, somewhere in the 0.60 to 0.75 range. That looks decent on the surface. But accuracy alone hides important details.

Seeing a NameError?

  • If you see NameError: name 'df_model' is not defined, make sure you have run all previous cells (Cells 1 through 4) first. Colab does not remember variables from cells that have not been executed in this session.
  • Click Runtime in the top menu, then Run all to rerun everything from the start.
  • Why is my model throwing a NameError in Google Colab?
Evaluate with a confusion matrix

An accuracy number tells you the percentage of correct predictions overall. It does not tell you whether the model is good at predicting both classes. A confusion matrix breaks down exactly how many gainers and losers the model got right versus wrong.

  • Add the following code to the bottom of the same Cell 5, right after the print line:
print("\nClassification Report:")
print(classification_report(y_test, y_pred, target_names=['Loser', 'Gainer']))

# Confusion matrix
fig, ax = plt.subplots(figsize=(6, 5))
cm = confusion_matrix(y_test, y_pred)
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', ax=ax,
            xticklabels=['Loser', 'Gainer'],
            yticklabels=['Loser', 'Gainer'])
ax.set_title('Confusion Matrix (First Attempt)')
ax.set_xlabel('Predicted')
ax.set_ylabel('Actual')
plt.show()

What does this code do?

  • classification_report prints precision, recall, and f1-score for each class. Precision measures how many of the stocks the model called "gainers" actually were gainers. Recall measures how many of the true gainers the model managed to find.
  • confusion_matrix creates a grid showing correct vs incorrect predictions for each class. The heatmap makes it easy to see where the model struggles.

Before you run this, do you think the model predicts gainers and losers equally well? Or does it favor one class over the other?

  • Run Cell 5 again by pressing Shift+Enter.

You should see the accuracy score, a classification report table, and a confusion matrix heatmap. Now look at the confusion matrix closely.

What should I see?

The confusion matrix reveals that the model is much better at predicting one class than the other. Compare the recall values in the classification report. One class has a significantly higher recall. This is class imbalance at work.

Remember the class distribution you printed in the previous step? The training data had more of one class, so the model learned to favor predicting that class. This is a real-world problem in machine learning, not a bug in your code.

Not seeing the heatmap?

  • Make sure plt.show() is the last line in the cell. Without it, Colab may not render the chart.
  • Check that you added the new code below the existing print(f"Accuracy...") line, not in a separate cell.
  • Why isn't my confusion matrix heatmap showing in Colab?

✔️ Awesome, I've got everything!

Great. Make sure Cell 5 is saved in your notebook before moving on.

ⓧ I'd like to double check the full code

Here is the complete Cell 5. Make sure your code matches this exactly:

# Cell 5: Train Random Forest (First Attempt)
X = df_model[feature_cols]
y = df_model[target_col]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, random_state=42
)

rf_model = RandomForestClassifier(n_estimators=100, random_state=42)
rf_model.fit(X_train, y_train)
y_pred = rf_model.predict(X_test)

accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy (first attempt): {accuracy:.2f}")
print("\nClassification Report:")
print(classification_report(y_test, y_pred, target_names=['Loser', 'Gainer']))

# Confusion matrix
fig, ax = plt.subplots(figsize=(6, 5))
cm = confusion_matrix(y_test, y_pred)
sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', ax=ax,
            xticklabels=['Loser', 'Gainer'],
            yticklabels=['Loser', 'Gainer'])
ax.set_title('Confusion Matrix (First Attempt)')
ax.set_xlabel('Predicted')
ax.set_ylabel('Actual')
plt.show()

Your model has a blind spot. It favors the majority class and struggles with the minority class. In the next step, you will fix this imbalance and discover which features matter most for predicting stock movement on NEPSE.

Visualize Feature Importance and Fix the Imbalance

Your first Random Forest model is trained, and the confusion matrix told an honest story. The model predicts one class far better than the other because the dataset has more losers (or gainers) than the opposite.

This is called class imbalance, and it happens constantly in real financial data. In this step, you will fix it by retraining the model with balanced class weights, then reveal which trading metrics actually drive stock movement on NEPSE.

In this step, get ready to:
  • Retrain the Random Forest with balanced class weights to fix the imbalance.
  • Compare the improved confusion matrix and classification report against your first attempt.
  • Build a feature importance chart that reveals which trading metrics matter most for NEPSE prediction.
Retrain the model with balanced class weights

When one class outnumbers the other, a model can score high accuracy by mostly predicting the majority class. The class_weight='balanced' parameter tells scikit-learn to automatically give more weight to the underrepresented class during training.

  • Create a new code cell below Cell 5 in your notebook.
  • Add the following code to train a balanced Random Forest and print its accuracy and classification report:
# Cell 6: Retrain with Balanced Classes and Feature Importance
rf_balanced = RandomForestClassifier(
    n_estimators=100, random_state=42, class_weight='balanced'
)
rf_balanced.fit(X_train, y_train)
y_pred_balanced = rf_balanced.predict(X_test)

accuracy_balanced = accuracy_score(y_test, y_pred_balanced)
print(f"Accuracy (balanced): {accuracy_balanced:.2f}")
print("\nClassification Report (balanced):")
print(classification_report(y_test, y_pred_balanced, target_names=['Loser', 'Gainer']))

What does this code do?

  • class_weight='balanced' adjusts the weight of each class inversely proportional to how often it appears. The minority class gets a bigger penalty for misclassification, forcing the model to pay closer attention to it.
  • Everything else stays the same: same n_estimators=100, same random_state=42, same train/test split. This isolates the effect of balancing so you can compare fairly.
  • The classification report prints precision, recall, and F1-score for both classes. Watch the recall row for the minority class specifically.
  • Run the cell by pressing the play button or Shift+Enter.

You should see the balanced accuracy score printed, followed by a classification report. Compare the recall values to your first attempt in Cell 5. The minority class recall should be higher this time.

Seeing a NameError?

  • Make sure you ran all the cells above (Cells 1 through 5) before running Cell 6. Colab cells depend on variables created by earlier cells.
  • If you restarted the Colab runtime, you need to re-run every cell from the top. Use Runtime > Run all to re-execute all cells in order.

Why is my balanced model throwing a NameError?

Visualize the balanced results and feature importance

The text report confirmed the improvement, but a side-by-side visual makes the change unmistakable. You will also answer the most interesting question in this project: which trading metrics actually matter for predicting stock movement on NEPSE?

  • In the same Cell 6, add the following code below the classification report lines you just wrote:
fig, axes = plt.subplots(1, 2, figsize=(14, 5))

# Updated confusion matrix
cm_balanced = confusion_matrix(y_test, y_pred_balanced)
sns.heatmap(cm_balanced, annot=True, fmt='d', cmap='Blues', ax=axes[0],
            xticklabels=['Loser', 'Gainer'],
            yticklabels=['Loser', 'Gainer'])
axes[0].set_title('Confusion Matrix (Balanced)')
axes[0].set_xlabel('Predicted')
axes[0].set_ylabel('Actual')

# Feature importance
importances = rf_balanced.feature_importances_
feat_imp = pd.Series(importances, index=feature_cols).sort_values()
feat_imp.plot(kind='barh', ax=axes[1], color='teal')
axes[1].set_title('Feature Importance for NEPSE Stock Prediction')
axes[1].set_xlabel('Importance')

plt.tight_layout()
plt.show()

most_important = feat_imp.idxmax()
print(f"\nMost important feature for predicting NEPSE stock movement: {most_important}")

What does this code do?

  • plt.subplots(1, 2) creates two charts side by side. The left chart is the updated confusion matrix. The right chart shows feature importance.
  • rf_balanced.feature_importances_ extracts how much each feature contributed to the model's decisions. Higher importance means the model relied on that feature more when splitting its decision trees.
  • feat_imp.idxmax() returns the name of the feature with the highest importance score, so you can see exactly which trading metric matters most.

Before you run the cell, think about this: which of the seven features (ltp, volume, turnover, trades, volume_per_trade, turnover_per_volume, price_range) do you think matters most for predicting whether a NEPSE stock is a daily gainer or loser?

  • Run Cell 6 again by pressing Shift+Enter.

You should see the accuracy and classification report printed first, then two charts side by side. The left chart is the updated confusion matrix showing more balanced predictions across both classes. The right chart is a horizontal bar chart ranking all seven features from least to most important.

Below the charts, a printed sentence reveals which feature the model considers most important for predicting NEPSE stock movement.

Why did the confusion matrix change?

Compare the balanced confusion matrix to your first attempt in Cell 5. The first model favored the majority class, correctly predicting it most of the time but struggling with the minority class.

With class_weight='balanced', the model trades some overall accuracy for better recall on the underrepresented class. In real-world finance, catching both gainers and losers is usually more valuable than being great at predicting only one.

Charts not showing up?

  • Make sure the visualization code is in the same cell as the training code (Cell 6). If you accidentally put it in a separate cell, the y_pred_balanced and rf_balanced variables should still work, but double check that the training code ran first.
  • If you see a blank chart area, check that you included plt.show() at the end of the visualization block.

Why aren't my confusion matrix and feature importance charts displaying?

✔️ Awesome, I've got everything!

Great. Double check that Cell 6 is saved and both the text output (accuracy, classification report) and the two charts (confusion matrix, feature importance) are visible.

ⓧ I'd like to double check the full code

Your notebook should now have six cells. Scroll through and confirm each one is present in order:

  • Cell 1: Import Libraries and Fetch Live NEPSE Data.
  • Cell 2: Clean Data and Add Sector Information.
  • Cell 3: Exploratory Data Analysis.
  • Cell 4: Feature Engineering.
  • Cell 5: Train Random Forest (First Attempt).
  • Cell 6: Retrain with Balanced Classes and Feature Importance.

If any cell is missing or out of order, scroll up to the corresponding step and re-add it.

You now have a balanced model that predicts both gainers and losers, and a clear answer to which metrics drive stock movement on NEPSE. Next up, you will take on an optional Secret Mission to investigate IPO performance against the broader market.

Secret mission

Track IPO Performance Against the Market

Your notebook analyzes the NEPSE market as a snapshot. But what about the newest companies that just went public? In this mission, you will pull IPO announcement data from the yonepse API and cross-reference it with current stock prices. The result is a clear answer: are recently IPO'd companies outperforming or underperforming the broader market?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely in Google Colab, so there are no ongoing costs or paid resources to worry about. Your notebook is saved in your Google Drive for free.

Resources you used:

  • Google Colab notebook NEPSE_Market_Analyzer (free, stored in your Google Drive).

Keep everything running

No action needed. Choose this if you want to keep your notebook and rerun it anytime to pull fresh NEPSE data.

  • Your notebook stays in your Google Drive under the Colab Notebooks folder.
  • You can reopen it at any time from colab.research.google.com and run all cells to pull the latest stock data.
  • The notebook uses no storage beyond a few kilobytes in your Google Drive.

Pause - I'll come back to this later

Since Colab notebooks do not run in the background, there is nothing to pause. Your notebook is already saved and idle.

  • Download a local copy as a backup. In your notebook, click File, then Download, then Download .ipynb.
  • Save the .ipynb file somewhere you can find it later.
  • When you come back, upload it to Colab or open it directly from Google Drive.

Delete - I don't want to use this again

Remove the notebook from your Google Drive entirely.

  • In your open notebook, click File in the Colab menu bar.
  • Click Move to trash.
  • To permanently delete it, open drive.google.com, click Trash in the left sidebar, find the notebook, right-click it, and select Delete forever.

Nice Work!

Nice Work!

You just built a complete data science pipeline that pulls live Nepal stock market data, visualizes it, and uses machine learning to predict daily stock movement. This notebook is yours to rerun anytime the market is open for fresh results.

You've learned how to:

  • Pull live financial data from a public API, clean it into a pandas DataFrame, and explore sector distributions and correlations through bar charts and heatmaps.
  • Engineer meaningful trading features and train a Random Forest classifier to predict whether NEPSE stocks are daily gainers or losers.
  • Diagnose class imbalance in a real-world dataset, fix it with balanced class weights, and use a feature importance chart to discover which trading metrics matter most.
  • Secret Mission: Pulled IPO announcement data, matched companies to live stock symbols, and compared recently listed companies against the market average.

Ready to quiz yourself?