Regex for Clinical NLP in Python
Build a regex pattern library to detect sections and extract respiratory findings.
Introduction
30 Second Summary
Clinical notes are messy. Doctors write in shorthand, formatting varies between systems, and critical findings like "wheezing" appear as "wheeze", "wheezes", "wheezing", or "wheezed" depending on context. Before any AI model can process these notes, someone has to write the rules that detect where sections start and what findings are buried in the text.
In this project, you will build a regex pattern library for clinical NLP using Python. You will break down patterns character by character, extract respiratory findings from clinical text, and detect sections in medical notes.
What You'll Build
A Python-based regex tutorial workbook (a single regex_clinical_nlp.py file) that teaches regex concepts using real clinical NLP examples, building toward section detection and respiratory finding extraction.
By the end of this project, you'll have:
- A character-by-character understanding of r"\bwheez(?:e|es|ing|ed)?\b" and the ability to write similar patterns for new findings.
- A section detection function that splits clinical notes into named sections like FINDINGS, IMPRESSION, and PLAN.
- A respiratory finding extractor that identifies wheezing, crackles, rhonchi, and rales across all their word forms.
- Secret Mission: Build a negation-aware regex that distinguishes "no wheezing" from "wheezing present" and correctly flags negated findings.
Are there any prerequisites?
You need basic Python knowledge: variables, strings, print statements, and functions. No prior regex experience is required.
Understand Word Boundaries and the Wheeze Pattern
Your environment is ready and the re module is confirmed working. Now it is time to write your first real pattern.
Clinical notes contain the same medical finding written in many different word forms. A doctor might write "wheezing", "wheeze", "wheezes", or "wheezed" all to describe the same symptom. Your regular expressions need to catch every form without accidentally matching unrelated words like "wheezy" or "nonwheezing".
In this step, get ready to:
- Break down the pattern r"\bwheez(?:e|es|ing|ed)?\b" character by character.
- Test what the pattern matches and what it correctly rejects.
- Understand why word boundaries prevent false matches in clinical text.
Break down the wheeze pattern character by character
The pattern r"\bwheez(?:e|es|ing|ed)?\b" is your first clinical regex. Every single character in this pattern serves a specific purpose. You will type it out with a full comment breakdown so you can reference it later.
- In regex_clinical_nlp.py, add the following code to start your clinical regex workbook:
import re
# ============================================================
# SECTION 1: Word Boundaries and the Wheeze Pattern
# ============================================================
# CHARACTER-BY-CHARACTER BREAKDOWN of r"\bwheez(?:e|es|ing|ed)?\b"
#
# r"..." = raw string (tells Python: don't interpret backslashes)
# \b = word boundary (the invisible edge between a word and a space/punctuation)
# w = literal letter 'w'
# h = literal letter 'h'
# e = literal letter 'e'
# e = literal letter 'e'
# z = literal letter 'z'
# ( = start of a group
# ?: = makes this group NON-CAPTURING (groups without extracting)
# e = option 1: literal 'e' (makes "wheeze")
# | = OR
# es = option 2: literal 'es' (makes "wheezes")
# | = OR
# ing = option 3: literal 'ing' (makes "wheezing")
# | = OR
# ed = option 4: literal 'ed' (makes "wheezed")
# ) = end of group
# ? = the ENTIRE group is optional (zero or one time)
# \b = word boundary (end of word)
wheeze_pattern = re.compile(r"\bwheez(?:e|es|ing|ed)?\b", re.IGNORECASE)What does this code do?
The comment block is a character-by-character reference guide for the entire pattern. It explains every symbol so you can come back and modify the pattern later.
The re.compile() call turns the raw regex string into a reusable pattern object. The re.IGNORECASE flag lets it match "Wheezing", "WHEEZING", and "wheezing" equally, because clinical notes have inconsistent capitalization.
- Save regex_clinical_nlp.py.
- You will not see any output yet. The next chunk adds tests that prove this pattern works.
Test what the pattern matches
A regex pattern is only trustworthy when you can prove it catches every word form you need. This test list covers all five forms a clinician might use when documenting wheezing.
- Add the following code below the wheeze_pattern line in regex_clinical_nlp.py:
# What it MATCHES:
print("=== MATCHES ===")
test_matches = ["wheez", "wheeze", "wheezes", "wheezing", "wheezed"]
for word in test_matches:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")What does this code do?
The list test_matches contains five word forms: the bare stem "wheez", plus four suffixed forms. The for loop runs wheeze_pattern.search() against each word and prints whether it matched.
The bare stem "wheez" matches because the ? after the group makes the entire suffix optional. This handles cases where clinical shorthand drops the ending.
- Save regex_clinical_nlp.py and run it in your terminal:
python regex_clinical_nlp.py- You should see all five words print MATCH next to them.
Seeing NO MATCH for everything?
- Check that wheeze_pattern = re.compile(...) appears above the test code. Python executes top to bottom, so the pattern must exist before the loop uses it.
- Make sure you typed wheeze_pattern.search(word) and not wheeze_pattern.match(word). Both work here, but search() is what the script uses.
Still stuck? Why is my wheeze pattern not matching any words?
Test what the pattern does not match
Proving what a pattern rejects is just as important as proving what it catches. A pattern that matches too broadly will generate false positives in your clinical pipeline.
- Add the following code below your match tests in regex_clinical_nlp.py:
# What it does NOT match:
print("\n=== DOES NOT MATCH ===")
test_no_match = ["wheezy", "wheeziness", "nonwheezing"]
for word in test_no_match:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# WHY designed this way:
# - \b at start: prevents matching "wheez" inside "nonwheezing"
# - (?:...) group: bundles the suffix options together
# - ? after group: lets us match the bare stem "wheez" too
# - \b at end: prevents matching "wheeze" inside "wheezer"
# - re.IGNORECASE: clinical notes use inconsistent capitalizationWhy do word boundaries matter here?
Without \b at the start, the pattern would find "wheez" hiding inside "nonwheezing". That would be a false positive because "nonwheezing" means the patient is NOT wheezing.
Without \b at the end, the pattern would match "wheeze" inside "wheezer" or "wheezy". The suffix group only allows e, es, ing, or ed. The trailing \b enforces that nothing else follows.
- Save regex_clinical_nlp.py and run it again:
python regex_clinical_nlp.py- You should now see both sections printed. All five words under === MATCHES === show MATCH, and all three words under === DOES NOT MATCH === show NO MATCH.
Seeing MATCH for wheezy or nonwheezing?
- Double check that both \b anchors are present in your pattern string. The pattern must start with \b and end with \b.
- Make sure you are using a raw string (r"..."). Without the r prefix, Python interprets \b as a backspace character instead of a word boundary.
Need help? Why is my pattern matching words it should reject?
✔️ Awesome, I've got everything!
Great. Double check you have saved regex_clinical_nlp.py before moving on.
ⓧ I'd like to double check the full code
Here is the complete regex_clinical_nlp.py file at this point:
import re
# ============================================================
# SECTION 1: Word Boundaries and the Wheeze Pattern
# ============================================================
# CHARACTER-BY-CHARACTER BREAKDOWN of r"\bwheez(?:e|es|ing|ed)?\b"
#
# r"..." = raw string (tells Python: don't interpret backslashes)
# \b = word boundary (the invisible edge between a word and a space/punctuation)
# w = literal letter 'w'
# h = literal letter 'h'
# e = literal letter 'e'
# e = literal letter 'e'
# z = literal letter 'z'
# ( = start of a group
# ?: = makes this group NON-CAPTURING (groups without extracting)
# e = option 1: literal 'e' (makes "wheeze")
# | = OR
# es = option 2: literal 'es' (makes "wheezes")
# | = OR
# ing = option 3: literal 'ing' (makes "wheezing")
# | = OR
# ed = option 4: literal 'ed' (makes "wheezed")
# ) = end of group
# ? = the ENTIRE group is optional (zero or one time)
# \b = word boundary (end of word)
wheeze_pattern = re.compile(r"\bwheez(?:e|es|ing|ed)?\b", re.IGNORECASE)
# What it MATCHES:
print("=== MATCHES ===")
test_matches = ["wheez", "wheeze", "wheezes", "wheezing", "wheezed"]
for word in test_matches:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# What it does NOT match:
print("\n=== DOES NOT MATCH ===")
test_no_match = ["wheezy", "wheeziness", "nonwheezing"]
for word in test_no_match:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# WHY designed this way:
# - \b at start: prevents matching "wheez" inside "nonwheezing"
# - (?:...) group: bundles the suffix options together
# - ? after group: lets us match the bare stem "wheez" too
# - \b at end: prevents matching "wheeze" inside "wheezer"
# - re.IGNORECASE: clinical notes use inconsistent capitalizationYou now have a fully annotated regex pattern that catches every valid form of "wheeze" while correctly rejecting words that look similar. Next up, you will learn quantifiers and use the same pattern structure to build matchers for crackles and rhonchi.
Learn Quantifiers and Write Finding Patterns
In the previous step, you broke down the wheeze pattern character by character and learned how \b word boundaries prevent false matches inside compound words.
Now you need to control how many times a part of your pattern repeats. Clinical notes are full of inconsistent spacing and word forms that vary by suffix. Quantifiers let you say "this part is optional" or "this can appear zero or more times" so a single pattern handles all the messiness.
In this step, get ready to:
- Learn the * (star) quantifier for matching optional whitespace in clinical data.
- Understand how ? (question mark) makes word suffixes optional in your patterns.
- Write new finding patterns for crackles and rhonchi using the same structure as the wheeze pattern.
Learn the star quantifier with clinical whitespace
The * quantifier means "zero or more of the previous thing." In clinical notes, spacing around colons and values is wildly inconsistent.
A respiratory rate might appear as "RR:18", "RR: 18", or "RR : 18". The pattern \s* handles all of these by matching zero or more whitespace characters.
- In regex_clinical_nlp.py, find the last line of your Section 1 code (the comment about re.IGNORECASE).
- Add the following code below it to start Section 2:
# ============================================================
# SECTION 2: Quantifiers - Star and Question Mark
# ============================================================
# * (star) = zero or more of the previous thing
# ? (question mark) = zero or one of the previous thing (OPTIONAL)
# Star example: matching optional whitespace
# Clinical notes have inconsistent spacing: "RR: 18" vs "RR:18" vs "RR : 18"
spacing_pattern = re.compile(r"RR\s*:\s*\d+")
spacing_tests = ["RR:18", "RR: 18", "RR : 18", "RR : 22"]
print("\n=== STAR (*) for optional whitespace ===")
for test in spacing_tests:
result = spacing_pattern.search(test)
print(f" '{test}' -> {'MATCH' if result else 'NO MATCH'}")What does this code do?
The pattern r"RR\s*:\s*\d+" breaks down like this:
- RR matches the literal letters "RR" (respiratory rate).
- \s* matches zero or more whitespace characters. This absorbs any spaces (or no spaces) before the colon.
- : matches the literal colon.
- \d+ matches one or more digits (the actual value).
The test loop checks four spacing variations. All four match because \s* absorbs however many spaces exist (including zero).
- Save regex_clinical_nlp.py.
- Run the script in your terminal by running:
python regex_clinical_nlp.py- You should see the Section 1 output followed by a new === STAR (*) for optional whitespace === block where all four spacing tests show MATCH.
All four tests showing NO MATCH?
- Make sure you have a backslash before the s in \s*. Without it, you are matching a literal letter "s" instead of whitespace.
- Check that you used a raw string (r"..."). Without the r prefix, Python interprets backslashes as escape sequences.
Help me fix my spacing_pattern regex
Understand how the question mark makes suffixes optional
The ? quantifier means "zero or one of the previous thing." It makes something optional.
You already saw this in your wheeze pattern from the previous step. The ? after the non-capturing group (?:e|es|ing|ed)? makes the entire suffix group optional. That is why "wheez" alone (the bare stem with no suffix) is a valid match.
Star vs question mark
- * means "zero or more" of the previous thing. It can repeat indefinitely.
- ? means "zero or one" of the previous thing. It either appears exactly once, or not at all.
Use * when the count is unpredictable (whitespace). Use ? when something is either there or not (a word suffix).
Write finding patterns for crackle and rhonchi
Now you will apply both ? and the (?:...)? structure to build patterns for two more respiratory findings. These follow the same design as the wheeze pattern.
- Add the crackle pattern below the spacing_pattern test loop by copying this code:
# Question mark in the wheeze pattern:
# (?:e|es|ing|ed)? <-- the ? makes the ENTIRE group optional
# This means "wheez" alone (with no suffix) is a valid match
# --- Practice: crackle variants ---
# Pattern: \bcrackl(?:e|es|ing)?\b
crackle_pattern = re.compile(r"\bcrackl(?:e|es|ing)?\b", re.IGNORECASE)
print("\n=== Crackle pattern ===")
crackle_tests = ["crackle", "crackles", "crackling", "crackly", "crackled"]
for word in crackle_tests:
result = crackle_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")What does this pattern do?
The pattern r"\bcrackl(?:e|es|ing)?\b" uses the same structure as the wheeze pattern:
- \b word boundaries on both sides prevent partial matches inside longer words.
- crackl is the common stem shared by all variants.
- (?:e|es|ing)? optionally matches one of three suffixes. Notice "crackly" and "crackled" are intentionally excluded because they are not standard clinical terms.
- Save regex_clinical_nlp.py and run it again.
- In the === Crackle pattern === output, you should see "crackle", "crackles", and "crackling" as MATCH, while "crackly" and "crackled" show NO MATCH.
Seeing MATCH for crackly or crackled?
- Check that your pattern ends with \b. Without the trailing word boundary, "crackl" inside "crackly" would match the stem.
- Verify that your suffix group only has e|es|ing as options. Adding "y" or "ed" would expand what matches.
Help me debug my crackle regex pattern
- Now add the rhonchi pattern below your crackle code:
# --- Practice: rhonchi/rhonchus ---
# Pattern: \brhonch(?:i|us)\b
rhonchi_pattern = re.compile(r"\brhonch(?:i|us)\b", re.IGNORECASE)
print("\n=== Rhonchi pattern ===")
rhonchi_tests = ["rhonchi", "rhonchus", "rhonchial"]
for word in rhonchi_tests:
result = rhonchi_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")Why is there no question mark on this pattern?
The rhonchi pattern uses (?:i|us) without the trailing ?. That means the suffix is required, not optional.
The bare stem "rhonch" is never used alone in clinical text. It always appears as either "rhonchi" (plural) or "rhonchus" (singular). Making the suffix required prevents matching partial words like "rhonchial".
- Save regex_clinical_nlp.py and run the script one more time.
- In the === Rhonchi pattern === output, you should see "rhonchi" and "rhonchus" as MATCH, and "rhonchial" as NO MATCH.
Seeing MATCH for rhonchial?
- Check that your suffix group is (?:i|us) without the ? after it. If you accidentally added ?, the bare stem "rhonch" inside "rhonchial" becomes a valid match.
- Make sure you have \b at the end of the pattern. The trailing word boundary stops the match at "rhonchi" and prevents it from matching the "rhonchi" prefix inside "rhonchial".
Help me debug my rhonchi regex pattern
✔️ Awesome, I've got everything!
Great. Double check you have saved regex_clinical_nlp.py before moving on.
ⓧ I'd like to double check the full code
Here is what your complete regex_clinical_nlp.py should look like so far:
import re
# ============================================================
# SECTION 1: Word Boundaries and the Wheeze Pattern
# ============================================================
# CHARACTER-BY-CHARACTER BREAKDOWN of r"\bwheez(?:e|es|ing|ed)?\b"
#
# r"..." = raw string (tells Python: don't interpret backslashes)
# \b = word boundary (the invisible edge between a word and a space/punctuation)
# w = literal letter 'w'
# h = literal letter 'h'
# e = literal letter 'e'
# e = literal letter 'e'
# z = literal letter 'z'
# ( = start of a group
# ?: = makes this group NON-CAPTURING (groups without extracting)
# e = option 1: literal 'e' (makes "wheeze")
# | = OR
# es = option 2: literal 'es' (makes "wheezes")
# | = OR
# ing = option 3: literal 'ing' (makes "wheezing")
# | = OR
# ed = option 4: literal 'ed' (makes "wheezed")
# ) = end of group
# ? = the ENTIRE group is optional (zero or one time)
# \b = word boundary (end of word)
wheeze_pattern = re.compile(r"\bwheez(?:e|es|ing|ed)?\b", re.IGNORECASE)
# What it MATCHES:
print("=== MATCHES ===")
test_matches = ["wheez", "wheeze", "wheezes", "wheezing", "wheezed"]
for word in test_matches:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# What it does NOT match:
print("\n=== DOES NOT MATCH ===")
test_no_match = ["wheezy", "wheeziness", "nonwheezing"]
for word in test_no_match:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# WHY designed this way:
# - \b at start: prevents matching "wheez" inside "nonwheezing"
# - (?:...) group: bundles the suffix options together
# - ? after group: lets us match the bare stem "wheez" too
# - \b at end: prevents matching "wheeze" inside "wheezer"
# - re.IGNORECASE: clinical notes use inconsistent capitalization
# ============================================================
# SECTION 2: Quantifiers - Star and Question Mark
# ============================================================
# * (star) = zero or more of the previous thing
# ? (question mark) = zero or one of the previous thing (OPTIONAL)
# Star example: matching optional whitespace
# Clinical notes have inconsistent spacing: "RR: 18" vs "RR:18" vs "RR : 18"
spacing_pattern = re.compile(r"RR\s*:\s*\d+")
spacing_tests = ["RR:18", "RR: 18", "RR : 18", "RR : 22"]
print("\n=== STAR (*) for optional whitespace ===")
for test in spacing_tests:
result = spacing_pattern.search(test)
print(f" '{test}' -> {'MATCH' if result else 'NO MATCH'}")
# Question mark in the wheeze pattern:
# (?:e|es|ing|ed)? <-- the ? makes the ENTIRE group optional
# This means "wheez" alone (with no suffix) is a valid match
# --- Practice: crackle variants ---
# Pattern: \bcrackl(?:e|es|ing)?\b
crackle_pattern = re.compile(r"\bcrackl(?:e|es|ing)?\b", re.IGNORECASE)
print("\n=== Crackle pattern ===")
crackle_tests = ["crackle", "crackles", "crackling", "crackly", "crackled"]
for word in crackle_tests:
result = crackle_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# --- Practice: rhonchi/rhonchus ---
# Pattern: \brhonch(?:i|us)\b
rhonchi_pattern = re.compile(r"\brhonch(?:i|us)\b", re.IGNORECASE)
print("\n=== Rhonchi pattern ===")
rhonchi_tests = ["rhonchi", "rhonchus", "rhonchial"]
for word in rhonchi_tests:
result = rhonchi_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")You now have three respiratory finding patterns in your toolkit. Each one uses word boundaries and non-capturing groups to match exactly the word forms that appear in clinical text. Next up, you will learn anchors that detect where something appears in a line, which is the key to finding section headers.
Use Anchors for Section Header Detection
Your regex toolkit already handles word forms and optional characters. Now you need a way to detect where a match sits inside a document.
Clinical notes are structured into named sections like FINDINGS, IMPRESSION, and PLAN. Each section header starts at the beginning of a line. Anchors (^ and $) let you match based on position rather than content, which is exactly what you need to find those headers.
In this step, get ready to:
- Learn the ^ (start of line) and $ (end of line) anchors.
- Understand how re.MULTILINE makes ^ match at the start of each line in a multi-line note.
- Write a pattern that detects section headers like FINDINGS:, IMPRESSION:, and PLAN: at the start of lines.
Write a section header pattern with ^ and re.MULTILINE
The ^ anchor matches the start of a string. By default, it only matches the very beginning of the entire text. In clinical notes that span many lines, you need it to match at the start of every line. The re.MULTILINE flag changes that behavior.
- In regex_clinical_nlp.py, add the following code at the bottom of your file (below the rhonchi pattern section):
# ============================================================
# SECTION 3: Anchors - Start and End of Line
# ============================================================
# ^ = start of string (or start of line with re.MULTILINE)
# $ = end of string (or end of line with re.MULTILINE)
# Without re.MULTILINE: ^ only matches the very start of the entire string
# With re.MULTILINE: ^ matches at the start of EACH line
# Section headers in clinical notes start at the beginning of a line
section_header_pattern = re.compile(r"^[A-Z ]+:", re.MULTILINE)
note = """FINDINGS: Bilateral wheezing noted.
IMPRESSION: Acute bronchospasm.
PLAN: Albuterol nebulizer treatment."""
headers = section_header_pattern.findall(note)
print("\n=== Section headers found with ^ ===")
print(f" {headers}")What does this code do?
The pattern r"^[A-Z ]+:" works character by character:
- ^ anchors the match to the start of a line (because of re.MULTILINE).
- [A-Z ] matches any uppercase letter or space (clinical headers are typically ALL CAPS).
- + means one or more of the previous character class.
- : is a literal colon that ends every section header.
The note variable holds a multi-line clinical note with three sections. The findall() call returns every header it finds at the start of a line.
Why re.MULTILINE is critical for clinical notes
Without re.MULTILINE, the ^ anchor only matches the very first character of the entire string. That means it would only find "FINDINGS:" and miss "IMPRESSION:" and "PLAN:" entirely.
With re.MULTILINE, ^ treats every newline as a fresh "start of line." This is essential because clinical notes are multi-line documents with many section headers.
- Save regex_clinical_nlp.py.
- Run the script in your terminal by running:
python regex_clinical_nlp.py- Near the bottom of the output, you should see:
=== Section headers found with ^ ===
['FINDINGS:', 'IMPRESSION:', 'PLAN:']Only seeing one header instead of three?
If your output only shows ['FINDINGS:'], check that you included re.MULTILINE as the second argument to re.compile(). Without it, ^ only matches the very start of the entire string.
help me debug my section header pattern
Detect line endings with $
The $ anchor is the mirror of ^. It matches the end of a line (or end of the entire string without re.MULTILINE). This is useful for checking whether lines in a clinical note end with proper punctuation.
- Add this code directly below the print(f" {headers}") line:
# $ example: detecting if a line ends with a period (complete sentence)
complete_line = re.compile(r"\.$", re.MULTILINE)
endings = complete_line.findall(note)
print(f" Lines ending with period: {len(endings)}")What does this code do?
- r"\.$" matches a literal period (\. escapes the dot so it means a real period, not "any character") at the end of a line ($).
- Since the note has three lines that each end with a period, len(endings) returns 3.
- Save regex_clinical_nlp.py and run the script again.
- You should now see an additional line of output: Lines ending with period: 3.
Seeing 0 lines ending with period?
Check that you escaped the dot with a backslash: \. means "literal period." A bare . in regex means "any character," which would match the last character of every line instead.
Also confirm you included re.MULTILINE so that $ matches the end of each line, not just the end of the entire string.
help me fix my end-of-line pattern
✔️ Awesome, I've got everything!
Great. Double check you have saved regex_clinical_nlp.py before moving on.
ⓧ I'd like to double check the full code
Here is the complete regex_clinical_nlp.py file up to this point. Compare yours against it:
import re
# ============================================================
# SECTION 1: Word Boundaries and the Wheeze Pattern
# ============================================================
# CHARACTER-BY-CHARACTER BREAKDOWN of r"\bwheez(?:e|es|ing|ed)?\b"
#
# r"..." = raw string (tells Python: don't interpret backslashes)
# \b = word boundary (the invisible edge between a word and a space/punctuation)
# w = literal letter 'w'
# h = literal letter 'h'
# e = literal letter 'e'
# e = literal letter 'e'
# z = literal letter 'z'
# ( = start of a group
# ?: = makes this group NON-CAPTURING (groups without extracting)
# e = option 1: literal 'e' (makes "wheeze")
# | = OR
# es = option 2: literal 'es' (makes "wheezes")
# | = OR
# ing = option 3: literal 'ing' (makes "wheezing")
# | = OR
# ed = option 4: literal 'ed' (makes "wheezed")
# ) = end of group
# ? = the ENTIRE group is optional (zero or one time)
# \b = word boundary (end of word)
wheeze_pattern = re.compile(r"\bwheez(?:e|es|ing|ed)?\b", re.IGNORECASE)
# What it MATCHES:
print("=== MATCHES ===")
test_matches = ["wheez", "wheeze", "wheezes", "wheezing", "wheezed"]
for word in test_matches:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# What it does NOT match:
print("\n=== DOES NOT MATCH ===")
test_no_match = ["wheezy", "wheeziness", "nonwheezing"]
for word in test_no_match:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# WHY designed this way:
# - \b at start: prevents matching "wheez" inside "nonwheezing"
# - (?:...) group: bundles the suffix options together
# - ? after group: lets us match the bare stem "wheez" too
# - \b at end: prevents matching "wheeze" inside "wheezer"
# - re.IGNORECASE: clinical notes use inconsistent capitalization
# ============================================================
# SECTION 2: Quantifiers - Star and Question Mark
# ============================================================
# * (star) = zero or more of the previous thing
# ? (question mark) = zero or one of the previous thing (OPTIONAL)
# Star example: matching optional whitespace
# Clinical notes have inconsistent spacing: "RR: 18" vs "RR:18" vs "RR : 18"
spacing_pattern = re.compile(r"RR\s*:\s*\d+")
spacing_tests = ["RR:18", "RR: 18", "RR : 18", "RR : 22"]
print("\n=== STAR (*) for optional whitespace ===")
for test in spacing_tests:
result = spacing_pattern.search(test)
print(f" '{test}' -> {'MATCH' if result else 'NO MATCH'}")
# Question mark in the wheeze pattern:
# (?:e|es|ing|ed)? <-- the ? makes the ENTIRE group optional
# This means "wheez" alone (with no suffix) is a valid match
# --- Practice: crackle variants ---
# Pattern: \bcrackl(?:e|es|ing)?\b
crackle_pattern = re.compile(r"\bcrackl(?:e|es|ing)?\b", re.IGNORECASE)
print("\n=== Crackle pattern ===")
crackle_tests = ["crackle", "crackles", "crackling", "crackly", "crackled"]
for word in crackle_tests:
result = crackle_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# --- Practice: rhonchi/rhonchus ---
# Pattern: \brhonch(?:i|us)\b
rhonchi_pattern = re.compile(r"\brhonch(?:i|us)\b", re.IGNORECASE)
print("\n=== Rhonchi pattern ===")
rhonchi_tests = ["rhonchi", "rhonchus", "rhonchial"]
for word in rhonchi_tests:
result = rhonchi_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# ============================================================
# SECTION 3: Anchors - Start and End of Line
# ============================================================
# ^ = start of string (or start of line with re.MULTILINE)
# $ = end of string (or end of line with re.MULTILINE)
# Without re.MULTILINE: ^ only matches the very start of the entire string
# With re.MULTILINE: ^ matches at the start of EACH line
# Section headers in clinical notes start at the beginning of a line
section_header_pattern = re.compile(r"^[A-Z ]+:", re.MULTILINE)
note = """FINDINGS: Bilateral wheezing noted.
IMPRESSION: Acute bronchospasm.
PLAN: Albuterol nebulizer treatment."""
headers = section_header_pattern.findall(note)
print("\n=== Section headers found with ^ ===")
print(f" {headers}")
# $ example: detecting if a line ends with a period (complete sentence)
complete_line = re.compile(r"\.$", re.MULTILINE)
endings = complete_line.findall(note)
print(f" Lines ending with period: {len(endings)}")You can now pin matches to specific positions in a document. Next up, you will learn how whitespace matching and character classes let you handle the messy, inconsistent formatting that real clinical notes are full of.
Apply Whitespace, Character Classes, and Alternation
Clinical notes are written by humans in a hurry. That means spacing is inconsistent. You will see "FINDINGS:" next to "FINDINGS :" next to "FINDINGS:\t" in the same hospital system.
In this step, you will learn three tools that handle this messiness: \s for whitespace, character classes like [A-Z] for ranges, and | (alternation) for matching multiple respiratory terms in a single pattern.
In this step, get ready to:
- Build a section header pattern that handles messy whitespace using \s*.
- Combine all your respiratory finding patterns into one using alternation.
- Test both patterns against realistic clinical text.
Match section headers with variable spacing
The \s metacharacter matches any single whitespace character: a space, tab, newline, or carriage return. Combined with * (zero or more), \s* absorbs however many spaces (or none) exist between tokens.
Character classes like [A-Z] match exactly one character from a range. [A-Z]+ matches one or more uppercase letters, which is exactly what a section header like "FINDINGS" looks like.
- In regex_clinical_nlp.py, scroll to the bottom of the file (after the Section 3 code) and add the following Section 4 code:
# ============================================================
# SECTION 4: Whitespace and Character Groups
# ============================================================
# \s = any whitespace character (space, tab, newline, carriage return)
# [abc] = match exactly ONE of: a, b, or c
# [A-Z] = match any single uppercase letter (range)
# [a-z] = match any single lowercase letter
# [0-9] = match any single digit
# | (pipe/alternation) = OR for entire expressions
# Why \s matters: clinical notes have messy spacing
# "FINDINGS:" vs "FINDINGS :" vs "FINDINGS:\n" vs "FINDINGS:\t"
header_with_spacing = re.compile(r"^[A-Z]+\s*:\s*", re.MULTILINE)
messy_note = """FINDINGS: Wheezing bilateral.
IMPRESSION :Acute asthma.
PLAN:
Start albuterol."""
print("\n=== \\s handles messy spacing ===")
matches = header_with_spacing.findall(messy_note)
for m in matches:
print(f" Found: '{repr(m)}'")What does this pattern do?
- ^[A-Z]+ matches one or more uppercase letters at the start of a line (because of re.MULTILINE).
- \s*:\s* matches a colon with zero or more whitespace characters on either side. This handles "FINDINGS:", "FINDINGS :", and "FINDINGS: " all at once.
- The messy_note test string deliberately uses inconsistent formatting so you can see the pattern handle all three variations.
- Save regex_clinical_nlp.py.
- Run the script in your terminal by running:
python regex_clinical_nlp.py- Scroll to the \s handles messy spacing section of the output. You should see three "Found:" lines, one for each header in the messy note, confirming the pattern matched despite the inconsistent spacing.
Not seeing three matches?
Make sure you included re.MULTILINE as the second argument to re.compile(). Without it, ^ only matches the very first line of the string.
Also check that the triple-quoted messy_note string has each header on its own line. If they are all on one line, only the first will match.
Help me debug my whitespace pattern
Combine all respiratory findings with alternation
So far you have built separate patterns for wheezing, crackles, and rhonchi. In a real clinical NLP pipeline, you need one pattern that catches all respiratory findings in a single pass over the text.
The | (pipe) character means "OR" for entire expressions. You can chain multiple alternatives inside a non-capturing group to create one pattern that matches any of them.
- Add the following code below the header_with_spacing test loop you just wrote:
# Alternation (|) for multiple respiratory terms
# Match ANY respiratory finding in one pattern
resp_terms = re.compile(
r"\b(?:wheez(?:e|es|ing|ed)?|crackl(?:e|es|ing)?|rhonch(?:i|us)|rales?)\b",
re.IGNORECASE
)
print("\n=== Alternation (|) for multiple findings ===")
clinical_sentences = [
"Bilateral wheezing on auscultation",
"Fine crackles at the bases",
"No rhonchi appreciated",
"Scattered rales heard bilaterally",
"Lungs clear to auscultation",
]
for sentence in clinical_sentences:
findings = resp_terms.findall(sentence)
print(f" '{sentence}' -> {findings}")What does this pattern do?
This pattern uses | inside a non-capturing group to chain together every respiratory finding pattern you have built so far.
- wheez(?:e|es|ing|ed)? covers all wheeze variants.
- crackl(?:e|es|ing)? covers crackle, crackles, and crackling.
- rhonch(?:i|us) covers the two standard forms.
- rales? matches "rale" or "rales" (the ? makes the trailing "s" optional).
The fifth test sentence ("Lungs clear to auscultation") should return an empty list. This confirms the pattern does not produce false positives on clean lung exams.
- Save regex_clinical_nlp.py and run the script again:
python regex_clinical_nlp.py- Scroll to the Alternation (|) for multiple findings output. You should see matched findings for the first four sentences and an empty list [] for the last sentence.
Seeing unexpected matches or empty lists?
Check that your | pipes are inside the outer (?:...) group. If a pipe sits outside the group, it splits the entire pattern at the wrong boundary.
Also confirm the closing \b is after the outer group's closing parenthesis, not inside it. The word boundary must wrap the entire expression.
Help me fix my alternation pattern
✔️ Awesome, I've got everything!
Great. Double check you have saved your file before moving on.
ⓧ I'd like to double check the full code
Here is the complete regex_clinical_nlp.py file up to this point. Compare yours against it line by line.
import re
# ============================================================
# SECTION 1: Word Boundaries and the Wheeze Pattern
# ============================================================
# CHARACTER-BY-CHARACTER BREAKDOWN of r"\bwheez(?:e|es|ing|ed)?\b"
#
# r"..." = raw string (tells Python: don't interpret backslashes)
# \b = word boundary (the invisible edge between a word and a space/punctuation)
# w = literal letter 'w'
# h = literal letter 'h'
# e = literal letter 'e'
# e = literal letter 'e'
# z = literal letter 'z'
# ( = start of a group
# ?: = makes this group NON-CAPTURING (groups without extracting)
# e = option 1: literal 'e' (makes "wheeze")
# | = OR
# es = option 2: literal 'es' (makes "wheezes")
# | = OR
# ing = option 3: literal 'ing' (makes "wheezing")
# | = OR
# ed = option 4: literal 'ed' (makes "wheezed")
# ) = end of group
# ? = the ENTIRE group is optional (zero or one time)
# \b = word boundary (end of word)
wheeze_pattern = re.compile(r"\bwheez(?:e|es|ing|ed)?\b", re.IGNORECASE)
# What it MATCHES:
print("=== MATCHES ===")
test_matches = ["wheez", "wheeze", "wheezes", "wheezing", "wheezed"]
for word in test_matches:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# What it does NOT match:
print("\n=== DOES NOT MATCH ===")
test_no_match = ["wheezy", "wheeziness", "nonwheezing"]
for word in test_no_match:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# WHY designed this way:
# - \b at start: prevents matching "wheez" inside "nonwheezing"
# - (?:...) group: bundles the suffix options together
# - ? after group: lets us match the bare stem "wheez" too
# - \b at end: prevents matching "wheeze" inside "wheezer"
# - re.IGNORECASE: clinical notes use inconsistent capitalization
# ============================================================
# SECTION 2: Quantifiers - Star and Question Mark
# ============================================================
# * (star) = zero or more of the previous thing
# ? (question mark) = zero or one of the previous thing (OPTIONAL)
# Star example: matching optional whitespace
# Clinical notes have inconsistent spacing: "RR: 18" vs "RR:18" vs "RR : 18"
spacing_pattern = re.compile(r"RR\s*:\s*\d+")
spacing_tests = ["RR:18", "RR: 18", "RR : 18", "RR : 22"]
print("\n=== STAR (*) for optional whitespace ===")
for test in spacing_tests:
result = spacing_pattern.search(test)
print(f" '{test}' -> {'MATCH' if result else 'NO MATCH'}")
# Question mark in the wheeze pattern:
# (?:e|es|ing|ed)? <-- the ? makes the ENTIRE group optional
# This means "wheez" alone (with no suffix) is a valid match
# --- Practice: crackle variants ---
# Pattern: \bcrackl(?:e|es|ing)?\b
crackle_pattern = re.compile(r"\bcrackl(?:e|es|ing)?\b", re.IGNORECASE)
print("\n=== Crackle pattern ===")
crackle_tests = ["crackle", "crackles", "crackling", "crackly", "crackled"]
for word in crackle_tests:
result = crackle_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# --- Practice: rhonchi/rhonchus ---
# Pattern: \brhonch(?:i|us)\b
rhonchi_pattern = re.compile(r"\brhonch(?:i|us)\b", re.IGNORECASE)
print("\n=== Rhonchi pattern ===")
rhonchi_tests = ["rhonchi", "rhonchus", "rhonchial"]
for word in rhonchi_tests:
result = rhonchi_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# ============================================================
# SECTION 3: Anchors - Start and End of Line
# ============================================================
# ^ = start of string (or start of line with re.MULTILINE)
# $ = end of string (or end of line with re.MULTILINE)
# Without re.MULTILINE: ^ only matches the very start of the entire string
# With re.MULTILINE: ^ matches at the start of EACH line
# Section headers in clinical notes start at the beginning of a line
section_header_pattern = re.compile(r"^[A-Z ]+:", re.MULTILINE)
note = """FINDINGS: Bilateral wheezing noted.
IMPRESSION: Acute bronchospasm.
PLAN: Albuterol nebulizer treatment."""
headers = section_header_pattern.findall(note)
print("\n=== Section headers found with ^ ===")
print(f" {headers}")
# $ example: detecting if a line ends with a period (complete sentence)
complete_line = re.compile(r"\.$", re.MULTILINE)
endings = complete_line.findall(note)
print(f" Lines ending with period: {len(endings)}")
# ============================================================
# SECTION 4: Whitespace and Character Groups
# ============================================================
# \s = any whitespace character (space, tab, newline, carriage return)
# [abc] = match exactly ONE of: a, b, or c
# [A-Z] = match any single uppercase letter (range)
# [a-z] = match any single lowercase letter
# [0-9] = match any single digit
# | (pipe/alternation) = OR for entire expressions
# Why \s matters: clinical notes have messy spacing
# "FINDINGS:" vs "FINDINGS :" vs "FINDINGS:\n" vs "FINDINGS:\t"
header_with_spacing = re.compile(r"^[A-Z]+\s*:\s*", re.MULTILINE)
messy_note = """FINDINGS: Wheezing bilateral.
IMPRESSION :Acute asthma.
PLAN:
Start albuterol."""
print("\n=== \\s handles messy spacing ===")
matches = header_with_spacing.findall(messy_note)
for m in matches:
print(f" Found: '{repr(m)}'")
# Alternation (|) for multiple respiratory terms
# Match ANY respiratory finding in one pattern
resp_terms = re.compile(
r"\b(?:wheez(?:e|es|ing|ed)?|crackl(?:e|es|ing)?|rhonch(?:i|us)|rales?)\b",
re.IGNORECASE
)
print("\n=== Alternation (|) for multiple findings ===")
clinical_sentences = [
"Bilateral wheezing on auscultation",
"Fine crackles at the bases",
"No rhonchi appreciated",
"Scattered rales heard bilaterally",
"Lungs clear to auscultation",
]
for sentence in clinical_sentences:
findings = resp_terms.findall(sentence)
print(f" '{sentence}' -> {findings}")You now have a single pattern that detects any respiratory finding in one pass. Next up, you will learn capturing groups to extract both section names and their content from clinical notes.
Master Capturing vs Non-Capturing Groups
So far, your patterns can detect whether a finding exists in clinical text. But detection alone is not enough for a real NLP pipeline.
You also need to extract the matched text so you can store it, label it, or pass it to another system. That is where capturing groups come in. In this step, you will learn how () capturing groups pull data out of matches, and how they differ from the (?:) non-capturing groups you have already been using.
In this step, get ready to:
- Learn the difference between capturing and non-capturing groups.
- Use capturing groups to extract section names and content from clinical notes.
- Write a pattern that captures specific respiratory findings from clinical sentences.
Understand capturing vs non-capturing groups
You have already used (?:) non-capturing groups throughout this workbook. In the wheeze pattern, (?:e|es|ing|ed) bundles suffix options together so the ? quantifier applies to the entire group.
What is the difference between () and (?:)?
A non-capturing group (?:...) groups alternatives together for logic purposes (applying quantifiers, bundling options), but does NOT store what it matched. You cannot retrieve it later.
A capturing group (...) does both: it groups the pattern AND stores the matched text. You access it with match.group(1), match.group(2), etc. This is how you extract data from a match.
Extract section headers with capturing groups
Clinical notes contain structured sections like FINDINGS, IMPRESSION, and PLAN. A capturing group lets you extract both the section name and its content in a single pattern match.
- In regex_clinical_nlp.py, scroll to the bottom of the file (after the Section 4 code) and add the following code:
# ============================================================
# SECTION 5: Groups - Capturing vs Non-Capturing
# ============================================================
# (pattern) = CAPTURING group - extracts the matched text
# (?:pattern) = NON-CAPTURING group - groups alternatives without extracting
# NON-CAPTURING in the wheeze pattern:
# r"\bwheez(?:e|es|ing|ed)?\b"
# The (?:...) groups the suffixes so ? applies to ALL of them
# But it does NOT extract/capture the suffix separately
# CAPTURING group: extract section name AND content
header_pattern = re.compile(r"^([A-Z ]+):\s*(.+)", re.MULTILINE)
note_text = """FINDINGS: Bilateral wheezing noted on exam.
IMPRESSION: Acute asthma exacerbation."""
print("\n=== Capturing groups extract data ===")
for match in header_pattern.finditer(note_text):
section_name = match.group(1) # first (capturing group)
section_content = match.group(2) # second (capturing group)
print(f" Section: '{section_name}' -> Content: '{section_content}'")What does this code do?
- The comments at the top recap what you already know about non-capturing groups from the wheeze pattern, then introduce the capturing group syntax.
- r"^([A-Z ]+):\s*(.+)" has TWO capturing groups. The first ([A-Z ]+) captures the section name (uppercase letters and spaces before the colon). The second (.+) captures everything after the colon and whitespace.
- finditer() returns Match objects (unlike findall() which returns strings). Each Match lets you call .group(1) and .group(2) to retrieve each captured piece separately.
- Save regex_clinical_nlp.py.
- Run your script in the terminal. You should see the two sections extracted separately: FINDINGS mapped to its content, and IMPRESSION mapped to its content.
Seeing empty output or no matches?
- Check that you used re.MULTILINE as the flag. Without it, ^ only matches the very start of the entire string, not the start of each line.
- Make sure the triple-quoted note_text string starts on a new line after the opening """. If there is a blank line before FINDINGS, the ^ anchor still works, but the pattern expects uppercase letters immediately after the line start.
Help me debug my capturing group pattern for section headers.
Capture respiratory findings from clinical text
Now that you can extract structured section data, apply the same technique to pull out specific respiratory findings from clinical sentences. The key difference: this time you wrap the entire finding pattern in a capturing group so findall() returns the captured text directly.
- Add the following code below the capturing group section you just wrote:
# Practice: capture the specific finding from clinical text
finding_capture = re.compile(
r"\b(wheez(?:e|es|ing|ed)?|crackl(?:e|es|ing)?|rhonch(?:i|us)|rales?)\b",
re.IGNORECASE
)
print("\n=== Capturing the finding itself ===")
test_sentences = [
"Bilateral wheezing noted on auscultation.",
"Fine crackles at both bases.",
"No rhonchi or rales appreciated.",
]
for sentence in test_sentences:
found = finding_capture.findall(sentence)
print(f" '{sentence}' -> captured: {found}")What does this code do?
- This pattern looks almost identical to resp_terms from Section 4, but with one critical change: the outer group uses (...) instead of (?:...). This makes it a capturing group.
- When findall() encounters a pattern with capturing groups, it returns only the captured text (not the full match). This means the output is the exact finding word, ready to label or store.
- The inner groups like (?:e|es|ing|ed) remain non-capturing. They handle suffix logic without interfering with what gets returned.
- Save regex_clinical_nlp.py and run your script.
- You should see captured findings printed as lists: ['wheezing'] for the first sentence, ['crackles'] for the second, and ['rhonchi', 'rales'] for the third.
Seeing empty lists or unexpected results?
- If you see empty lists, double-check the opening parenthesis placement. The outer ( must come immediately after \b and wrap the entire alternation of findings.
- If you see only suffixes (like ['ing']) instead of full words, you may have accidentally made one of the inner suffix groups capturing by using (...) instead of (?:...).
Help me fix my finding capture pattern.
Why mix capturing and non-capturing groups?
In clinical NLP, you often need to match complex patterns (handling word forms, optional suffixes) while only extracting the meaningful piece. Non-capturing groups handle the structural logic. The capturing group wraps the piece you actually want to pull out of the text for labeling or storage.
✔️ Awesome, I've got everything!
Double check you have saved regex_clinical_nlp.py and your script runs without errors.
ⓧ I'd like to double check the full code
Here is the full regex_clinical_nlp.py up to and including Section 5:
import re
# ============================================================
# SECTION 1: Word Boundaries and the Wheeze Pattern
# ============================================================
# CHARACTER-BY-CHARACTER BREAKDOWN of r"\bwheez(?:e|es|ing|ed)?\b"
#
# r"..." = raw string (tells Python: don't interpret backslashes)
# \b = word boundary (the invisible edge between a word and a space/punctuation)
# w = literal letter 'w'
# h = literal letter 'h'
# e = literal letter 'e'
# e = literal letter 'e'
# z = literal letter 'z'
# ( = start of a group
# ?: = makes this group NON-CAPTURING (groups without extracting)
# e = option 1: literal 'e' (makes "wheeze")
# | = OR
# es = option 2: literal 'es' (makes "wheezes")
# | = OR
# ing = option 3: literal 'ing' (makes "wheezing")
# | = OR
# ed = option 4: literal 'ed' (makes "wheezed")
# ) = end of group
# ? = the ENTIRE group is optional (zero or one time)
# \b = word boundary (end of word)
wheeze_pattern = re.compile(r"\bwheez(?:e|es|ing|ed)?\b", re.IGNORECASE)
# What it MATCHES:
print("=== MATCHES ===")
test_matches = ["wheez", "wheeze", "wheezes", "wheezing", "wheezed"]
for word in test_matches:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# What it does NOT match:
print("\n=== DOES NOT MATCH ===")
test_no_match = ["wheezy", "wheeziness", "nonwheezing"]
for word in test_no_match:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# WHY designed this way:
# - \b at start: prevents matching "wheez" inside "nonwheezing"
# - (?:...) group: bundles the suffix options together
# - ? after group: lets us match the bare stem "wheez" too
# - \b at end: prevents matching "wheeze" inside "wheezer"
# - re.IGNORECASE: clinical notes use inconsistent capitalization
# ============================================================
# SECTION 2: Quantifiers - Star and Question Mark
# ============================================================
# * (star) = zero or more of the previous thing
# ? (question mark) = zero or one of the previous thing (OPTIONAL)
# Star example: matching optional whitespace
# Clinical notes have inconsistent spacing: "RR: 18" vs "RR:18" vs "RR : 18"
spacing_pattern = re.compile(r"RR\s*:\s*\d+")
spacing_tests = ["RR:18", "RR: 18", "RR : 18", "RR : 22"]
print("\n=== STAR (*) for optional whitespace ===")
for test in spacing_tests:
result = spacing_pattern.search(test)
print(f" '{test}' -> {'MATCH' if result else 'NO MATCH'}")
# Question mark in the wheeze pattern:
# (?:e|es|ing|ed)? <-- the ? makes the ENTIRE group optional
# This means "wheez" alone (with no suffix) is a valid match
# --- Practice: crackle variants ---
# Pattern: \bcrackl(?:e|es|ing)?\b
crackle_pattern = re.compile(r"\bcrackl(?:e|es|ing)?\b", re.IGNORECASE)
print("\n=== Crackle pattern ===")
crackle_tests = ["crackle", "crackles", "crackling", "crackly", "crackled"]
for word in crackle_tests:
result = crackle_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# --- Practice: rhonchi/rhonchus ---
# Pattern: \brhonch(?:i|us)\b
rhonchi_pattern = re.compile(r"\brhonch(?:i|us)\b", re.IGNORECASE)
print("\n=== Rhonchi pattern ===")
rhonchi_tests = ["rhonchi", "rhonchus", "rhonchial"]
for word in rhonchi_tests:
result = rhonchi_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# ============================================================
# SECTION 3: Anchors - Start and End of Line
# ============================================================
# ^ = start of string (or start of line with re.MULTILINE)
# $ = end of string (or end of line with re.MULTILINE)
# Without re.MULTILINE: ^ only matches the very start of the entire string
# With re.MULTILINE: ^ matches at the start of EACH line
# Section headers in clinical notes start at the beginning of a line
section_header_pattern = re.compile(r"^[A-Z ]+:", re.MULTILINE)
note = """FINDINGS: Bilateral wheezing noted.
IMPRESSION: Acute bronchospasm.
PLAN: Albuterol nebulizer treatment."""
headers = section_header_pattern.findall(note)
print("\n=== Section headers found with ^ ===")
print(f" {headers}")
# $ example: detecting if a line ends with a period (complete sentence)
complete_line = re.compile(r"\.$", re.MULTILINE)
endings = complete_line.findall(note)
print(f" Lines ending with period: {len(endings)}")
# ============================================================
# SECTION 4: Whitespace and Character Groups
# ============================================================
# \s = any whitespace character (space, tab, newline, carriage return)
# [abc] = match exactly ONE of: a, b, or c
# [A-Z] = match any single uppercase letter (range)
# [a-z] = match any single lowercase letter
# [0-9] = match any single digit
# | (pipe/alternation) = OR for entire expressions
# Why \s matters: clinical notes have messy spacing
# "FINDINGS:" vs "FINDINGS :" vs "FINDINGS:\n" vs "FINDINGS:\t"
header_with_spacing = re.compile(r"^[A-Z]+\s*:\s*", re.MULTILINE)
messy_note = """FINDINGS: Wheezing bilateral.
IMPRESSION :Acute asthma.
PLAN:
Start albuterol."""
print("\n=== \\s handles messy spacing ===")
matches = header_with_spacing.findall(messy_note)
for m in matches:
print(f" Found: '{repr(m)}'")
# Alternation (|) for multiple respiratory terms
# Match ANY respiratory finding in one pattern
resp_terms = re.compile(
r"\b(?:wheez(?:e|es|ing|ed)?|crackl(?:e|es|ing)?|rhonch(?:i|us)|rales?)\b",
re.IGNORECASE
)
print("\n=== Alternation (|) for multiple findings ===")
clinical_sentences = [
"Bilateral wheezing on auscultation",
"Fine crackles at the bases",
"No rhonchi appreciated",
"Scattered rales heard bilaterally",
"Lungs clear to auscultation",
]
for sentence in clinical_sentences:
findings = resp_terms.findall(sentence)
print(f" '{sentence}' -> {findings}")
# ============================================================
# SECTION 5: Groups - Capturing vs Non-Capturing
# ============================================================
# (pattern) = CAPTURING group - extracts the matched text
# (?:pattern) = NON-CAPTURING group - groups alternatives without extracting
# NON-CAPTURING in the wheeze pattern:
# r"\bwheez(?:e|es|ing|ed)?\b"
# The (?:...) groups the suffixes so ? applies to ALL of them
# But it does NOT extract/capture the suffix separately
# CAPTURING group: extract section name AND content
header_pattern = re.compile(r"^([A-Z ]+):\s*(.+)", re.MULTILINE)
note_text = """FINDINGS: Bilateral wheezing noted on exam.
IMPRESSION: Acute asthma exacerbation."""
print("\n=== Capturing groups extract data ===")
for match in header_pattern.finditer(note_text):
section_name = match.group(1) # first (capturing group)
section_content = match.group(2) # second (capturing group)
print(f" Section: '{section_name}' -> Content: '{section_content}'")
# Practice: capture the specific finding from clinical text
finding_capture = re.compile(
r"\b(wheez(?:e|es|ing|ed)?|crackl(?:e|es|ing)?|rhonch(?:i|us)|rales?)\b",
re.IGNORECASE
)
print("\n=== Capturing the finding itself ===")
test_sentences = [
"Bilateral wheezing noted on auscultation.",
"Fine crackles at both bases.",
"No rhonchi or rales appreciated.",
]
for sentence in test_sentences:
found = finding_capture.findall(sentence)
print(f" '{sentence}' -> captured: {found}")You can now extract structured data from clinical notes using capturing groups. Next up, you will combine everything you have learned into a complete section detection function that splits an entire clinical note into named sections.
Build the Section Detection Function
You have all the regex building blocks: word boundaries, quantifiers, anchors, whitespace matchers, character classes, and groups. Now it is time to combine them into a real-world function.
Clinical notes are structured into named sections like HISTORY, FINDINGS, IMPRESSION, and PLAN. A section detection function splits a note into those sections so downstream tools can process each one independently.
In this step, get ready to:
- Build a complete section detection function using re.compile() with a lookahead pattern.
- Test it against a sample clinical note with HISTORY, FINDINGS, IMPRESSION, and PLAN sections.
- Understand common regex mistakes in section detection and how the pattern avoids them.
Write the detect_sections function
Detecting sections requires a pattern that finds uppercase headers followed by a colon, then captures everything until the next header (or the end of the string). This is where re.DOTALL becomes critical. It makes . match newline characters so .*? can span across multiple lines within one section.
- In regex_clinical_nlp.py, scroll to the bottom of your file (after Section 5) and add the following section header and function:
# ============================================================
# SECTION 6: Section Detection Function
# ============================================================
def detect_sections(note):
"""Extract sections from a clinical note using regex.
Handles common section headers like FINDINGS:, IMPRESSION:,
HISTORY:, PLAN:, ASSESSMENT:
"""
section_pattern = re.compile(
r"(?:^|\n)\s*([A-Z][A-Z /]+):\s*\n?(.*?)(?=(?:\n\s*[A-Z][A-Z /]+:)|\Z)",
re.DOTALL
)
sections = {}
for match in section_pattern.finditer(note):
name = match.group(1).strip()
content = match.group(2).strip()
sections[name] = content
return sectionsWhat does this pattern do?
This is the most complex pattern in the project. Here is what each piece does:
- (?:^|\n) matches the start of the string OR a newline (where a section header can begin).
- \s* allows optional whitespace before the header.
- ([A-Z][A-Z /]+) is a capturing group that grabs the section name (uppercase letters, spaces, and slashes).
- :\s*\n? matches the colon after the header, optional whitespace, and an optional newline.
- (.*?) is a non-greedy capture of the section content (as little as possible).
- (?=(?:\n\s*[A-Z][A-Z /]+:)|\Z) is a lookahead that stops capturing when it sees the next section header OR the end of the string.
The re.DOTALL flag makes . match newlines, so (.*?) can span multiple lines of content within one section.
You will not see any output change yet. The function needs test data to produce results, which you will add next.
Test against a sample clinical note
A section detection function is only useful if it handles real clinical formatting. Clinical notes include multi-line sections, blank lines between sections, and varying amounts of content per section.
- Add the following test code directly below your detect_sections function:
sample_note = """HISTORY: 45 year old male with history of asthma presents with
increased shortness of breath over the past 3 days.
FINDINGS: Bilateral expiratory wheezing. No crackles or rhonchi.
Respiratory rate 22. O2 sat 94% on room air.
IMPRESSION: Acute asthma exacerbation.
PLAN: Albuterol nebulizer q4h. Prednisone 40mg daily x 5 days.
Follow up in 48 hours."""
print("\n=== Section Detection ===")
sections = detect_sections(sample_note)
for name, content in sections.items():
print(f"\n --- {name} ---")
print(f" {content[:80]}...")What does this test do?
The sample_note is a multi-line clinical note with four sections: HISTORY, FINDINGS, IMPRESSION, and PLAN. Some sections span multiple lines and there are blank lines between them.
The test loop calls detect_sections() and prints each section name with the first 80 characters of its content. This confirms the function correctly splits the note at each header boundary.
- Save regex_clinical_nlp.py.
- Run the script in your terminal by running:
python regex_clinical_nlp.py- Scroll to the bottom of the output. You should see a "=== Section Detection ===" heading followed by four detected sections: HISTORY, FINDINGS, IMPRESSION, and PLAN, each with their content printed below.
Not seeing four sections?
If you see fewer than four sections, check that the triple-quoted string for sample_note starts and ends with """ and that each section header (HISTORY, FINDINGS, IMPRESSION, PLAN) starts on its own line.
If you see a NameError for detect_sections, make sure the function definition appears above the test code in your file.
Help me debug my section detection function
Understand common regex mistakes in section detection
The pattern you just wrote avoids two mistakes that trip up most clinical NLP developers. Understanding them helps you debug section detection in any future project.
What mistakes does this pattern avoid?
Greedy matching eating multiple sections. If you use (.*) instead of (.*?), the greedy .* gobbles up everything (including the next section headers) until the very last possible match. The ? makes it non-greedy, so it stops as soon as the lookahead spots the next header.
Forgetting to handle multi-line content. Without re.DOTALL, the . character does not match newlines. That means (.*?) would only capture the first line of a section, silently dropping the rest. The re.DOTALL flag fixes this by letting . match every character including newlines.
✔️ Awesome, I've got everything!
Double check you have saved regex_clinical_nlp.py and that running it prints all four sections (HISTORY, FINDINGS, IMPRESSION, PLAN) at the bottom of your output.
ⓧ I'd like to double check the full code
Here is the complete regex_clinical_nlp.py file up to this point:
import re
# ============================================================
# SECTION 1: Word Boundaries and the Wheeze Pattern
# ============================================================
# CHARACTER-BY-CHARACTER BREAKDOWN of r"\bwheez(?:e|es|ing|ed)?\b"
#
# r"..." = raw string (tells Python: don't interpret backslashes)
# \b = word boundary (the invisible edge between a word and a space/punctuation)
# w = literal letter 'w'
# h = literal letter 'h'
# e = literal letter 'e'
# e = literal letter 'e'
# z = literal letter 'z'
# ( = start of a group
# ?: = makes this group NON-CAPTURING (groups without extracting)
# e = option 1: literal 'e' (makes "wheeze")
# | = OR
# es = option 2: literal 'es' (makes "wheezes")
# | = OR
# ing = option 3: literal 'ing' (makes "wheezing")
# | = OR
# ed = option 4: literal 'ed' (makes "wheezed")
# ) = end of group
# ? = the ENTIRE group is optional (zero or one time)
# \b = word boundary (end of word)
wheeze_pattern = re.compile(r"\bwheez(?:e|es|ing|ed)?\b", re.IGNORECASE)
# What it MATCHES:
print("=== MATCHES ===")
test_matches = ["wheez", "wheeze", "wheezes", "wheezing", "wheezed"]
for word in test_matches:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# What it does NOT match:
print("\n=== DOES NOT MATCH ===")
test_no_match = ["wheezy", "wheeziness", "nonwheezing"]
for word in test_no_match:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# WHY designed this way:
# - \b at start: prevents matching "wheez" inside "nonwheezing"
# - (?:...) group: bundles the suffix options together
# - ? after group: lets us match the bare stem "wheez" too
# - \b at end: prevents matching "wheeze" inside "wheezer"
# - re.IGNORECASE: clinical notes use inconsistent capitalization
# ============================================================
# SECTION 2: Quantifiers - Star and Question Mark
# ============================================================
# * (star) = zero or more of the previous thing
# ? (question mark) = zero or one of the previous thing (OPTIONAL)
# Star example: matching optional whitespace
# Clinical notes have inconsistent spacing: "RR: 18" vs "RR:18" vs "RR : 18"
spacing_pattern = re.compile(r"RR\s*:\s*\d+")
spacing_tests = ["RR:18", "RR: 18", "RR : 18", "RR : 22"]
print("\n=== STAR (*) for optional whitespace ===")
for test in spacing_tests:
result = spacing_pattern.search(test)
print(f" '{test}' -> {'MATCH' if result else 'NO MATCH'}")
# Question mark in the wheeze pattern:
# (?:e|es|ing|ed)? <-- the ? makes the ENTIRE group optional
# This means "wheez" alone (with no suffix) is a valid match
# --- Practice: crackle variants ---
# Pattern: \bcrackl(?:e|es|ing)?\b
crackle_pattern = re.compile(r"\bcrackl(?:e|es|ing)?\b", re.IGNORECASE)
print("\n=== Crackle pattern ===")
crackle_tests = ["crackle", "crackles", "crackling", "crackly", "crackled"]
for word in crackle_tests:
result = crackle_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# --- Practice: rhonchi/rhonchus ---
# Pattern: \brhonch(?:i|us)\b
rhonchi_pattern = re.compile(r"\brhonch(?:i|us)\b", re.IGNORECASE)
print("\n=== Rhonchi pattern ===")
rhonchi_tests = ["rhonchi", "rhonchus", "rhonchial"]
for word in rhonchi_tests:
result = rhonchi_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# ============================================================
# SECTION 3: Anchors - Start and End of Line
# ============================================================
# ^ = start of string (or start of line with re.MULTILINE)
# $ = end of string (or end of line with re.MULTILINE)
# Without re.MULTILINE: ^ only matches the very start of the entire string
# With re.MULTILINE: ^ matches at the start of EACH line
# Section headers in clinical notes start at the beginning of a line
section_header_pattern = re.compile(r"^[A-Z ]+:", re.MULTILINE)
note = """FINDINGS: Bilateral wheezing noted.
IMPRESSION: Acute bronchospasm.
PLAN: Albuterol nebulizer treatment."""
headers = section_header_pattern.findall(note)
print("\n=== Section headers found with ^ ===")
print(f" {headers}")
# $ example: detecting if a line ends with a period (complete sentence)
complete_line = re.compile(r"\.$", re.MULTILINE)
endings = complete_line.findall(note)
print(f" Lines ending with period: {len(endings)}")
# ============================================================
# SECTION 4: Whitespace and Character Groups
# ============================================================
# \s = any whitespace character (space, tab, newline, carriage return)
# [abc] = match exactly ONE of: a, b, or c
# [A-Z] = match any single uppercase letter (range)
# [a-z] = match any single lowercase letter
# [0-9] = match any single digit
# | (pipe/alternation) = OR for entire expressions
# Why \s matters: clinical notes have messy spacing
# "FINDINGS:" vs "FINDINGS :" vs "FINDINGS:\n" vs "FINDINGS:\t"
header_with_spacing = re.compile(r"^[A-Z]+\s*:\s*", re.MULTILINE)
messy_note = """FINDINGS: Wheezing bilateral.
IMPRESSION :Acute asthma.
PLAN:
Start albuterol."""
print("\n=== \\s handles messy spacing ===")
matches = header_with_spacing.findall(messy_note)
for m in matches:
print(f" Found: '{repr(m)}'")
# Alternation (|) for multiple respiratory terms
# Match ANY respiratory finding in one pattern
resp_terms = re.compile(
r"\b(?:wheez(?:e|es|ing|ed)?|crackl(?:e|es|ing)?|rhonch(?:i|us)|rales?)\b",
re.IGNORECASE
)
print("\n=== Alternation (|) for multiple findings ===")
clinical_sentences = [
"Bilateral wheezing on auscultation",
"Fine crackles at the bases",
"No rhonchi appreciated",
"Scattered rales heard bilaterally",
"Lungs clear to auscultation",
]
for sentence in clinical_sentences:
findings = resp_terms.findall(sentence)
print(f" '{sentence}' -> {findings}")
# ============================================================
# SECTION 5: Groups - Capturing vs Non-Capturing
# ============================================================
# (pattern) = CAPTURING group - extracts the matched text
# (?:pattern) = NON-CAPTURING group - groups alternatives without extracting
# NON-CAPTURING in the wheeze pattern:
# r"\bwheez(?:e|es|ing|ed)?\b"
# The (?:...) groups the suffixes so ? applies to ALL of them
# But it does NOT extract/capture the suffix separately
# CAPTURING group: extract section name AND content
header_pattern = re.compile(r"^([A-Z ]+):\s*(.+)", re.MULTILINE)
note_text = """FINDINGS: Bilateral wheezing noted on exam.
IMPRESSION: Acute asthma exacerbation."""
print("\n=== Capturing groups extract data ===")
for match in header_pattern.finditer(note_text):
section_name = match.group(1) # first (capturing group)
section_content = match.group(2) # second (capturing group)
print(f" Section: '{section_name}' -> Content: '{section_content}'")
# Practice: capture the specific finding from clinical text
finding_capture = re.compile(
r"\b(wheez(?:e|es|ing|ed)?|crackl(?:e|es|ing)?|rhonch(?:i|us)|rales?)\b",
re.IGNORECASE
)
print("\n=== Capturing the finding itself ===")
test_sentences = [
"Bilateral wheezing noted on auscultation.",
"Fine crackles at both bases.",
"No rhonchi or rales appreciated.",
]
for sentence in test_sentences:
found = finding_capture.findall(sentence)
print(f" '{sentence}' -> captured: {found}")
# ============================================================
# SECTION 6: Section Detection Function
# ============================================================
def detect_sections(note):
"""Extract sections from a clinical note using regex.
Handles common section headers like FINDINGS:, IMPRESSION:,
HISTORY:, PLAN:, ASSESSMENT:
"""
section_pattern = re.compile(
r"(?:^|\n)\s*([A-Z][A-Z /]+):\s*\n?(.*?)(?=(?:\n\s*[A-Z][A-Z /]+:)|\Z)",
re.DOTALL
)
sections = {}
for match in section_pattern.finditer(note):
name = match.group(1).strip()
content = match.group(2).strip()
sections[name] = content
return sections
sample_note = """HISTORY: 45 year old male with history of asthma presents with
increased shortness of breath over the past 3 days.
FINDINGS: Bilateral expiratory wheezing. No crackles or rhonchi.
Respiratory rate 22. O2 sat 94% on room air.
IMPRESSION: Acute asthma exacerbation.
PLAN: Albuterol nebulizer q4h. Prednisone 40mg daily x 5 days.
Follow up in 48 hours."""
print("\n=== Section Detection ===")
sections = detect_sections(sample_note)
for name, content in sections.items():
print(f"\n --- {name} ---")
print(f" {content[:80]}...")Your section detection function can now split any clinical note into named sections. Next, you will build a respiratory finding extractor that combines all the patterns you have written into a single powerful tool.
Build the Respiratory Finding Extractor
Your detect_sections() function can now split a clinical note into named chunks. The next challenge is pulling specific medical findings out of those chunks.
In this step, you will combine every regex concept you have learned so far into a single extraction function. This function will scan clinical text for respiratory findings like wheezing, crackles, and rhonchi, returning structured data you could feed into a downstream pipeline.
In this step, get ready to:
- Build a complete respiratory finding extractor combining \b, (?:), |, ?, and character groups into one pattern.
- Test the extractor against realistic clinical sentences.
- Add practice exercises to extend the extractor with new findings.
Write the extract_respiratory_findings function
Earlier, you built individual patterns for wheezing, crackles, and rhonchi. You also used alternation to match multiple terms at once with resp_terms. Now you will wrap all of that into a reusable function that returns structured data instead of raw strings.
- In regex_clinical_nlp.py, scroll to the bottom of the file (below the Section 6 code).
- Add the following Section 7 code at the end of your file:
# ============================================================
# SECTION 7: Respiratory Finding Extraction
# ============================================================
def extract_respiratory_findings(text):
"""Extract respiratory findings from clinical text.
Returns list of dicts with 'finding' and 'raw_match' keys.
"""
pattern = re.compile(
r"\b(wheez(?:e|es|ing|ed)?|crackl(?:e|es|ing)?|rhonch(?:i|us)|rales?)\b",
re.IGNORECASE
)
results = []
for match in pattern.finditer(text):
finding = match.group(1)
results.append({
"finding": finding.lower(),
"raw_match": match.group(0).strip()
})
return resultsWhat does this function do?
This function combines every concept you have learned into one working extractor:
- The pattern uses a capturing group () so match.group(1) returns the exact text that matched.
- re.finditer() loops through ALL matches in the text, not just the first one.
- Each result is a dictionary with "finding" (lowercased for consistency) and "raw_match" (the original text as written in the note).
- Returning structured dicts instead of raw strings means downstream code can filter, count, or classify findings without re-parsing.
You will not see any new output yet. The function is defined but nothing calls it. The next chunk adds the test sentences.
Test the extractor against clinical sentences
A pattern is only useful if you know exactly what it catches and what it misses. The test sentences below cover the realistic range you would see in clinical documentation: positive findings, negated findings, plural forms, and clean lungs with no findings at all.
- Add this test block directly below the extract_respiratory_findings function:
print("\n=== Respiratory Finding Extraction ===")
test_texts = [
"Bilateral wheezing noted on auscultation.",
"No crackles or rhonchi appreciated.",
"Scattered rales heard at both bases.",
"Lungs clear, no wheezes, crackles, or rales.",
"Expiratory wheeze in right lower lobe.",
]
for text in test_texts:
findings = extract_respiratory_findings(text)
print(f"\n Text: '{text}'")
for f in findings:
print(f" -> {f['finding']} (raw: '{f['raw_match']}')")
if not findings:
print(" -> No respiratory findings detected")What does this test block do?
- Each sentence in test_texts represents a different clinical scenario: a single finding, multiple findings, and no findings.
- The loop prints the original sentence, then each extracted finding with its lowercased form and raw match.
- The if not findings fallback handles sentences where the extractor finds nothing (like "Lungs clear to auscultation" with no respiratory finding terms).
- Save regex_clinical_nlp.py.
- Run the script by executing this command in your terminal:
python regex_clinical_nlp.py- Scroll to the bottom of the output. You should see the "=== Respiratory Finding Extraction ===" header followed by each test sentence and the findings extracted from it.
What should I see?
The output for the first sentence should show wheezing as the finding. The second sentence should show both crackles and rhonchi. The fourth sentence ("Lungs clear, no wheezes, crackles, or rales.") should extract all three terms.
Notice that the extractor finds the words even when they are negated ("No crackles"). It does not understand meaning yet. That limitation is exactly what the Secret Mission addresses.
Not seeing any extraction output?
- Make sure the test block is outside the function body (not indented inside extract_respiratory_findings). It should start at column 0 with no leading spaces.
- Check that you have two blank lines between the function definition and the print statement. Python needs blank lines to close the function scope.
- If you see a NameError for extract_respiratory_findings, the function definition may have a typo or incorrect indentation.
Help me debug my respiratory finding extractor
Add practice exercises
Real clinical notes contain misspellings and additional respiratory terms your current pattern does not cover. The exercises below challenge you to extend the extractor on your own.
- Add this practice exercises block at the very end of regex_clinical_nlp.py:
# ============================================================
# PRACTICE EXERCISES
# ============================================================
print("\n" + "=" * 60)
print("PRACTICE EXERCISES")
print("=" * 60)
# Exercise 1: Add "stridor" to the finding pattern
# Hint: stridor has no common variants, just add it as |stridor
# Exercise 2: Add "ronchi" (common misspelling of rhonchi)
# Hint: modify the rhonch part to also catch "ronch"
# Exercise 3: Write a pattern for breath sounds:
# Should match: "diminished", "decreased", "absent", "clear"
# Hint: r"\b(?:diminished|decreased|absent|clear)\b"
# Exercise 4: The pattern below has a bug. What does it match incorrectly?
# buggy_pattern = re.compile(r"wheez(?:e|es|ing|ed)?")
# Hint: try it against "nonwheezing" - what happens without \b?
# Exercise 5: Write a section header pattern that also catches
# lowercase headers like "findings:" and "Findings:"
# Hint: use re.IGNORECASE flagWhat do these exercises teach?
- Exercise 1 (stridor) teaches you to add a term with no morphological variants. Just append |stridor inside the alternation group.
- Exercise 2 (ronchi) teaches you to handle a common misspelling by broadening the existing pattern.
- Exercise 4 highlights why \b word boundaries are essential. Without them, your pattern matches inside compound words like "nonwheezing".
- Save regex_clinical_nlp.py and run the script one more time:
python regex_clinical_nlp.py- At the very end of the output, you should see the "PRACTICE EXERCISES" banner printed. The exercises themselves are comments for you to attempt on your own.
Seeing a SyntaxError?
- Check that the triple-quoted strings and parentheses in your extract_respiratory_findings function are all properly closed. A missing closing parenthesis on re.compile() will cause errors that point to the wrong line.
- Make sure the practice exercises section uses only comments (lines starting with #) for the exercise instructions. Any uncommented code there could conflict with the rest of the file.
Help me fix a SyntaxError in my regex script
✔️ Awesome, I've got everything!
Great. Double check you have saved your file.
ⓧ I'd like to double check the full code
Here is the complete regex_clinical_nlp.py file as it should look at the end of this step:
import re
# ============================================================
# SECTION 1: Word Boundaries and the Wheeze Pattern
# ============================================================
# CHARACTER-BY-CHARACTER BREAKDOWN of r"\bwheez(?:e|es|ing|ed)?\b"
#
# r"..." = raw string (tells Python: don't interpret backslashes)
# \b = word boundary (the invisible edge between a word and a space/punctuation)
# w = literal letter 'w'
# h = literal letter 'h'
# e = literal letter 'e'
# e = literal letter 'e'
# z = literal letter 'z'
# ( = start of a group
# ?: = makes this group NON-CAPTURING (groups without extracting)
# e = option 1: literal 'e' (makes "wheeze")
# | = OR
# es = option 2: literal 'es' (makes "wheezes")
# | = OR
# ing = option 3: literal 'ing' (makes "wheezing")
# | = OR
# ed = option 4: literal 'ed' (makes "wheezed")
# ) = end of group
# ? = the ENTIRE group is optional (zero or one time)
# \b = word boundary (end of word)
wheeze_pattern = re.compile(r"\bwheez(?:e|es|ing|ed)?\b", re.IGNORECASE)
# What it MATCHES:
print("=== MATCHES ===")
test_matches = ["wheez", "wheeze", "wheezes", "wheezing", "wheezed"]
for word in test_matches:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# What it does NOT match:
print("\n=== DOES NOT MATCH ===")
test_no_match = ["wheezy", "wheeziness", "nonwheezing"]
for word in test_no_match:
result = wheeze_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# WHY designed this way:
# - \b at start: prevents matching "wheez" inside "nonwheezing"
# - (?:...) group: bundles the suffix options together
# - ? after group: lets us match the bare stem "wheez" too
# - \b at end: prevents matching "wheeze" inside "wheezer"
# - re.IGNORECASE: clinical notes use inconsistent capitalization
# ============================================================
# SECTION 2: Quantifiers - Star and Question Mark
# ============================================================
# * (star) = zero or more of the previous thing
# ? (question mark) = zero or one of the previous thing (OPTIONAL)
# Star example: matching optional whitespace
# Clinical notes have inconsistent spacing: "RR: 18" vs "RR:18" vs "RR : 18"
spacing_pattern = re.compile(r"RR\s*:\s*\d+")
spacing_tests = ["RR:18", "RR: 18", "RR : 18", "RR : 22"]
print("\n=== STAR (*) for optional whitespace ===")
for test in spacing_tests:
result = spacing_pattern.search(test)
print(f" '{test}' -> {'MATCH' if result else 'NO MATCH'}")
# Question mark in the wheeze pattern:
# (?:e|es|ing|ed)? <-- the ? makes the ENTIRE group optional
# This means "wheez" alone (with no suffix) is a valid match
# --- Practice: crackle variants ---
# Pattern: \bcrackl(?:e|es|ing)?\b
crackle_pattern = re.compile(r"\bcrackl(?:e|es|ing)?\b", re.IGNORECASE)
print("\n=== Crackle pattern ===")
crackle_tests = ["crackle", "crackles", "crackling", "crackly", "crackled"]
for word in crackle_tests:
result = crackle_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# --- Practice: rhonchi/rhonchus ---
# Pattern: \brhonch(?:i|us)\b
rhonchi_pattern = re.compile(r"\brhonch(?:i|us)\b", re.IGNORECASE)
print("\n=== Rhonchi pattern ===")
rhonchi_tests = ["rhonchi", "rhonchus", "rhonchial"]
for word in rhonchi_tests:
result = rhonchi_pattern.search(word)
print(f" '{word}' -> {'MATCH' if result else 'NO MATCH'}")
# ============================================================
# SECTION 3: Anchors - Start and End of Line
# ============================================================
# ^ = start of string (or start of line with re.MULTILINE)
# $ = end of string (or end of line with re.MULTILINE)
# Without re.MULTILINE: ^ only matches the very start of the entire string
# With re.MULTILINE: ^ matches at the start of EACH line
# Section headers in clinical notes start at the beginning of a line
section_header_pattern = re.compile(r"^[A-Z ]+:", re.MULTILINE)
note = """FINDINGS: Bilateral wheezing noted.
IMPRESSION: Acute bronchospasm.
PLAN: Albuterol nebulizer treatment."""
headers = section_header_pattern.findall(note)
print("\n=== Section headers found with ^ ===")
print(f" {headers}")
# $ example: detecting if a line ends with a period (complete sentence)
complete_line = re.compile(r"\.$", re.MULTILINE)
endings = complete_line.findall(note)
print(f" Lines ending with period: {len(endings)}")
# ============================================================
# SECTION 4: Whitespace and Character Groups
# ============================================================
# \s = any whitespace character (space, tab, newline, carriage return)
# [abc] = match exactly ONE of: a, b, or c
# [A-Z] = match any single uppercase letter (range)
# [a-z] = match any single lowercase letter
# [0-9] = match any single digit
# | (pipe/alternation) = OR for entire expressions
# Why \s matters: clinical notes have messy spacing
# "FINDINGS:" vs "FINDINGS :" vs "FINDINGS:\n" vs "FINDINGS:\t"
header_with_spacing = re.compile(r"^[A-Z]+\s*:\s*", re.MULTILINE)
messy_note = """FINDINGS: Wheezing bilateral.
IMPRESSION :Acute asthma.
PLAN:
Start albuterol."""
print("\n=== \\s handles messy spacing ===")
matches = header_with_spacing.findall(messy_note)
for m in matches:
print(f" Found: '{repr(m)}'")
# Alternation (|) for multiple respiratory terms
# Match ANY respiratory finding in one pattern
resp_terms = re.compile(
r"\b(?:wheez(?:e|es|ing|ed)?|crackl(?:e|es|ing)?|rhonch(?:i|us)|rales?)\b",
re.IGNORECASE
)
print("\n=== Alternation (|) for multiple findings ===")
clinical_sentences = [
"Bilateral wheezing on auscultation",
"Fine crackles at the bases",
"No rhonchi appreciated",
"Scattered rales heard bilaterally",
"Lungs clear to auscultation",
]
for sentence in clinical_sentences:
findings = resp_terms.findall(sentence)
print(f" '{sentence}' -> {findings}")
# ============================================================
# SECTION 5: Groups - Capturing vs Non-Capturing
# ============================================================
# (pattern) = CAPTURING group - extracts the matched text
# (?:pattern) = NON-CAPTURING group - groups alternatives without extracting
# NON-CAPTURING in the wheeze pattern:
# r"\bwheez(?:e|es|ing|ed)?\b"
# The (?:...) groups the suffixes so ? applies to ALL of them
# But it does NOT extract/capture the suffix separately
# CAPTURING group: extract section name AND content
header_pattern = re.compile(r"^([A-Z ]+):\s*(.+)", re.MULTILINE)
note_text = """FINDINGS: Bilateral wheezing noted on exam.
IMPRESSION: Acute asthma exacerbation."""
print("\n=== Capturing groups extract data ===")
for match in header_pattern.finditer(note_text):
section_name = match.group(1) # first (capturing group)
section_content = match.group(2) # second (capturing group)
print(f" Section: '{section_name}' -> Content: '{section_content}'")
# Practice: capture the specific finding from clinical text
finding_capture = re.compile(
r"\b(wheez(?:e|es|ing|ed)?|crackl(?:e|es|ing)?|rhonch(?:i|us)|rales?)\b",
re.IGNORECASE
)
print("\n=== Capturing the finding itself ===")
test_sentences = [
"Bilateral wheezing noted on auscultation.",
"Fine crackles at both bases.",
"No rhonchi or rales appreciated.",
]
for sentence in test_sentences:
found = finding_capture.findall(sentence)
print(f" '{sentence}' -> captured: {found}")
# ============================================================
# SECTION 6: Section Detection Function
# ============================================================
def detect_sections(note):
"""Extract sections from a clinical note using regex.
Handles common section headers like FINDINGS:, IMPRESSION:,
HISTORY:, PLAN:, ASSESSMENT:
"""
section_pattern = re.compile(
r"(?:^|\n)\s*([A-Z][A-Z /]+):\s*\n?(.*?)(?=(?:\n\s*[A-Z][A-Z /]+:)|\Z)",
re.DOTALL
)
sections = {}
for match in section_pattern.finditer(note):
name = match.group(1).strip()
content = match.group(2).strip()
sections[name] = content
return sections
sample_note = """HISTORY: 45 year old male with history of asthma presents with
increased shortness of breath over the past 3 days.
FINDINGS: Bilateral expiratory wheezing. No crackles or rhonchi.
Respiratory rate 22. O2 sat 94% on room air.
IMPRESSION: Acute asthma exacerbation.
PLAN: Albuterol nebulizer q4h. Prednisone 40mg daily x 5 days.
Follow up in 48 hours."""
print("\n=== Section Detection ===")
sections = detect_sections(sample_note)
for name, content in sections.items():
print(f"\n --- {name} ---")
print(f" {content[:80]}...")
# ============================================================
# SECTION 7: Respiratory Finding Extraction
# ============================================================
def extract_respiratory_findings(text):
"""Extract respiratory findings from clinical text.
Returns list of dicts with 'finding' and 'raw_match' keys.
"""
pattern = re.compile(
r"\b(wheez(?:e|es|ing|ed)?|crackl(?:e|es|ing)?|rhonch(?:i|us)|rales?)\b",
re.IGNORECASE
)
results = []
for match in pattern.finditer(text):
finding = match.group(1)
results.append({
"finding": finding.lower(),
"raw_match": match.group(0).strip()
})
return results
print("\n=== Respiratory Finding Extraction ===")
test_texts = [
"Bilateral wheezing noted on auscultation.",
"No crackles or rhonchi appreciated.",
"Scattered rales heard at both bases.",
"Lungs clear, no wheezes, crackles, or rales.",
"Expiratory wheeze in right lower lobe.",
]
for text in test_texts:
findings = extract_respiratory_findings(text)
print(f"\n Text: '{text}'")
for f in findings:
print(f" -> {f['finding']} (raw: '{f['raw_match']}')")
if not findings:
print(" -> No respiratory findings detected")
# ============================================================
# PRACTICE EXERCISES
# ============================================================
print("\n" + "=" * 60)
print("PRACTICE EXERCISES")
print("=" * 60)
# Exercise 1: Add "stridor" to the finding pattern
# Hint: stridor has no common variants, just add it as |stridor
# Exercise 2: Add "ronchi" (common misspelling of rhonchi)
# Hint: modify the rhonch part to also catch "ronch"
# Exercise 3: Write a pattern for breath sounds:
# Should match: "diminished", "decreased", "absent", "clear"
# Hint: r"\b(?:diminished|decreased|absent|clear)\b"
# Exercise 4: The pattern below has a bug. What does it match incorrectly?
# buggy_pattern = re.compile(r"wheez(?:e|es|ing|ed)?")
# Hint: try it against "nonwheezing" - what happens without \b?
# Exercise 5: Write a section header pattern that also catches
# lowercase headers like "findings:" and "Findings:"
# Hint: use re.IGNORECASE flagSecret mission
Negation-Aware Regex
Your finding extractor catches "wheezing" in "No wheezing noted" just the same as "Bilateral wheezing on auscultation." In real clinical NLP pipelines, that's a critical failure. Can you build a regex system that distinguishes negated findings from present ones?
Clean Up Your Resources
Clean Up Your Resources
This project runs entirely locally using Python's built-in re module, so there are no ongoing costs or cloud resources to worry about.
Decide whether to keep your script for future reference, or delete it entirely.
Resources you used:
- regex_clinical_nlp.py (your local Python script).
Keep everything running
No action needed. Choose this if you plan to keep building on your regex patterns or want a reference script for future clinical NLP work.
- Your regex_clinical_nlp.py file stays wherever you saved it in VS Code.
- Python 3.13 remains installed on your machine for other projects.
Pause - I'll come back to this later
Nothing to pause. This project has no running processes or background services. Your file will be right where you left it when you come back.
Delete - I don't want to use this again
Remove the project file to start fresh.
- Delete regex_clinical_nlp.py by right-clicking the file tab in VS Code and selecting Close, then delete the file from Finder (macOS) or File Explorer (Windows).
Nice Work!
Nice Work!
You just built a clinical NLP regex toolkit from scratch using nothing but Python's built-in re module. You can now write, debug, and reason about regex patterns for medical text processing.
You learned how to:
- Break down r"\bwheez(?:e|es|ing|ed)?\b" character by character and write similar patterns for crackles, rhonchi, and rales using word boundaries, non-capturing groups, and alternation.
- Build a section detection function that splits multi-line clinical notes into named sections like FINDINGS, IMPRESSION, and PLAN using anchors, character classes, and the re.MULTILINE flag.
- Build a respiratory finding extractor combining \b, (?:), |, ?, and capturing groups into one pattern that handles all word forms.
- Build a negation-aware classifier that distinguishes "no wheezing" from "wheezing present" and correctly flags negated findings in clinical text.
Ready to quiz yourself?