Python AI/ML Coding Studio

Lesson 01 - Orientation

Write and run Python in a safe browser laboratory

The coding workbench combines an editor, standard-output console, variable inspector, virtual project files, plotting canvas, guided missions, and a purpose-built teaching interpreter. It supports core Python plus compact NumPy-, pandas-, Matplotlib-, and scikit-learn-style APIs for small educational AI/ML experiments.

Run a scriptInspect output and variablesUnderstand the supported subsetSave or export project work

1

What executes here

A deterministic educational runtime designed for small scripts

PY

Python foundations

Variables, expressions, strings, lists, dictionaries, sets, conditionals, loops, comprehensions, functions, and common built-ins.

DATA

Data analysis

Small numerical arrays, DataFrames, Series, CSV data, descriptive statistics, sorting, grouping, and transformation.

ML

Machine learning

Synthetic datasets, train/test splitting, feature scaling, linear and logistic regression, k-nearest neighbors, K-Means, metrics, and charts.

Important simulation boundary

This page does not launch the computer's Python installation, install packages, access the network, or execute arbitrary native code. It interprets a documented teaching subset inside the browser. Use an actual Python environment for production behavior, full language coverage, large datasets, or third-party packages.

2

First run

Use the supplied hello.py project file

01

Read the starting script

The editor contains variables, a list, an f-string, and arithmetic.

# Start here: edit this script and press Run
name = "student"
print(f"Hello, {name}!")
values = [12, 18, 21, 25, 29]
print("count:", len(values))
print("mean:", round(sum(values) / len(values), 2))

02

Run with Ctrl+Enter

The Run button and Ctrl+Enter execute the active .py file. Output and errors appear beside the editor.

print("Python workbench ready")

03

Inspect variables

After execution, open the Variables inspector to see names, types, dimensions, and previews.

course = "AI/ML"
values = [2, 4, 8, 16]
mean_value = sum(values) / len(values)
print(course, mean_value)

04

Change and rerun

Edit the name or values, save, and run again. Files and progress persist in this browser.

name = "researcher"
print(f"Welcome, {name}")

Knowledge check

Which statement best describes the runtime?

Lesson 02 - Syntax and types

Build expressions from values, names, and operators

Python uses indentation to express blocks and dynamic types to describe values at runtime. Clear names, small expressions, and explicit conversions make scripts easier to understand and test.

Assign variablesUse numeric and string typesFormat outputEvaluate comparisons and booleans

1

Common built-in types

Choose a representation that matches the data

Type	Example	Typical use
Integer	`count = 12`	Counts, indexes, identifiers
Float	`accuracy = 0.91`	Measurements and continuous values
String	`model = "baseline"`	Text and labels
Boolean	`approved = True`	Conditions and flags
None	`result = None`	No value or not-yet-computed state
List / dictionary	`[1, 2]` / `{"a": 1}`	Sequences and keyed records

2

Expression practice

Run each example and inspect the resulting variables

01

Numeric operators

Python distinguishes true division, floor division, remainder, and exponentiation.

total = 17
print(total / 5)
print(total // 5)
print(total % 5)
print(2 ** 8)

02

String methods and f-strings

Methods transform or test text; f-strings embed evaluated expressions.

system = "  Radar Node  "
clean = system.strip().lower()
print(f"normalized={clean}")

03

Conversions

Convert external text deliberately before numeric work.

raw = "42"
value = int(raw)
print(value + 8)

04

Comparisons and boolean logic

and, or, and not combine conditions.

accuracy = 0.89
latency_ms = 47
acceptable = accuracy >= 0.85 and latency_ms < 60
print("acceptable:", acceptable)

Indentation is syntax

Use four spaces consistently for each block level. Mixing block levels changes program structure and can cause an IndentationError.

Knowledge check

What does // calculate for positive integers?

Lesson 03 - Collections

Organize sequences, mappings, and unique values

Lists preserve order and are mutable, tuples conventionally represent fixed records, dictionaries map keys to values, and sets retain unique elements. Indexing, slicing, methods, and comprehensions make these structures practical.

Index and slice sequencesUpdate listsUse dictionary keys and valuesBuild comprehensions

1

Collection choices

Structure affects how data is accessed and changed

LIST

List

Ordered, mutable values. Useful for examples, predictions, measurements, and rows.

TUP

Tuple

Ordered values commonly treated as a fixed record or multiple return value.

DICT

Dictionary

Key-value mapping for named fields, configuration, and lookup tables.

SET

Set

Unique values for membership tests, deduplication, and category discovery.

2

Collection lab

Modify and derive data without hiding intent

01

Index and slice a list

Indexes begin at zero; negative indexes count from the end; slices stop before the final index.

scores = [72, 81, 89, 94, 78]
print(scores[0])
print(scores[-1])
print(scores[1:4])

02

Use list methods

append changes the existing list; sorted returns a new list.

scores = [72, 81, 89]
scores.append(94)
ordered = sorted(scores, reverse=True)
print(scores)
print(ordered)

03

Work with a dictionary

items returns key-value pairs suitable for tuple unpacking in a loop.

systems = {"gateway": "online", "database": "maintenance", "sensor": "online"}
for name, state in systems.items():
    print(name, state)

04

Build a comprehension

A comprehension expresses a transform and optional filter in one readable construct.

values = [2, 3, 4, 5, 6]
squares_of_even = [value ** 2 for value in values if value % 2 == 0]
print(squares_of_even)

05

Deduplicate with a set

Sets are useful when order is less important than uniqueness.

labels = ["cat", "dog", "cat", "bird", "dog"]
unique_labels = set(labels)
print(unique_labels)

Knowledge check

Which collection maps unique keys to associated values?

Lesson 04 - Control flow

Make decisions and repeat work deliberately

Conditionals select a path based on data. for loops iterate over a finite collection; while loops continue while a condition remains true. Loop variables, termination conditions, and readable nesting are essential to correct behavior.

Use if, elif, and elseIterate with forUse range and enumerateControl loop termination

1

Decision structure

Evaluate the most specific conditions before broader defaults

conditionalaccuracy = 0.91
if accuracy >= 0.90:
    rating = "excellent"
elif accuracy >= 0.80:
    rating = "acceptable"
else:
    rating = "review"
print(rating)

2

Loop practice

Use the loop form that best communicates the boundary

01

Iterate over values

A for loop assigns each collection item to the loop variable.

models = ["baseline", "tree", "linear"]
for model in models:
    print("evaluating", model)

02

Track positions with enumerate

enumerate produces index-value pairs without manual counters.

scores = [0.81, 0.87, 0.91]
for index, score in enumerate(scores, start=1):
    print(index, score)

03

Pair sequences with zip

zip aligns values by position and stops at the shortest input.

names = ["A", "B", "C"]
latencies = [42, 57, 38]
for name, latency in zip(names, latencies):
    print(name, latency)

04

Use a bounded while loop

Update the variable that controls termination.

epoch = 1
loss = 1.0
while epoch <= 5:
    loss = loss * 0.7
    print(epoch, round(loss, 3))
    epoch += 1

05

Use break and continue

continue skips the remainder of one iteration; break exits the loop.

values = [3, -1, 8, 0, 14]
for value in values:
    if value < 0:
        continue
    if value == 0:
        break
    print(value)

Protect against accidental infinite loops

The teaching runtime enforces a maximum execution-step count. A real Python environment does not automatically repair a missing loop update or incorrect termination condition.

Knowledge check

Which statement skips directly to the next loop iteration?

Lesson 05 - Functions

Package logic behind a clear interface

Functions reduce repetition, isolate decisions, receive inputs through parameters, and communicate results with return values. Small functions are easier to test, compose, and explain than one long script.

Define and call functionsPass parametersReturn valuesUnderstand local scope

1

Function anatomy

A name, parameters, body, and return contract

functiondef mean_value(values):
    if len(values) == 0:
        return None
    return sum(values) / len(values)

result = mean_value([4, 8, 15, 16, 23, 42])
print(result)

2

Guided function lab

Build from a scalar return to a structured summary

01

Define a reusable calculation

Parameters receive values from the caller.

def celsius_to_fahrenheit(celsius):
    return celsius * 9 / 5 + 32

print(celsius_to_fahrenheit(20))

02

Use a default parameter

Defaults make an argument optional while preserving an explicit override.

def classify(score, threshold=0.8):
    if score >= threshold:
        return "pass"
    return "review"

print(classify(0.86))
print(classify(0.86, threshold=0.9))

03

Return several related fields

A dictionary gives names to each returned statistic.

def summarize(values):
    return {"count": len(values), "mean": sum(values) / len(values), "minimum": min(values), "maximum": max(values)}

summary = summarize([81, 89, 94, 78, 91])
print(summary)

04

Use local variables

Names assigned inside a function belong to that function call unless intentionally returned.

def squared_error(actual, predicted):
    difference = actual - predicted
    return difference ** 2

error = squared_error(10, 8.5)
print(error)

Knowledge check

What is the purpose of return?

Lesson 06 - Files

Move data between scripts and persistent project files

Files let one execution produce input for another. Use a context manager to close a file, distinguish read, write, and append modes, and validate the structure of CSV or JSON data before analysis.

Read and write textUse context managersLoad CSV dataPreserve reproducible inputs

1

Virtual project storage

Files persist in this page but never touch the host filesystem

Mode	Meaning	Effect in this lab
`r`	Read	Requires an existing project file
`w`	Write	Creates or replaces the file before writing
`a`	Append	Adds text after existing content

2

File and CSV lab

Create evidence, read it back, and analyze a table

01

Write a text report

with ensures the virtual file is closed after the block.

with open("run_report.txt", "w") as report:
    report.write("model=baseline\n")
    report.write("accuracy=0.84\n")

print("report written")

02

Read the report

read returns the complete file contents.

with open("run_report.txt", "r") as report:
    contents = report.read()
print(contents)

03

Load the included CSV

The pandas-style read_csv function converts rows and columns into a DataFrame.

import pandas as pd

df = pd.read_csv("experiments.csv")
print(df)
print(df.describe())

04

Export a DataFrame as CSV text

to_csv returns text in the teaching API; write it to another project file.

import pandas as pd

df = pd.read_csv("experiments.csv")
exported = df.to_csv(index=False)
with open("experiments_copy.csv", "w") as output:
    output.write(exported)
print("rows:", df.shape[0])

Data is part of the program

Record where data came from, what each column means, what cleaning was applied, and which rows belong to training, validation, or test sets. Reproducibility requires more than saving source code.

Knowledge check

What does opening a file with mode w do?

Lesson 07 - Numerical arrays

Use vector-shaped data for scientific calculations

NumPy arrays provide compact numerical operations in real Python. This lab includes a small compatible teaching surface for array construction, ranges, reshaping, summary statistics, random samples, and dot products.

Create numerical arraysInspect shapeReshape valuesCalculate statistics and dot products

1

Array dimensions

Shape communicates the number of rows, columns, and higher dimensions

1D

Vector

A one-dimensional sequence such as five measurements has shape [5] in the inspector.

2D

Matrix

Rows by columns, such as 100 examples by four features.

DOT

Dot product

Combines corresponding values and is fundamental to linear models.

RNG

Deterministic random data

A fixed seed makes generated teaching examples reproducible.

2

NumPy-style practice

Run small operations and inspect dimensions

01

Create and summarize an array

The included API recognizes the familiar numpy alias.

import numpy as np

values = np.array([4, 8, 15, 16, 23, 42])
print("mean:", np.mean(values))
print("median:", np.median(values))
print("std:", round(np.std(values), 3))

02

Create a numerical range

arange excludes the stop value, like Python range.

import numpy as np

values = np.arange(0, 12, 2)
print(values)
print("shape:", values.shape)

03

Reshape a sequence

The total element count must remain unchanged.

import numpy as np

values = np.arange(1, 13)
matrix = np.reshape(values, [3, 4])
print(matrix)
print("shape:", matrix.shape)

04

Calculate a dot product

The result is the sum of element-wise products.

import numpy as np

weights = [0.4, -0.2, 0.7]
features = [3.0, 1.5, 2.0]
score = np.dot(weights, features)
print("linear score:", score)

05

Generate repeatable samples

Seed the random generator before creating values.

import numpy as np

np.random.seed(17)
samples = np.random.normal(0, 1, 8)
print(samples)

Knowledge check

Can an array of 12 elements be reshaped into 5 rows by 3 columns?

Lesson 08 - DataFrames

Analyze named columns and tabular records

A DataFrame represents rows and named columns. Column selection, descriptive statistics, sorting, filtering, grouping, and deliberate missing-data treatment form the foundation of many data-science workflows.

Construct a DataFrameSelect columnsDescribe numeric dataSort and group records

1

Tabular mental model

Rows are observations; columns are variables

Operation	Teaching API	Question answered
Inspect dimensions	`df.shape`	How many rows and columns?
Preview rows	`df.head()`	What does the table look like?
Select a column	`df["accuracy"]`	What are the values for one variable?
Summarize	`df.describe()`	What are count, center, spread, and range?
Sort	`df.sort_values("accuracy")`	Which rows rank highest or lowest?
Group	`df.groupby("group").agg(...)`	How do summaries differ by category?

2

DataFrame lab

Create, inspect, transform, and summarize

01

Create a DataFrame

A dictionary of equal-length columns is a convenient constructor.

import pandas as pd

data = {"model": ["A", "B", "C"], "accuracy": [0.82, 0.88, 0.86], "latency_ms": [42, 57, 38]}
df = pd.DataFrame(data)
print(df)
print("shape:", df.shape)

02

Select and summarize a column

A selected column behaves like a Series.

import pandas as pd

df = pd.read_csv("experiments.csv")
accuracy = df["accuracy"]
print("mean:", round(accuracy.mean(), 3))
print("maximum:", accuracy.max())

03

Sort rows

Sort by a metric and choose ascending or descending order.

import pandas as pd

df = pd.read_csv("experiments.csv")
ranked = df.sort_values("accuracy", ascending=False)
print(ranked)

04

Add a derived column

Assigning a list creates or replaces a column.

import pandas as pd

df = pd.read_csv("experiments.csv")
df["accuracy_pct"] = [round(value * 100, 1) for value in df["accuracy"]]
print(df)

05

Describe all numeric columns

describe produces one summary row per numeric column in this teaching API.

import pandas as pd

df = pd.read_csv("experiments.csv")
summary = df.describe()
print(summary)

Knowledge check

What does df["accuracy"] select?

Lesson 09 - ML workflow

Separate problem definition, training, evaluation, and deployment

A machine-learning script is only one part of an evidence chain. Define the target and unit of observation, establish a baseline, separate data before learning transformations, train on one partition, evaluate on held-out examples, document limitations, and monitor post-deployment behavior.

Define features and targetSplit before fittingPrevent leakageRecord reproducible evidence

1

A disciplined experiment

Each stage should have a clear input and output

Questiontarget, population, cost

→

Data splittrain and held-out test

→

Fit + evaluatemetrics, errors, limitations

2

Workflow safeguards

Avoid common reasons a promising experiment fails in use

BASE

Baseline

Compare the model with a simple rule, class frequency, or historical process.

SPLIT

Held-out data

Do not use test labels to choose features, transformations, hyperparameters, or thresholds.

LEAK

Leakage control

Fit scalers, imputers, encoders, and feature selection only on training data.

SEED

Reproducibility

Record random seeds, data versions, code, parameters, and environment details.

ERR

Error analysis

Inspect false positives, false negatives, large residuals, and subgroup performance.

OPS

Monitoring

Define drift, performance, safety, and rollback indicators before deployment.

3

Train/test pattern

Fit transformations and models on training rows only

workflow patternX_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42
)

scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

model.fit(X_train, y_train)
predictions = model.predict(X_test)

Knowledge check

Why should the scaler be fit on X_train rather than all rows?

Lesson 10 - Classification

Predict categories and examine the kinds of mistakes

Classification maps features to discrete labels. Accuracy is useful when errors and class frequencies are balanced, but a confusion matrix exposes which actual categories become which predicted categories.

Generate labeled dataScale featuresFit logistic regressionRead a confusion matrix

1

Classification evidence

One score rarely describes every consequence

Metric or artifact	Meaning	Caution
Accuracy	Fraction of predictions that are correct	Can hide minority-class failure
Confusion matrix	Counts by actual and predicted label	Requires careful axis interpretation
Precision	How often a positive prediction is correct	Depends on threshold and prevalence
Recall	How many actual positives are found	May trade off against precision
Probability / score	Model confidence-like output	May require calibration before operational use

2

Guided classification lab

Use the supplied deterministic binary dataset

01

Generate and split data

stratify requests a representative label mix in each partition in the teaching splitter.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split

X, y = make_classification(n_samples=120, n_features=4, class_sep=1.6, random_state=12)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42, stratify=y)
print(len(y_train), len(y_test))

02

Scale training and test data correctly

Fit the scaler on training values, then reuse it for test values.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

X, y = make_classification(n_samples=120, n_features=4, class_sep=1.6, random_state=12)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42, stratify=y)
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
print("training means:", scaler.mean_)

03

Train and predict

The lab implements a compact gradient-descent logistic model.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

X, y = make_classification(n_samples=120, n_features=4, class_sep=1.6, random_state=12)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42, stratify=y)
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
model = LogisticRegression(learning_rate=0.2, epochs=500)
model.fit(X_train, y_train)
predictions = model.predict(X_test)
print(predictions)

04

Evaluate outcomes

Use accuracy together with the confusion matrix.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, confusion_matrix

X, y = make_classification(n_samples=120, n_features=4, class_sep=1.6, random_state=12)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42, stratify=y)
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
model = LogisticRegression(learning_rate=0.2, epochs=500)
model.fit(X_train, y_train)
predictions = model.predict(X_test)
print("accuracy:", round(accuracy_score(y_test, predictions), 3))
print(confusion_matrix(y_test, predictions))

Operational thresholds are policy choices

A mathematically higher score is not automatically a better operational decision. Costs, safety, legal requirements, class prevalence, human review, and acceptable error tradeoffs matter.

Knowledge check

What additional information does a confusion matrix provide beyond accuracy?

Lesson 11 - Regression

Predict continuous values and measure residual error

Regression estimates a numerical target. Linear regression learns coefficients and an intercept; residual metrics quantify prediction differences, while R-squared compares explained variation with a mean-only reference.

Generate regression dataFit linear coefficientsPredict held-out targetsInterpret MSE, MAE, and R2

1

Regression metrics

Different summaries emphasize different error properties

Metric	Calculation idea	Interpretation
MAE	Mean absolute residual	Average error magnitude in target units
MSE	Mean squared residual	Penalizes larger errors more strongly
RMSE	Square root of MSE	Returns to target units while retaining squared-error emphasis
R2	Relative reduction in squared error	1 is perfect; 0 resembles predicting the mean; negative can be worse

2

Linear-regression lab

Fit a two-feature model and inspect learned parameters

01

Generate a repeatable dataset

Noise controls how tightly the target follows the linear signal.

from sklearn.datasets import make_regression

X, y = make_regression(n_samples=80, n_features=2, noise=8, random_state=7)
print("rows:", len(X))
print("features:", len(X[0]))

02

Reserve test rows

A held-out partition estimates behavior on unseen examples from the same generator.

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split

X, y = make_regression(n_samples=80, n_features=2, noise=8, random_state=7)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)
print(len(y_train), len(y_test))

03

Fit and predict

The model solves a regularized normal equation for this small teaching dataset.

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

X, y = make_regression(n_samples=80, n_features=2, noise=8, random_state=7)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)
model = LinearRegression()
model.fit(X_train, y_train)
predictions = model.predict(X_test)
print("coefficients:", model.coef_)
print("intercept:", round(model.intercept_, 3))

04

Calculate complementary metrics

Compare absolute, squared, and variance-relative views.

from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

X, y = make_regression(n_samples=80, n_features=2, noise=8, random_state=7)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)
model = LinearRegression()
model.fit(X_train, y_train)
predictions = model.predict(X_test)
print("MAE:", round(mean_absolute_error(y_test, predictions), 3))
print("MSE:", round(mean_squared_error(y_test, predictions), 3))
print("R2:", round(r2_score(y_test, predictions), 3))

Knowledge check

Why can MSE be more sensitive to a large residual than MAE?

Lesson 12 - Clustering

Discover groups without supplied target labels

Clustering is unsupervised: the algorithm organizes examples by feature similarity rather than learning known labels. K-Means alternates between assigning points to the nearest center and updating centers from assigned points.

Generate cluster dataChoose a cluster countFit K-MeansAssess and visualize the result

1

K-Means iteration

Assignment and center update repeat until stable or limited

Initialize centersseed controls starting choices

→

Assign pointsnearest center

↔

Update centersmean of assigned points

2

Clustering and plot lab

Create three compact groups and examine the labels

01

Generate blob data

The example provides true labels only for comparison; K-Means does not use them.

from sklearn.datasets import make_blobs

X, true_labels = make_blobs(n_samples=90, centers=3, cluster_std=0.65, random_state=5)
print("rows:", len(X))

02

Fit K-Means

fit_predict returns the assigned cluster index for each row.

from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans

X, true_labels = make_blobs(n_samples=90, centers=3, cluster_std=0.65, random_state=5)
model = KMeans(n_clusters=3, random_state=5, max_iter=100)
labels = model.fit_predict(X)
print("centers:", model.cluster_centers_)
print("inertia:", round(model.inertia_, 3))

03

Calculate silhouette score

Values closer to 1 generally indicate compact and separated clusters for the chosen representation.

from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

X, true_labels = make_blobs(n_samples=90, centers=3, cluster_std=0.65, random_state=5)
model = KMeans(n_clusters=3, random_state=5, max_iter=100)
labels = model.fit_predict(X)
score = silhouette_score(X, labels)
print("silhouette:", round(score, 3))

04

Render points and centers

The browser plot uses labels to distinguish point groups and x markers for centers.

from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt

X, true_labels = make_blobs(n_samples=90, centers=3, cluster_std=0.65, random_state=5)
model = KMeans(n_clusters=3, random_state=5, max_iter=100)
labels = model.fit_predict(X)
plt.scatter([row[0] for row in X], [row[1] for row in X], c=labels)
plt.scatter([row[0] for row in model.cluster_centers_], [row[1] for row in model.cluster_centers_], marker="x", s=120)
plt.title("K-Means Teaching Example")
plt.xlabel("feature 1")
plt.ylabel("feature 2")
plt.show()

Clusters are not ground truth

K-Means will produce groups even when the chosen features, distance metric, scale, or cluster count do not represent a meaningful real-world distinction. Interpret clusters using domain evidence.

Knowledge check

Does K-Means require known target labels during fitting?

Lesson 13 - Evaluation

Test more than headline performance

Reliable evaluation asks whether a model generalizes, where it fails, whether important subgroups are treated acceptably, whether the data represents intended use, and whether post-deployment monitoring can detect deterioration.

Recognize overfittingCompare training and validation behaviorAnalyze errors and subgroupsDocument limitations and monitoring

1

Threats to trustworthy evaluation

High test performance can still be misleading

LEAK

Data leakage

Information unavailable at prediction time or derived from the test set contaminates training.

SHIFT

Distribution shift

Future or operational data differs from the development sample.

IMB

Class imbalance

Aggregate accuracy can obscure weak minority-class recall.

DUP

Near duplicates

Related rows split across train and test can inflate apparent generalization.

BIAS

Representation gaps

Some populations, environments, sensors, or conditions may be underrepresented.

DRIFT

Performance drift

Inputs, labels, processes, adversaries, or user behavior change after release.

2

Learning-curve visualization

Compare training and validation trends rather than one final number

01

Plot two loss sequences

A widening validation gap can be evidence of overfitting, although context and uncertainty matter.

import matplotlib.pyplot as plt

epochs = [1, 2, 3, 4, 5, 6]
train_loss = [0.92, 0.66, 0.48, 0.37, 0.31, 0.28]
validation_loss = [0.97, 0.72, 0.55, 0.46, 0.43, 0.45]
plt.plot(epochs, train_loss, label="train")
plt.plot(epochs, validation_loss, label="validation")
plt.title("Learning Curves")
plt.xlabel("epoch")
plt.ylabel("loss")
plt.legend()
plt.grid(True)
plt.show()

02

Create a simple error audit

Preserve actual, predicted, and context fields for review.

actual = [1, 0, 1, 1, 0, 0]
predicted = [1, 1, 1, 0, 0, 0]
errors = []
for index, pair in enumerate(zip(actual, predicted)):
    a, p = pair
    if a != p:
        errors.append({"index": index, "actual": a, "predicted": p})
print(errors)

3

Model-card questions

Document intended use before operational handoff

Question	Evidence to record
Purpose	Decision supported, user, population, and excluded uses
Data	Source, time period, sampling, labels, consent or authority, and transformations
Performance	Metrics, uncertainty, baselines, subgroups, conditions, and thresholds
Limitations	Known failure modes, blind spots, assumptions, and misuse risks
Operations	Monitoring, feedback, human review, incident response, and rollback triggers

Knowledge check

What pattern most strongly suggests overfitting in the displayed learning curves?

Lesson 14 - Capstone

Complete and document a small classification experiment

The capstone integrates data generation, held-out evaluation, leakage-aware scaling, model training, predictions, metrics, visualization, and model-card documentation. Treat the script output as evidence to explain—not merely a score to report.

Run an end-to-end experimentVerify held-out evaluationVisualize outcomesComplete a model card

1

Capstone requirements

Use capstone_ai.py and model_card.md

Operational Classification Study

Prepare a reproducible browser-based experiment for a technical review.

Open and inspect capstone_ai.py.
Verify that a fixed random seed is used for data generation and splitting.
Verify that scaling is fit on training data and only applied to test data afterward.
Train logistic regression and produce held-out predictions.
Report the number of test rows, accuracy, and a confusion matrix.
Render the correct-versus-incorrect result chart.
Change one experimental parameter, rerun, and compare the evidence.
Complete Purpose, Data source, Target definition, Known limitations, and Monitoring and rollback plan in model_card.md.

2

Suggested verification

Evidence should match the source and the output

CODE

Source review

Confirm split, scaler, model, metric, and plotting calls before running.

OUT

Execution evidence

Save the transcript containing test size, accuracy, confusion matrix, and errors if any.

PLOT

Visualization

Use the chart as one summary, not as a substitute for the confusion matrix or limitations.

CARD

Model card

State intended use, data limitations, failure risks, monitoring, and rollback criteria.

3

Transition to a full environment

Carry the reasoning into standard Python tools

Python Tutorial

Official introduction to Python syntax, control flow, data structures, modules, files, errors, classes, and the standard library.

Language

Python Library Reference

Authoritative reference for built-in functions, types, modules, exceptions, file operations, and standard-library behavior.

Reference

NumPy User Guide

Official guides for multidimensional arrays, indexing, vectorized operations, numerical routines, and interoperability.

Numerical computing

pandas User Guide

Official guidance for Series, DataFrames, indexing, missing data, grouping, merging, time series, and input/output.

Data analysis

Matplotlib Tutorials

Official tutorials for figures, axes, plotting functions, annotations, styling, and visualization patterns.

Visualization

scikit-learn User Guide

Official machine-learning guidance for preprocessing, supervised and unsupervised methods, model selection, metrics, inspection, and common pitfalls.

Machine learning

Before using a real AI/ML environment

Review dependencies and data rights, use isolated environments, pin and scan packages, protect sensitive data, validate resource requirements, test on representative held-out data, document intended use, and plan monitoring and rollback.

Knowledge check

Which capstone sequence is most defensible?