Course project for CS5330 (Pattern Recognition and Computer Vision, Prof. Bruce Maxwell) with Ananda Sangli and Julee Chung. We built a class-agnostic obstacle detection pipeline for autonomous surface vessels — one that treats every floating obstacle as “hit-or-not-hit” rather than trying to classify it, because on the water what you need to avoid isn’t neatly categorizable.
RQ1: Does synthetic IMU-based horizon stabilization improve detection on rocking footage?
Boats pitch and roll. Detectors trained on stable dashcam data assume a level horizon, and every degree of roll shifts objects relative to the frame. We synthesized IMU readings from video motion and applied rotational homography to stabilize each frame before detection — asking whether removing the roll before inference beats letting the detector figure it out.
RQ2: Does Kalman-filter tracking reduce bounding-box jitter and identity flicker across frames?
Frame-by-frame detection produces boxes that shimmer and swap IDs across consecutive frames — bad for downstream avoidance logic. We ran detections through a Kalman filter that models each obstacle’s position and velocity, smoothing box coordinates and maintaining stable identity across occlusions.
Dataset. Unified benchmark from USVTrack (tracking annotations) and LaRS (semantic segmentation for the maritime domain). Combining them gave us both the class-agnostic labels we wanted and the domain variance (open water, harbors, obstacles at different scales) needed for a generalizable detector.
Detector. YOLOv8n — nano variant chosen for real-time inference budgets on the kind of hardware an actual surface vessel would carry. Not the largest model available, deliberately.
RQ1 pipeline.
RQ2 pipeline.
Full result videos across all test scenarios — including RQ1 horizon stabilization comparisons and RQ2 Kalman-filtered tracking sequences — are available in the project results folder on Google Drive.
A multi-class brain tumor classifier deployed on Arxelos, comparing a 30-layer CNN I built from scratch against ViT-B/16 on the same MRI scans. Every prediction ships with interpretability maps.
A pituitary tumor scan. The CNN classified it as “no tumor” at 97% confidence. The ViT correctly identified it as pituitary at 50.3% confidence.
That confidence asymmetry is the actual finding, not the misclassification. The CNN wasn’t uncertain and wrong — it was confidently wrong, which is the worst kind of failure mode for a clinical model. The ViT wasn’t confidently right — it was hedging but correct, which is honest calibration in the face of an ambiguous case.
To make the interpretability comparison honest, I ran the same tooling on a “no tumor” scan both models correctly classified.
CNN Grad-CAM was broad and warm, centered on ventricles, thalami, and basal ganglia — reasonable attention to normal anatomical landmarks. When the CNN is right, it’s at least looking in the right neighborhood.
ViT Attention Rollout was tightly focused on bilateral hotspots corresponding to choroid plexus / thalamic structures — the ViT was using anatomical symmetry as evidence of normalcy. A more sophisticated reasoning strategy than the CNN’s “everything looks normal-shaped.”
google/vit-base-patch16-224pytorch-grad-cam), Attention Rollout (custom implementation), SmoothGradConsensus confidence. The 97% wrong CNN vs. 50% correct ViT pattern is the argument for a joint metric that penalizes disagreement rather than displaying either raw softmax. Multiplying the two distributions and renormalizing is the naive version; a properly calibrated ensemble is the real move. Either way, showing 97% confidence on the CNN’s misclassification without context is a UI failure I want to fix.
Sharper Attention Rollout. My current implementation averages heads uniformly. The Chefer et al. (2021) method that combines gradients with attention would produce cleaner localization, particularly for smaller lesions where uniform-head averaging blurs the signal.
Annotation audit. In the pituitary failure case, the ViT’s attention on the parietal region (not the sellar area where pituitary tumors actually sit) suggests the ground-truth label may itself be wrong on that scan. Before extending the training set, I’d want to review the annotations on the ambiguous cases with someone who reads MRIs for a living.
A self-driving perception stack in miniature. Three complementary models running on the same BDD100K dashcam frame — RT-DETR tells you what is in the scene, a from-scratch U-Net tells you where you can drive, MiDaS tells you how far everything is. Each handles a different piece of the scene-understanding problem; together they form a full spatial picture from a single frame.
Deployed live at arxelos.com/perception.
When I first started training RT-DETR on BDD100K, my data loader reported only 1,156 training images. The dataset should have had ~70,000. That’s a factor of 60 off.
First instinct was a wrong path or incomplete download, but du -sh on the directory checked out — the bytes were there. Ran two globs side by side:
glob.glob(f"{IMG_TRAIN}/*.jpg") # → 1,156
glob.glob(f"{IMG_TRAIN}/**/*.jpg", recursive=True) # → ~70,000
The images existed, just not where my code was looking. BDD100K’s zip stores training images across four subdirectories — trainA/, trainB/, testA/, testB/ — inside images/100k/train/. A non-recursive glob only caught what was sitting in the top-level directory.
But the recursive glob alone wasn’t enough. Ultralytics (the RT-DETR training framework) uses a path-swapping convention where it replaces images/ with labels/ to find annotation files. With images nested in subdirectories but labels stored flat, the path swap would silently break — training would run but on wrong image/label pairs.
Fix required two changes: switch to recursive=True globs everywhere, and flatten all subdirectory images into the top-level train/ and val/ directories with shutil.move(). That restored the one-to-one images/ ↔ labels/ mapping Ultralytics expects.
Takeaway. The code fix was ~10 lines. The lesson was bigger: never trust assumptions about dataset layout, and never trust that a training framework’s file-path conventions are documented where you’d expect. I now run a recursive file count as the first step after downloading any dataset, before writing a line of training code. Cheap insurance against silent training failures — the kind where loss goes down and metrics look reasonable, but the model is learning on the wrong data.
Arxelos is the platform I built to deploy my AI/ML work under one roof — not scattered notebooks.
Building one platform forced decisions individual notebooks avoid:
pytorch-grad-cam, custom Attention Rollout, SmoothGradSports video games (EA FC, NBA 2K, Madden) auto-generate faces for scouted international players — badly. Same faces recur, ethnic demographics don’t match countries, and detail is allocated unevenly across groups. Four-person course project exploring whether a conditional GAN could produce a fairer baseline for auto-generated player faces.
No significant ethnic bias detected at the model’s capability level. Trained on FairFace (62,575 images filtered to ages 20–49, resized to 128×128), the cGAN produced faces of comparable quality across all 7 ethnic categories at the optimal checkpoint (epoch 33 of 50):
The important word is comparable. Absolute quality was limited — an FID around 150 means generated faces are far from photorealistic, and the perceptual survey confirmed the ceiling. But the distribution of quality across ethnicities was flat, which was the fairness question being tested.
Owned hyperparameter tuning, model training, and optimal checkpoint selection across two full training runs on a Colab A100. Trade-offs I was navigating:
Teammates handled evaluation methodology (Caleb Lee), data pipeline (Gaurav Bhatnagar) and UI (Yohanan Ben-Gad).
The model at ~100K parameters is undersized for the problem — that’s the ceiling on realism, and it’s why FID sits around 150. A larger backbone (StyleGAN2 at even a modest scale) would likely bring FID into the 30–50 range. The interesting piece — per-ethnicity variance under 3% is the methodology that carries forward, and it should hold or improve at higher model capacity.
A structured self-study challenge that walks ML end-to-end, from NumPy fundamentals to transformers and GANs. Three phases, capstone projects between them, so each phase forces a synthesis before moving on.
46/100 days complete. Phase 1 and Phase 2 done, Phase 3 in progress.
The capstones between phases are the point. It’s easy to work through tutorials and feel like you understand something; it’s harder to build an end-to-end pipeline that has to run. Requiring a synthesis project before moving on makes gaps visible — feature engineering isn’t a concept until I’ve built a pipeline for it, SHAP isn’t understood until I’ve explained it on my own model.
Rain prediction (Day 15). Binary classification on Australian weather. Missing data handling → encoding → feature scaling → model selection.
Loan default prediction (Days 38–40). End-to-end pipeline on Lending Club data: EDA → feature engineering → hyperparameter tuning → SHAP interpretability → production pipeline. Standalone repo.
Undergraduate research paper co-authored with Rohit Chauhan and Suryajeet Gupta at Quantum University. We benchmarked three CNN variants — a custom sequential model, a custom functional (multi-branch) model, and a VGG-16 transfer-learning setup across chest X-ray and CT-scan classification for three conditions.
Sequential model outperformed on X-ray tasks; the functional model (7×7 and 1×1 branches concatenated, feeding five 3×3 conv layers) generalized better on CT cancer classification.
The reported accuracies are strong but the test sets are small (85 images for TB, 278 for cancer) and the training data leaves external validity unproven. A properly held-out external cohort, cross-validation, and interpretability checks would be table stakes today — themes that motivated my later work.