Repository navigation
Tags: mobilint/mblt-vision-python
Tags
Remove SAM2 mask generation and SA-V support (#23) * Remove SAM2 mask generation and SA-V support Move SAM2HieraLarge promptable mask generation off main; the full implementation is preserved on the jm/sam branch. Removed: - mblt_vision.mask_generation (SAM2HieraLarge, prompt encoder port, graph contracts) and models/SAM2HieraLarge.yaml - the mask_generation task from VISION_TASKS and the top-level exports - SA-V dataset support: datasets/sa-v.yaml, CustomSAV, organize_sav, readiness checks, eval_sav, and benchmark/organize_sav.py - CLI point prompts, two-artifact path overrides, and SA-V val options - the mask_generation branch of Results - SAM/SA-V tests and documentation in the READMEs, AGENTS.md, and the mblt-vision skill Kept, as generic improvements that landed with the SAM work: wrapper.download_hub_artifact, _find_existing_source never returning the organized cache, the letterbox/postprocess fixes, MANIFEST.in, and the sample assets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore: set version to 0.0.5 0.0.5 was never released, so the SAM removal ships as 0.0.5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Restore NYU evaluation rule and document the SAM2 removal The SAM cleanup cut the NYU Depth ranking rule out of AGENTS.md along with the adjacent SA-V bullets; restore it. Record the removal of the mask_generation surface (shipped in 0.0.3/0.0.4) as a breaking change in the Vision README. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
perf: cut postprocess and scoring cost (0.0.4), and floor the timm re… …size (#9) * fix: derive the timm resize the way timm derives it timm's transforms_factory sizes the pre-crop resize as int(math.floor(input_size / crop_pct)), and these 35 classifier configurations carried a rounded value: 249 instead of 248 at 224/0.9, 236 instead of 235 at 224/0.95. The one-pixel difference shifts every interpolated pixel, so the tensors differed from timm's own evaluation pipeline by up to 3.5 in normalized units. The values come from the matching mblt-model-ops model, whose source.yaml proves the checkpoint is a timm one. Reading them from timm by configuration name would be wrong here: several names (ConvNext_Base) resolve in timm yet carry torchvision's transform in this package, and ConvNext_Base's 232 is torchvision's, not a rounding mistake. All 77 classifier variants shared with mblt-model-ops now agree with it on Reader, Resize, CenterCrop, SetOrder and Normalize. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * perf: cut postprocess and scoring cost, and release it as 0.0.4 The same review that flagged slow score computation in mblt-model-ops applies here: these evaluators and postprocessors are the counterparts of that repository's `datasets/*/evaluator.py`, and nothing compares the two automatically. Each rewrite below is checked output-identical against the implementation it replaces on randomized inputs, degenerate cases included. `rotated_nms` built one `count x count` pairwise probIoU matrix. At the candidate caps a dense low-confidence OBB head reaches that matrix alone is 3.4 GiB in float32 and probIoU builds about ten intermediates of the same size, so 30000 candidates needed tens of gigabytes and did not run at all. It now tiles the pairwise IoU in `ROTATED_NMS_BLOCK`-wide blocks and drops a column as soon as any higher-scored candidate suppresses it: a candidate needs only one such overlap to fall, so a suppressed column cannot change its verdict and skipping it is exact. The `row >= column` mask on each block's own triangle is what the `triu_(diagonal=1)` of the single-matrix form did. 8000 candidates go 1.64 s -> 0.54 s, and 30000 now run in 5.7 s spread or 2.0 s clustered. `to_string` variable-length-encoded RLE counts one Python loop per count, which is 724 ms for a 300-mask image — more than the mask rasterisation it follows. It now encodes every count in lockstep, tracking which counts still have 5-bit chunks left, and extracts row-major so each count's chunks stay together in the order the per-count loop produced. 666 ms -> 130 ms over 300 masks, taking `_encode_segmentation_masks` from 970 ms to 225 ms. `eval_widerface` scored each image three times, once per difficulty setting, and walked 1000 score cut-offs and every prediction in Python. Which faces a setting ignores is the only per-setting input, so `image_matches` computes the IoU matrix and each prediction's best match once and all three settings reuse it; `matched_image_eval` vectorises the greedy loop, because a prediction only leaves the proposal list when it matched an ignored face and a face is recalled once at the first prediction that matched it; and `score_threshold_index` replaces the per-cut-off scan with one searchsorted over the running suffix maximum, which is what makes the qualifying indices a prefix. 43 ms -> 0.9 ms per image over all three settings, about 139 s -> 3 s over the split. `_match_predictions` sorted and filtered ten times, once per IoU threshold. The candidate set at a higher threshold is a prefix of the sorted set at the lowest one, and "first occurrence per prediction, then per target" over a prefix keeps exactly the entries of the full-list result that fall inside it, so one sort and one filter answer all ten. The ignored-target test becomes one reduction over the targets instead of materialising a targets x predictions x thresholds boolean cube. 3.9 ms -> 0.8 ms per image. `eval_ade20k`'s metric validated a full-resolution class map with eight elementwise passes and two floors before the histogram it guards. The range test already rejects NaN and both infinities, so only a floating-point array needs the integral test and an integer array needs neither; both maps are gathered once, and validating predictions only ever concerned the pixels with a valid target. 14.4 ms -> 11.4 ms. One port did not land. Compacting the candidate tensors inside `non_max_suppression`, which is 1.8x in the numpy version, measured about 2x *slower* here: six boolean index operations per iteration cost more in torch than one gather through a shrinking index. It keeps the shrinking index, and that measurement is recorded in the function so it is not re-attempted. Nothing here changes behavior, so mblt-model-ops' parity suite passes against 0.0.3 as well; the version moves to 0.0.4 so its pin can name the release these speed-ups are in. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the degenerate WiderFace images the vectorised match dropped Two findings from the Codex review of #9. `image_matches` reduced the overlap matrix with `argmax(1)`, which raises on an empty axis where the per-prediction loop it replaced simply never ran. An image with no predictions therefore raised `ValueError` instead of returning two empty arrays. The top-level evaluator skips that combination, but `image_eval` is a callable helper and a speed rewrite may not narrow what its inputs may be. Both empty tables now short-circuit to "no prediction matched anything", and `matched_image_eval` no longer indexes an empty ignore table either — an image with predictions but no annotated faces previously crashed in `np.max` on an empty row, so it now returns the same "every prediction is a proposal, nothing recalled" the caller's own no-face branch counts. A test pins all three degenerate shapes, and 500 randomised non-degenerate images confirm the guards changed nothing else. The floor-based timm resize contract reached AGENTS.md but not the skill that covers preprocessing work, nor CLAUDE.md. Both now carry it. `mblt_vision/README.md` documents no per-model resize derivation, so it has nothing to restate. That drift had a cause worth recording: `.agents/skills/<name>/` and `.claude/skills/<name>/` are real directories here, not the symlink pair mblt-model-ops uses, so the previous commit's skill edit landed in one copy only. AGENTS.md now says to write both and to diff the pair before committing, and the two copies are identical again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: make the DOTA threshold collapse depend on a stable IoU order Codex was right on #9. Collapsing ten IoU thresholds into one sorted pass rests on the candidate set at a higher threshold being a *prefix* of the sorted set at the lowest one — and `np.argsort`'s default is unstable, so equal IoUs can be ordered one way in the full candidate set and another once the lower-IoU entries are gone. The prefix then stops being the prefix and the collapse picks different pairs than a per-threshold pass would. Its counterexample reproduces exactly: with IoUs [[0.75, 0.75], [0.50, 0.50]] and thresholds [0.5, 0.75], the per-threshold form marked prediction 0 at 0.5 and prediction 1 at 0.75 while the collapse marked prediction 0 at both. `kind="stable"` restores the argument, and measurably so: over 4000 tie-heavy matrices the collapse now agrees with a stable per-threshold pass on every one, where the unstable sort disagreed on 45. Worth recording what the sweep also showed about the form being replaced. Re-sorting each threshold independently with an unstable sort let a prediction come out a true positive at IoU >= 0.75 while not being one at IoU >= 0.50 — 58 non-monotone prediction rows in that sweep, which is not an ordering preference but a metric that contradicts itself. Both the stable per-threshold form and this one produce none. So correctness here is agreement with the stable ordering, and "preserve the old result exactly" would have meant preserving that contradiction. `test_dota_threshold_collapse_survives_tied_ious` pins the counterexample, 200 tie-heavy matrices against a stable per-threshold reference, and monotonicity. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: compare WiderFace score cut-offs in the score dtype, not float64 Carried over from the review of mblt-model-ops!30, which found the same defect in that repository's counterpart. `score_threshold_index` widened the score array to float64 before locating the cut-offs, which silently moved the comparison the official curve makes. The loop it replaced compares a float32 score array against a Python float, and numpy resolves that in float32 by value-based casting, so a score stored as `float32(0.9)` *reaches* the 0.9 cut-off. As float64 the same score is 0.899999976... and misses it. Not a corner case: `norm_score` min-max maps the highest score in the run to exactly 1.0, and with 1000 evenly spaced cut-offs any score landing on one shifts both the proposal and the recall count there. The cut-offs are now cast to the score array's dtype, reproducing exactly what numpy's array-versus-scalar promotion did. Integer score arrays are widened to float64, which is what the original comparison promoted them to. Verified on 400 boundary-heavy tables against the per-cut-off scan, all identical, and `test_widerface_cutoffs_are_compared_in_the_score_dtype` builds score tables out of the cut-offs themselves and requires every cut-off to be reached — the property a widened comparison breaks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
fix: accept single-class end2end YOLOv10 exports in NMS-free postproc… …essing (#8) * refactor: remove YOLOv6 models and update YOLOv8 image sizes for consistency * feat: add face detection support for YOLO models with dedicated postprocessing classes * docs: sync face-detection contracts and input geometry with mblt-model-ops Document the single-class face_detection postprocessing contract and the face-model input geometry taken from mblt-model-ops pipeline.yaml, including the YOLOv8m-face/YOLOv8l-face 960x960 exception, across AGENTS.md, CLAUDE.md, both SKILL.md copies, and mblt_vision/README.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: add the anchor-based YOLOv5-face and YOLOv7-face model configs model-ops carries 24 face models; only 15 were listed here. Add the nine missing anchor-based configs from ../mblt-model-ops/models: YOLOv5{n,n-0.5,s,m}-face (deepcam-cn) and YOLOv7{,s,-tiny,-lite-s,-lite-t}-face (derronqi), taking pre_cfg/post_cfg verbatim from each model's pipeline.yaml. These are the first shipped YAMLs that reach YOLOAnchorFaceDetectionPost. Their published ONNX exports emit three raw (batch, 3, H, W, 6) heads with the upstream landmark pairs stripped, so they decode through the shared anchor path with nc = 1 and iou_thres 0.5. Verified end to end against both the model-ops model.onnx files and the Hub-resolved exports: all nine decode, agree with the shipped YOLOv8n-face reference to within ~2px on the same face, restore into original image coordinates in bounds, and label every row "face". Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: update .gitignore to include MXQ, MBLT, and ONNX file types * feat: add support for restoring class column in end2end single-class exports * fix: bound the single-class end2end restore to max_det rows At nc == 1 a five-column tensor is ambiguous: it is both the end2end export layout (xyxy + confidence, class column stripped) and the width of a converted xywh candidate tensor (4 + nc), which belongs to non_e2e. Separate them by row count. An end2end export is top-k selected and never exceeds max_det rows, while a converted tensor carries one row per anchor (8400 at 640x640), so only the bounded form is treated as already-decoded detections. * fix: address review findings on the single-class end2end restore Three issues from the Codex review of 7ddcd64: - Row count is not provenance. A small single-class configuration produces fewer candidates than max_det (a 64x64 three-head model yields 84), so a converted (B, 84, 5) tensor passed the bound and skipped filter_conversion and dual_topk. Separate the two by the one exact property available: a converted tensor carries precisely one row per anchor, and the configuration knows its own anchor count. Ties are read as converted, so a genuine export of that shape fails loudly instead of silently skipping candidate selection. - Squeezing whichever leading axis happened to be 1 dropped the batch for a (B, 1, N, 5) compiler output with B > 1, which then fell back to raw-head decoding and raised the very error this restore exists to prevent. Squeeze the compiler's axis 1 specifically. - With e2e disabled the restored tensor returned a per-image list, bypassing the non_e2e export contract the CLI promises. Stack it to (B, max_det, 6), matching the decoded-triplet branch above. * fix: skip the end2end restore when the anchor count is unresolvable `tensor.shape[1] == self._anchor_grid_size()` is false for every integer row count when the grid size is None, so a configuration with unresolved anchors lost the guard entirely and had every ambiguous five-column tensor restored as a final export -- the exact silent bypass of conversion and candidate selection the guard exists to prevent. Resolve the grid size first and skip restoration when it is unavailable. Without an anchor count there is nothing to separate a stripped export from a converted candidate tensor, so an unrecognized tensor now fails loudly on raw-head decoding instead. This does not affect the shipped path: anchors resolve at construction, before any inference. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: parkjinman98 <parkjinman98@users.noreply.github.com>
Jm/hotfix (#3) * fix: enhance YOLO anchor postprocessor to support YOLOv7 ONNX head layout * fix: update YOLO anchor postprocessor to handle decode-enabled outputs correctly * fix: add support for decode-enabled YOLOv10 output triplet in NMS-free postprocessor * fix: add support for decode-enabled outputs in YOLO anchorless and DFL-free postprocessors * fix: update WiderFace evaluation metrics to rank by Hard-set AP and retain Medium/Easy AP as secondary metrics * fix: add test for pose evaluation conversion with empty NMS output * fix: update NYU Depth evaluation metrics to include delta1 as primary and abs_rel/RMSE as secondary metrics * Fix decoded YOLO postprocessing review findings * Harden decoded YOLOv10 and pose outputs * Validate decoded YOLO output ranges * Preserve decoded non-E2E output contracts * fix: update return types in YOLODetectionPostBase methods for clarity