Auto-process manually uploaded images (if enabled) #909
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* feat: configure default related models with new projects * feat: allow specifying which pipelines are enabled by default * feat: add denmark/uk model to default pipelines * feat: add tests for default enabled pipelines * chore: rename default station * fix: undefined variables in certain cases * chore: cleanup typos and comments * Update ami/ml/models/processing_service.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * fix: live processing endpoint URL
✅ Deploy Preview for antenna-preview canceled.
|
…tart-prototype-auto-process
Collaborator
Author
|
@mihow put this behind a feature flag in the project settings and then I think we can merge it The EXIF reader can always happen by default - consider falling back to todays date |
mihow
changed the title
Contributor
There was a problem hiding this comment.
Pull Request Overview
This pull request implements a quickstart prototype with auto-processing capabilities for uploaded images. The changes introduce media utilities for extracting timestamps and calculating checksums, orchestration modules for automated pipeline processing, and modifications to the image upload flow to automatically process images upon upload.
- Added media utility functions for timestamp extraction from EXIF data and filename, plus file checksum calculation
- Created ML orchestration modules for processing individual images and selecting default pipelines
- Modified image upload workflow to automatically extract timestamps, calculate checksums, and trigger ML processing
- Updated SourceImageCollection default method from "common_combined" to "full"
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| ami/utils/media.py | New utility functions for extracting timestamps from images and calculating file checksums |
| ami/ml/orchestration/processing.py | Processing orchestration for individual source images with automatic job creation |
| ami/ml/orchestration/pipelines.py | Pipeline selection logic with default pipeline determination |
| ami/ml/orchestration/init.py | Module initialization for orchestration package |
| ami/ml/models/pipeline.py | Added custom QuerySet and Manager for pipeline filtering by project |
| ami/main/models.py | Updated image upload process with automatic timestamp extraction and processing |
| ami/main/migrations/0061_alter_sourceimagecollection_method.py | Database migration for SourceImageCollection method field reordering |
| ami/main/api/serializers.py | Removed filename timestamp validation and added process_now parameter |
…tart-prototype-auto-process
…tart-prototype-auto-process
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces a workflow for automatically processing manually uploaded images, if enabled (via feature flag). This was implemented as a prototype to allow users to test data more quickly (https://github.com/RolnickLab/antenna/milestone/29). It will be disabled by default however the prototype introduced several features and fixes that are helpful in general (and will be available by default).
orchestration/where methods likeprocess_single_image andget_default_pipelinecan live. Many methods fromml/models.pyandmain/models.pyshould be moved here, as well as some of the upcoming post_processing functions such astracking.py` See Split ami.main into multiple modules #260, Add support for occurrence tracking #863, Add class masking (draft) #915, [Draft] Taxon rank classifications #857