Enhance Checkpoint Management, Debugging, and File Utilities #1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…e redunant snapshots at the end
…e redunant snapshots at the end
Co-authored-by: n-poulsen <45132115+n-poulsen@users.noreply.github.com>
…e redunant snapshots at the end
…lelte the redunant snapshots at the end" This reverts commit 31d8652.
- adding in CZI badge! 🔥
* Benchmarking user guide * Update Benchmarking_shuffle_guide.md * Update _toc.yml --------- Co-authored-by: Mackenzie Mathis <mathis@rowland.harvard.edu> Co-authored-by: Mackenzie Mathis <mackenzie.mathis@epfl.ch> Co-authored-by: Anna Stuckert <anna.stuckert97@gmail.com> Co-authored-by: Marc Canela <marc.canela@outlook.com>
- broken link fix
…#2736) * Create OtherData.md - new doc, old wiki page! * Update _toc.yml - add to toc
* Create citation.md * Update _toc.yml * Update README.md
…e redunant snapshots at the end
• Added a get_checkpoint_epoch function to retrieve the current epoch from a PyTorch checkpoint. • Updated checkpoint handling logic to calculate the adapted checkpoint path.
…efully. Returns 0 if 'epoch' is not found in the checkpoint, improving robustness.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Updated the method to convert dynamic skeleton keypoint names to numerical indices directly from the class attribute, improving clarity and functionality. Added detailed docstring to explain the method's purpose and usage.
There was a problem hiding this comment.
Pull Request Overview
This pull request enhances checkpoint management, debugging capabilities, and file utilities across the DeepLabCut codebase. The changes focus on improving model checkpoint handling during video inference, adding debugging tools for pose estimation, and introducing utility functions for file downloads and JSON formatting.
Key Changes:
- Added checkpoint epoch retrieval functionality for dynamic checkpoint path generation during adaptation training
- Enhanced debugging capabilities with print statements for keypoints, bounding boxes, and batch structure analysis
- Introduced utility functions for downloading files from Hugging Face Hub and JSON indentation formatting
Reviewed Changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| deeplabcut/modelzoo/video_inference.py | Added checkpoint epoch retrieval function and torch import for checkpoint management |
| deeplabcut/pose_estimation_pytorch/runners/train.py | Added debugging comments and print statements for batch structure analysis |
| deeplabcut/pose_estimation_pytorch/data/utils.py | Added debug prints for detecting anomalous keypoints and bounding boxes |
| deeplabcut/pose_estimation_pytorch/data/image.py | Added commented safety checks for zero-dimension bounding boxes |
| deeplabcut/pose_estimation_pytorch/apis/evaluation.py | Added extensive visualization functionality for PFM-specific prediction analysis |
| deeplabcut/utils/make_labeled_video.py | Enhanced video processing to load additional video metadata |
| deeplabcut/common_tools/json_tools.py | New utility for reformatting JSON files with proper indentation |
| deeplabcut/common_tools/download.py | New utility for downloading files from Hugging Face Hub |
| .circleci/data | Added CircleCI data path configuration |
| else: | ||
| return 0 | ||
|
|
||
| def get_checkpoint_epoch(checkpoint_path): |
There was a problem hiding this comment.
The function get_checkpoint_epoch is defined twice with identical signatures at lines 38 and 54. This will cause the second definition to overwrite the first, making the first function unreachable.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Simplified the method's docstring to enhance clarity while maintaining its functionality. Removed unnecessary comments to streamline the code.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces several new functionalities and improvements across multiple modules, focusing on enhanced file handling, model checkpoint management, debugging, and video processing. Below is a summary of the most significant changes:
New Features:
download_from_huggingface_hubindeeplabcut/common_tools/download.pyto facilitate downloading files from the Hugging Face Hub into a specified local directory. This includes support for subfolder paths and examples for usage.convert_json_indent_folder_levelindeeplabcut/common_tools/json_tools.pyto reformat JSON files with proper indentation, ensuring better readability and organization.Model Checkpoint Management:
get_checkpoint_epochindeeplabcut/modelzoo/video_inference.pyto extract the current epoch number from PyTorch checkpoint files. This is used to dynamically generate checkpoint paths during adaptation training. [1] [2] [3]video_inference_superanimalto handle checkpoint paths dynamically based on current and adapted epochs, ensuring compatibility with various training stages. [1] [2]Debugging Enhancements:
deeplabcut/pose_estimation_pytorch/data/utils.pyto log keypoints and bounding box data when anomalies (e.g., all-zero values) are detected. [1] [2]deeplabcut/pose_estimation_pytorch/runners/train.pyto provide insights into batch structure and dataset paths during training. [1] [2]Miscellaneous Improvements:
torchlibrary import indeeplabcut/modelzoo/video_inference.pyto support new checkpoint-related functionality.proc_videoindeeplabcut/utils/make_labeled_video.pyto load additional video-related data (full_data) for improved processing.These changes collectively improve the functionality, maintainability, and debugging capabilities of the codebase.