Skip to content

Add face keypoints/animal pose ops & Extend ops for frame-sequence input#966

Open
Qirui-jiao wants to merge 2 commits intomainfrom
dev/face_and_animal_pose_ops
Open

Add face keypoints/animal pose ops & Extend ops for frame-sequence input#966
Qirui-jiao wants to merge 2 commits intomainfrom
dev/face_and_animal_pose_ops

Conversation

@Qirui-jiao
Copy link
Copy Markdown
Collaborator

Add ops:

  • video_animal_pose_mapper: Detect quadruped animal pose on the video with ViTPose.
  • video_face_keypoints_mapper: Detect face keypoints (98 points) on the video with LDEQ_RwR.

Enable existing ops to support frame-sequence input:

  • video_camera_calibration_static_moge_mapper
  • video_depth_estimation_mapper
  • video_hand_reconstruction_hawor_mapper
  • video_whole_body_pose_estimation_mapper

@Qirui-jiao Qirui-jiao added enhancement New feature or request dj:op issues/PRs about some specific OPs labels Apr 14, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces two new video mappers, VideoAnimalPoseMapper and VideoFaceKeypointsMapper, for detecting animal poses and face keypoints, respectively. It also updates several existing video mappers to support input from pre-extracted frames. The review feedback identifies several critical issues, including an AttributeError in the animal pose mapper, a missing comma in a category list, and incorrect return types in multiple mappers that would break the processing pipeline. Additionally, there are suggestions to simplify video name extraction and improve the robustness of path handling.

Comment thread data_juicer/ops/mapper/video_animal_pose_mapper.py Outdated
Comment thread data_juicer/ops/mapper/video_animal_pose_mapper.py Outdated
Comment thread data_juicer/ops/mapper/video_camera_calibration_static_moge_mapper.py Outdated
Comment thread data_juicer/ops/mapper/video_hand_reconstruction_hawor_mapper.py Outdated
Comment thread data_juicer/ops/mapper/video_whole_body_pose_estimation_mapper.py Outdated
Comment thread data_juicer/ops/mapper/video_animal_pose_mapper.py Outdated
Comment thread data_juicer/ops/mapper/video_camera_calibration_static_moge_mapper.py Outdated
Comment thread data_juicer/ops/mapper/video_face_keypoints_mapper.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dj:op issues/PRs about some specific OPs enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant