Skip to content

Add example: deploy Faster-Whisper as a custom model on a SageMaker Triton MME#4887

Closed
eldadcohen1 wants to merge 12 commits into
aws:mainfrom
eldadcohen1:mme-triton-faster-whisper
Closed

Add example: deploy Faster-Whisper as a custom model on a SageMaker Triton MME#4887
eldadcohen1 wants to merge 12 commits into
aws:mainfrom
eldadcohen1:mme-triton-faster-whisper

Conversation

@eldadcohen1

Copy link
Copy Markdown

Use Faster-Whisper as a custom model artifact from Hugging Face that is served on Amazon SageMaker as a multi-model endpoint with Triton.

Description of changes:
Adds a new example notebook under sagemaker-triton/mme-triton-custom-faster-whisper/ showing how to package a pre-trained Faster-Whisper-Large-v3 model, build a custom Triton container, and deploy it to a SageMaker GPU multi-model endpoint. Described in the readme.

This PR supersedes #4608, which had a stale branch and a head branch of main. This one is rebased on the latest upstream main and uses a dedicated feature branch.

Testing is done:
Tested on AWS SageMaker.

Merge Checklist

  • I have read the CONTRIBUTING doc and adhered to the example notebook best practices
  • I have updated any necessary documentation, including READMEs
  • I have tested my notebook(s) and ensured it runs end-to-end
  • I have linted my notebook(s) and code using black-nb -l 100 {path}/{notebook-name}.ipynb

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@eldadcohen1

Copy link
Copy Markdown
Author

@mollyheamazon can you please take a look

@eldadcohen1

Copy link
Copy Markdown
Author

Hi @atqy — could a maintainer help get the CI moving on this PR?

The three required AWS CodeBuild us-west-2 (SageMakerNotebooksProd-USWEST2-...) checks are stuck as "expected" and have never started. I verified via the GitHub API that the head commit (c5da5c5) has 0 check-runs and the only posted status is ReadTheDocs (success) — sagemaker-bot has not posted a CI report for this commit, which means the CodeBuild builds were never triggered, not that they're running.

Some context:

  • This PR supersedes added an example for how to deploy mme with triton for custom model b… #4608 (same content). I rebased onto the latest upstream main (it was ~49 commits behind) and moved it to a dedicated feature branch instead of main, in case the trigger was tied to the branch name — but the CodeBuild jobs still didn't fire.
  • The notebook uses docker build, so the amazon-sagemaker-examples-pr execution check should skip it (per the testing repo's is_notebook_skipped logic).
  • Black formatting and link-check were already passing on the prior commits.

Could you please re-trigger the CodeBuild checks for this PR (or let me know if the fork-PR CI pipeline is currently inactive and how you'd like me to proceed)? Happy to push an empty commit or close/reopen if that helps re-fire the webhook. Thanks!

@eldadcohen1

Copy link
Copy Markdown
Author

Superseded by #4888, which targets the current default branch and ports the example into the new repository structure (deploy_and_monitor/sm-triton_mme_faster_whisper/). The main branch this PR targeted is the legacy layout and its CI no longer triggers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant