Skip to content

Add example: deploy Faster-Whisper as a custom model on a SageMaker Triton MME#4888

Open
eldadcohen1 wants to merge 1 commit into
aws:defaultfrom
eldadcohen1:sm-triton_mme_faster_whisper
Open

Add example: deploy Faster-Whisper as a custom model on a SageMaker Triton MME#4888
eldadcohen1 wants to merge 1 commit into
aws:defaultfrom
eldadcohen1:sm-triton_mme_faster_whisper

Conversation

@eldadcohen1

Copy link
Copy Markdown

Adds an example showing how to deploy a pre-trained Faster-Whisper-Large-v3 model from Hugging Face as a custom model on an Amazon SageMaker multi-model endpoint (MME) backed by Triton, including building a custom Triton container and running audio-transcription inference.

Description of changes:

  • New example under deploy_and_monitor/sm-triton_mme_faster_whisper/ (notebook, README.md, and a sample audio file), following the current repository structure and sm-triton_* naming convention.
  • Registered the notebook in the Deploy and Monitor toctree in index.rst.
  • CI badge URLs and kernelspec (conda_python3) match the sibling Triton examples.

This supersedes #4608 / #4887, which targeted the legacy main branch and the old sagemaker-triton/ layout. This PR targets the current default branch and the new folder structure.

Testing is done:
Tested on AWS SageMaker.

Merge Checklist

  • I have read the CONTRIBUTING doc and adhered to the example notebook best practices
  • I have updated any necessary documentation, including READMEs
  • I have tested my notebook(s) and ensured it runs end-to-end
  • I have linted my notebook(s) and code using black-nb -l 100 {path}/{notebook-name}.ipynb

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

…riton MME

Ports the multi-model-endpoint Triton custom-model (Faster-Whisper-Large-v3)
example into the new repository structure under deploy_and_monitor/, updates
the CI badge URLs and kernelspec to match sibling examples, and registers the
notebook in the Deploy and Monitor toctree.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@eldadcohen1

Copy link
Copy Markdown
Author

@mollyheamazon can you please review and approve

@mollyheamazon

Copy link
Copy Markdown

The notebook uses backend: "python" in config.pbtxt, but there's no cell that writes the required model.py into the model directory. Could you add the %%writefile cell for it? Without it a reader can't reproduce end-to-end.

Another note: This notebook uses raw boto3 calls rather than the SageMaker Python SDK. It might be a better fit for awsdocs/aws-doc-sdk-examples (which hosts boto3 examples)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants