tubeshowcase●1,672,522 videos♡ My collection 0
← Learn to Code

Learn to Code37:11

Azure Cognitive Service deployment: AI inference with NVIDIA Triton Server | BRKFP04

Microsoft Developer ↗

From the creator

Join us to see how Azure Cognitive Services utilize NVIDIA Triton Inference Server for inference at scale. We highlight two use cases: deploying first-ever Mixture of Expert model for document translation and acoustic model for Microsoft Teams Live Captioning. Tune in to learn about serving models with NVIDIA Triton, ONNX Runtime and custom backends. Additional Resource: Live Captions in Teams meeting -- https://support.microsoft.com/en-us/office/use-live-captions-in-a-teams-meeting-4be2d304-f675-4b57-8347-cbd000a21260 Southpaw Token-based load balancer @Scale announcement -- https://atscaleconference.com/videos/southpaw-token-based-service-load-balancing-scaling-and-qos-system-osama-abuelsorour-amr-mahdi/ Triton GitHub -- https://github.com/triton-inference-server Custom C/C++ backends -- https://github.com/triton-inference-server/backend Writing a custom Python backend -- https://github.com/triton-inference-server/python_backend Southpaw Token-based load balancer -- https://atscaleconference.com/videos/southpaw-token-based-service-load-balancing-scaling-and-qos-system-osama-abuelsorour-amr-mahdi/ Custom Python backends -- https://github.com/triton-inference-server/backend Triton GitHub -- https://github.com/triton-inference-server Recommended Next Step: Please visit the NVIDIA showcase -- https://mybuild.microsoft.com/en-US/partners/0528d9cd-18ea-4592-9dbc-719716b01dd9 Microsoft Build 2022

Published 2022-05-27 · Watch on YouTube ↗
Availability and playback restrictions are controlled by the original channel.

Keep exploring Learn to Code