This article provides recommendations regarding how to transcribe audio files offline with as much throughput as possible.

Offline transcription is handled by a task queue which chops the available audio in pieces and processes all of the pieces as much as possible simultaneously given the number of available recognizer ports.

Edge Deployments

An Edge deployment will have fix number (will call it N) of concurrent transcriptions that it is capable of  and that number will depend on the number of installed GPUs.
If one looks at the problem simplistically, submitting 1 or N transcriptions sessions simultaneously should give the same throughput. However, that does not account for the fact that before the file pieces can be processed in parallel, the file has to be uploaded which has to be completed before transcription starts.

Thus, the 1 simultaneous session is least effective because the recognizer would be idle when the upload is in progress.

The N simultaneous sessions scenario has a problem when all of the uploads align - in that case the /data API service needs to have sufficient memory allocated to hold the N files in memory. We suggest if you want to try the N-simultaneous scenario, please let us know the max size of the files that you will transcribe so that we can custom tailor your Edge configuration.

The most optimal for throughput setup is schematically illustrated below:

This setup separates the upload part from the processing (transcription) part.

• Transcription is handled by a fixed pool of 3 or 4 processing threads.
• When a transcription thread is ready to process a new audio it retrieves it from the queue of audio UUIDs that have been uploaded using the /data API
• Should the uuid queue size drop to less than a preset amount, a new upload tread is launched using an unbounded executor
• The upload threads take the audio file names from the master queue/list of the audio files to be transcribed.

This way we always are uploading only the optimal and small number of audio files at a time. And the transcription threads are always busy.

Cloud Deployments

TBD ...