Audio/Speech Interpretation

Overview

Hive offers a suite of audio-related products capable of moderating and extracting a wide range of information from your audio and video streams. Given a clip, our models can execute the following capabilities:

  • transcription: extract out all speech
  • speech moderation: detect undesirable speech
  • audio sound moderation: detect undesirable sounds
  • audio sound classification: identify different kinds of sounds
  • language classification: determine which language was spoken

Our customers currently use our products for a wide range of applications, from auditing social media videos, streams, and chat rooms, to doing brand-mention analysis on TV programs. Please read the below sections for more information on each of our products.

Speech-to-Text Transcription

Hive's Speech-to-Text API ingests an audio stream and returns each word that was spoken, along with a confidence score and timestamp for that word. We additionally return a fully punctuated transcript of the entire text. If you wish to use multiple languages, we also offer automatic language detection where you can pass in any audio clip and we'll identify/transcribe to the correct language automatically.

We currently support the following languages:

  • English
  • Spanish
  • Portuguese
  • French
  • Hindi
  • German

These languages are in progress and will be included in future releases [tentative list that will grow over time]:

  • Italian (2023 Q2)
  • Arabic (2023 Q1)
  • Chinese (2023 Q2)
  • Japanese (2023 Q2)

We are actively working on adding in new languages. If you are interested in a language that is not currently supported, please contact [email protected]

Speech Moderation

Hive offers a speech moderation API built on top of our speech-to-text and text moderation models. Through one endpoint, Hive Speech Moderation will return back all the information present in the Speech-to-Text transcription endpoint (i.e. a confidence score and timestamp for each detected word as a well as a punctuated transcript across all detected words), as well as scores for our supported moderation classes.

We currently can support the following languages and their corresponding moderation classes:

LanguageSexualViolenceHateBullying
EnglishModelModelModelModel
SpanishModelModelModelModel
PortugueseModelModelModelModel
FrenchModelModelModelModel
HindiModelModelModelModel
ArabicModelModel2023 Q1Model
GermanModel-2023 Q1Model
ItalianPattern Match / Model(2023 Q2)---
TurkishPattern Match---
ChinesePattern Match / Model(2023 Q2)Pattern MatchPattern MatchPattern Match
RussianPattern Match---
DutchPattern Match---
KoreanPattern Match---
JapanesePattern Match / Model(2023 Q2)---
VietnamesePattern Match---
RomanianPattern Match---
PolishPattern Match---

We are actively working on adding more languages for this API suite. If you are interested in a language that is not currently supported or is not on the roadmap, please contact [email protected] For more details about our text moderation models, please refer to the text content moderation docs page.

Sound Moderation and Classification

We are currently working on a series of audio sounds/events to help you moderate/classify your content. We hope this information helps you to filter out bad audio that goes beyond the raw transcript and gain more insight into your content. Today, Hive's model covers the following classes:

  • sexual noises
  • yelling/shouting (coming 2023 Q1)

Supported File Types

Video Formats:
mp4
webm
avi
mkv
wmv
mov

Audio Formats
flac
mp3
ogg
wav
m4a