Contextual Scene Classification

Overview

The purpose of Contextual Scene Classification is to identify the context of visual content. A key use case would be content-based ad targeting: helping media companies to sell more granular contextually-targeted ad spots.

Input: Hive's API can receive both images and videos. Videos have their frames sampled and treated as separate images. Support for text and audio inputs will be made available separately.

Output: For each image, Hive's API will return a set of classes with a confidence score from zero to one. These classes are explained below.

Content Alignment Classes

Hive's API returns classes from four broad categories: settings, subjects, objects and the IAB Content Taxonomy. The IAB Content Taxonomy is an industry-standard for ad-targeting use cases. More information on these classes is available here.

Settings:

  • airplane_indoor
  • amusement_park
  • apartment_outdoor
  • aquarium_or_sealife
  • bar_or_nightclub_indoor
  • beach_outdoor
  • body_shop
  • car_indoor
  • casino_indoor
  • concert_or_auditorium_outdoor
  • concert_or_threatre_indoor
  • construction_site
  • courtroom_indoor
  • cruise_outdoor
  • factory_indoor
  • farm_crop
  • gym_indoor
  • home_bathroom
  • hospital_indoor
  • hotel_indoor
  • hotel_outdoor
  • house_indoor
  • house_outdoor
  • kitchen_indoor
  • library_bookstore
  • movie_theatre
  • museum_indoor
  • no_location
  • office_indoor
  • other_indoor
  • other_outdoor
  • park_outdoor
  • place_of_worship
  • pool_or_hot_tub_indoor
  • pool_or_hot_tub_outdoor
  • pre_k_through_12_indoor
  • restaurant_indoor
  • retail_outdoor
  • retail_store_indoor
  • road_outdoor
  • sports_arena
  • tv_studio
  • university_indoor
  • university_outdoor
  • wilderness_outdoor
  • zoo_or_wildlife

Subjects:

  • award_show
  • birth
  • birthday
  • cooking
  • crashes
  • funeral
  • graduation
  • musical_performance
  • other_subject
  • political_event
  • runway_show
  • sports_&_athletic_activities
  • wedding

Objects:

  • no_motor_vehicle
  • yes_motor_vehicle
  • no_lawnmower
  • yes_lawnmower
  • no_tractor
  • yes_tractor
  • no_cargo_ship
  • yes_cargo_ship
  • no_boat_dock
  • yes_boat_dock
  • no_fashion
  • yes_fashion
  • no_shaver
  • yes_shaver
  • no_flower
  • yes_flower
  • no_holiday_decor
  • yes_holiday_decor
  • no_food
  • yes_food
  • no_pots_pans
  • yes_pots_pans
  • no_wine
  • yes_wine
  • no_beer
  • yes_beer
  • no_children_toys
  • yes_children_toys
  • no_tablet
  • yes_tablet
  • no_gambling
  • yes_gambling
  • no_video_games
  • yes_video_games
  • no_camera
  • yes_camera
  • no_gym_equipment
  • yes_gym_equipment
  • no_baby
  • yes_baby
  • no_children
  • yes_children
  • no_medical_staff
  • yes_medical_staff
  • no_train
  • yes_train
  • no_airplane
  • yes_airplane
  • no_cell_phone
  • yes_cell_phone
  • no_tv
  • yes_tv
  • no_person
  • yes_person
  • no_dog
  • yes_dog
  • no_cat
  • yes_cat
  • no_bed
  • yes_bed
  • no_public_transportation
  • yes_public_transportation
  • no_motorcycle
  • yes_motorcycle
  • no_commercial_truck
  • yes_commercial_truck
  • no_station_wagon
  • yes_station_wagon
  • no_suv
  • yes_suv
  • no_van
  • yes_van
  • no_pick_up_truck
  • yes_pick_up_truck
  • no_computer
  • yes_computer
  • no_small_standard_car
  • yes_small_standard_car
  • no_dining_table
  • yes_dining_table
  • no_outer_space
  • yes_outer_space

Expected Output

For each frame there will be a set of classes each with a score scaled from 0 (negative) to 1 (positive), indicating the level of confidence for the classification.

Supported File Types

Image Formats:
gif
jpg
png
webp

Video Formats:
mp4
webm
avi
flv
mkv
mpg
wmv
mov