Contextual Scene Classification

Overview

The purpose of Contextual Scene Classification is to identify the context of visual content. A key use case would be content-based ad targeting: helping media companies to sell more granular contextually-targeted ad spots.

Input: Hive's API can receive both images and videos. Videos have their frames sampled and treated as separate images. Support for text and audio inputs will be made available separately.

Output: For each image, Hive's API will return a set of classes with a confidence score from zero to one. These classes are explained below.

Content Alignment Classes

Hive's API returns classes from four broad categories: settings, subjects, objects and the IAB Content Taxonomy. The IAB Content Taxonomy is an industry-standard for ad-targeting use cases. More information on these classes is available here.

Settings:

• airplane_indoor
• amusement_park
• apartment_outdoor
• aquarium_or_sealife
• bar_or_nightclub_indoor
• beach_outdoor
• body_shop
• car_indoor
• casino_indoor
• concert_or_auditorium_outdoor
• concert_or_theatre_indoor
• construction_site
• courtroom_indoor
• cruise_outdoor
• factory_indoor
• farm_crop
• gym_indoor
• home_bathroom
• hospital_indoor
• hotel_indoor
• hotel_outdoor
• house_indoor
• house_outdoor
• kitchen_indoor
• library_bookstore
• movie_theatre
• museum_indoor
• no_location
• office_indoor
• other_indoor
• other_outdoor
• park_outdoor
• place_of_worship
• pool_or_hot_tub_indoor
• pool_or_hot_tub_outdoor
• pre_k_through_12_indoor
• restaurant_indoor
• retail_outdoor
• retail_store_indoor
• road_outdoor
• sports_arena
• tv_studio
• university_indoor
• university_outdoor
• wilderness_outdoor
• zoo_or_wildlife

Subjects:

• award_show
• birth
• birthday
• cooking
• crashes
• funeral
• graduation
• musical_performance
• other_subject
• political_event
• runway_show
• sports_&_athletic_activities
• wedding

Objects:

• no_motor_vehicle
• yes_motor_vehicle
• no_lawnmower
• yes_lawnmower
• no_tractor
• yes_tractor
• no_cargo_ship
• yes_cargo_ship
• no_boat_dock
• yes_boat_dock
• no_fashion
• yes_fashion
• no_shaver
• yes_shaver
• no_flower
• yes_flower
• no_holiday_decor
• yes_holiday_decor
• no_food
• yes_food
• no_pots_pans
• yes_pots_pans
• no_wine
• yes_wine
• no_beer
• yes_beer
• no_children_toys
• yes_children_toys
• no_tablet
• yes_tablet
• no_gambling
• yes_gambling
• no_video_games
• yes_video_games
• no_camera
• yes_camera
• no_gym_equipment
• yes_gym_equipment
• no_baby
• yes_baby
• no_children
• yes_children
• no_medical_staff
• yes_medical_staff
• no_train
• yes_train
• no_airplane
• yes_airplane
• no_cell_phone
• yes_cell_phone
• no_tv
• yes_tv
• no_person
• yes_person
• no_dog
• yes_dog
• no_cat
• yes_cat
• no_bed
• yes_bed
• no_public_transportation
• yes_public_transportation
• no_motorcycle
• yes_motorcycle
• no_commercial_truck
• yes_commercial_truck
• no_station_wagon
• yes_station_wagon
• no_suv
• yes_suv
• no_van
• yes_van
• no_pick_up_truck
• yes_pick_up_truck
• no_computer
• yes_computer
• no_small_standard_car
• yes_small_standard_car
• no_dining_table
• yes_dining_table
• no_outer_space
• yes_outer_space

Expected Output

For each frame there will be a set of classes each with a score scaled from 0 (negative) to 1 (positive), indicating the level of confidence for the classification.

Supported File Types

Image Formats:
gif
jpg
png
webp

Video Formats:
mp4
webm
avi
mkv
wmv
mov