Our LLMs (large language models) produce text in response to a given text prompt. The input and output response combined can be up to 32,768 tokens (about 64,000 characters). The model produces writing across many genres and formats for a wide variety of use cases, including answering questions, writing stories, participating in conversations, and programming in multiple programming languages.

Input

A sample JSON input is shown below:

{
  "text_data": "Cossacks came from what backgrounds?", // Current prompt
  "options": {
    "system_prompt": "Please provide formal responses", //Optional
    "roles": { //Optional
      "user": "Bob",
      "model": "HPT"
    },
    "prompt_history": [
      {
        "content": "this is the first question", 
        "role": "HPT" 
      },
      {
        "content": "this is the first answer",
        "role": "Bob" // the role here must be one of the strings specified in "roles" obj
      },
      // ...
      {
        "content": "this is the newest question",
        "role": "HPT"
      },
      {
        "content": "this is the newest answer",
        "role": "Bob"
      }
    ],
    "max_tokens": 125, // optional input 
    "temperature": 0.8, // optional input 
    "top_p": 0.95 // optional input 
  }
}

Optional input definitions:

Option	Type	Default	Detailed Explanation
system_prompt	String		Provides context for how the model should respond to all requests (e.g. "You are a helpful assistant to...").
roles	Dict		Takes in two optional inputs, "user" and "model". user (string): This will be the name of the customer in prompt_history. model (string): This will be the name of the model in prompt_history If either of these inputs are not explicitly defined verbatim as "user" and "model" within "prompt_history", but instead as some custom inputs "A" and "B", then you must provide the correct mapping within "roles". For instance, custom input "A" maps to "user" and "B" maps to "model."
prompt_history	List	[]	Provides the chat history in chronological order, where the last item is the most recent chat. Each item needs to be a dictionary with the keys "content" and "role". Note: If either role is not explicitly defined as "user" or "model" verbatim, you must modify the "roles" input to provide the correct mapping from your custom role names to "user" and "model" respectively.
max_tokens	Int	5000	Range: [1,4096] This is the maximum token window that you want for your model completion output. We will default to 5000 tokens. In the future, we plan to support up to 32,768 tokens.
temperature	Float	0.8	Range: [0,1] If this value is 0, a repeated request should yield the same response. If this value is 1, a repeated request should yield different responses.
top_p	Float	0.8	Range: [0,1] This is a value between 0 and 1. If the value is 0, requests should be more diverse than when set to a higher value.

Output

A sample JSON response is shown below:

{
    "task_id": "9d8473a0-82a6-11ef-bc2e-97ff9ad6e279",
    "created_on": "2024-10-04T23:15:59.066Z",
    "moderated_on": "2024-10-04T23:15:59.852Z",
    "moderated_by": "classifier",
    "task_units": 130,
    "charge": 0.000065,
    "state": "finished",
    "history": [
        {
            "state": "created",
            "event_on": "2024-10-04T23:15:59.066Z"
        },
        {
            "state": "downloaded",
            "event_on": "2024-10-04T23:15:59.091Z"
        },
        {
            "state": "finished",
            "event_on": "2024-10-04T23:15:59.852Z"
        }
    ],
    "status": [
        {
            "_version": 2,
            "status": {
                "code": "0",
                "message": "SUCCESS"
            },
            "response": {
                "input": {
                    "id": "9d8473a0-82a6-11ef-bc2e-97ff9ad6e279",
                    "created_on": "2024-10-04T23:15:59.066Z",
                    "user_id": 6600,
                    "project_id": 1105330597,
                    "charge": 0.000065,
                    "model": "Llama3_2_1B_Instruct_v0_10_02_24",
                    "model_version": 1,
                    "model_type": "LARGE_LANGUAGE_MODEL",
                    "text": "What do you think about a cat playing with a yarn ball"
                },
                "output": [
                    {
                        "task_units": 130,
                        "collection_id": null,
                        "input_tokens": 47,
                        "total_output_tokens": 83,
                        "total_tokens": 130,
                        "task_units_per_input_token": 1,
                        "task_units_per_output_token": 1,
                        "augmentations": [],
                        "choices": [
                            {
                                "message": "Wazzzzzzzzup, I think it's a pretty purr-fect idea! Cats are natural predators, and they love to stalk and pounce on small toys, so a yarn ball would be a great fit for their playful energy! Wazzzzzzzzup, who knows, maybe the cat will even use its claws to unravel the yarn, creating a fun little game of yarn-based mischief!",
                                "stop_reason": "finish",
                                "output_tokens": 83
                            }
                        ]
                    }
                ]
            }
        }
    ],
    "hash": "24a9e91e57478a7c9be1a7f74c145f318a3198c3ed069f6e0c31b3ea11f38895-0",
    "callback_metadata": null,
    "internal_metadata": null,
    "callback_url": "https://webhook.site/74735289-5ec6-405d-927b-1e7ee4e5efaa",
    "callback_status_code": 429,
    "callback_status_message": null,
    "callback_retry_count": 9,
    "original_filename": null,
    "original_url": null,
    "label_data": null,
    "task_description_translations": null,
    "text_data": "What do you think about a cat playing with a yarn ball",
    "task_type": "real",
    "focus": null,
    "static_objects": null,
    "status_to_edit": null,
    "media_objects": [],
    "extra_media_objects": null,
    "media_url": null,
    "media_urls": [],
    "media_type": null,
    "media_deleted": false,
    "result_deleted": false,
    "invalidated_previous_task_id": null,
    "invalidated_next_task_id": null,
    "task_tags": null,
    "project_override": null,
    "subtask_data": null,
    "task_options": null,
    "error": null,
    "error_code": null,
    "pipeline_data": {
        "pipeline_task": null,
        "component_tasks": null
    },
    "confirmation_data": {
        "pa_task": null,
        "confirmation_tasks": null
    }
}

For a list of common errors, refer to our Common Errors page.

What’s Next

What’s Next