Guides

Image input

Send images to a model: how each protocol writes them, base64 versus image links, and what happens across protocols.

Models that can “see” read the images you send them, for example to describe a picture, read text in it or analyse a chart.

First check that the model accepts images. Open Models & Pricing and look at Input types in the model’s details. A model without image input either ignores the image or returns an error.

How each protocol writes it

Images go in the content array of a user message, next to the text.

OpenAI Chat Completions:

{
  "role": "user",
  "content": [
    {"type": "text", "text": "What is in this image?"},
    {"type": "image_url", "image_url": {"url": "data:image/png;base64,iVBORw0KGgo..."}}
  ]
}

url can be base64 data in the form data:<image type>;base64,<encoded data>, or an https:// link to the image.

Anthropic Messages:

{
  "role": "user",
  "content": [
    {"type": "image", "source": {"type": "base64", "media_type": "image/png", "data": "iVBORw0KGgo..."}},
    {"type": "text", "text": "What is in this image?"}
  ]
}

Gemini:

{
  "role": "user",
  "parts": [
    {"inlineData": {"mimeType": "image/png", "data": "iVBORw0KGgo..."}},
    {"text": "What is in this image?"}
  ]
}

Prefer base64 over image links

base64: the image data travels inside the request and doesn’t depend on any outside address. This is the most reliable option.
Image links: when NoviaHub has to convert the format, it downloads the image itself and then passes it to the model. Calling a Claude model through Chat Completions is one such case. So the link must be reachable directly from the public internet; it can’t be an internal address, an address that needs a login, or one that no longer works. If the download fails, the request fails. In our tests, calling a Claude model through Chat Completions with an image link that returned 404 gave HTTP 500, error code convert_request_failed and the message get file data failed: failed to download file, status code: 404. When the target is the Gemini format, the message starts with get file data from '<link>' failed.

Across protocols

NoviaHub converts images to the target protocol’s format automatically. We confirmed these cases in tests:

Your request	Protocol the model actually uses	What the model receives
Chat `image_url` (base64)	Anthropic (Claude models)	`{"type": "image", "source": {"type": "base64", ...}}`
Chat `image_url` (base64)	Gemini	`{"inlineData": {"mimeType": ..., "data": ...}}`
Anthropic base64 image	OpenAI Chat	`image_url` holding a base64 address that starts with `data:`

Billing

Images are converted to input tokens by the vendor’s rules and charged at the input price. Some models price image input separately: if the price table in the model’s details has an Image In column, that price applies. You can see how many tokens each call actually used in the details of Usage logs.