Microsoft

Florence-2

About

Florence-2 is a 232M open-weight model from Microsoft. Vision foundation model for captioning, OCR, detection, and region tasks. It accepts text and images. Released June 15, 2024.

Parameters
232M
Published parameter count.
License
MIT
License on the weights.

Pricing

Token rates are US dollars per 1M tokens. Image rates are per 1K images. Audio rates are per 1M audio seconds.

Rate	Price
Input / 1M	$0.10
Cached input / 1M	$0.03
Output / 1M	$0.30

Compare

Florence-2 is one of the catalog's detection models. The table is each model's published price, size, and context.

Model	What it does	Price	Parameters	Context
Florence-2	Vision foundation model for captioning, OCR, detection, and region tasks.	$0.10 / 1M input	232M	—
PP-OCRv6	PaddleOCR PP-OCRv6 medium text detection and recognition; scene OCR JSONL on image chat; text returns plain text; document_url fan-out via text.	$0.02 / 1M input	20M	—
DiffusionGemma 26B-A4B	Block-diffusion Gemma 4 26B-A4B: chat over text and up to 8 images, and System One typed decisions on POST /typesafe/v1/systemone.	$0.35 / 1M input	26B	16K
Gemini 3.5 Flash Lite	Fastest and cheapest Gemini tier. Multimodal chat; emits no reasoning tokens.	$0.30 / 1M input	—	1M

Benchmarks

Published scores for Florence-2, from the Florence-2 model card.

Evals

Reading	Example
Qualitative	Clear structure, grounded in the input
Quantitative	COCO caption CIDEr: 140.
Cost and performance	Lower listed rate, mid-pack latency

Performance

Latency is the end-to-end round trip. Throughput is completion tokens divided by that time. It is a request rate, not decode speed.

Latency
180 ms
Throughput
—
Requests
679

Examples

Dense region captions
Florence-2 names every region in a vegetables-on-counter shot.
{"regions":[{"label":"a tomato","box_xyxy":[58,140,224,396]}, ...]}
Dashboard caption
`more_detailed_caption` describes a shadcn dashboard screenshot.
{"caption":"A modern analytics dashboard with KPI cards, ..."}

Supported inputs

textIgnored
A text part beside the media is accepted and ignored. A request with no media part is a 400.
image_urlYes
One per request. A second image_url part is a 400.
video_urlNo
Not accepted.
document_urlNo
Not accepted.

Default method caption. The caption and ocr methods emit a Markdown string; the region methods emit a region container.
Alias: microsoft/Florence-2-base-ft. Weights: microsoft/Florence-2-base-ft.

Methods

Methods this model serves. Payload shapes are in the docs.

Method	Returns
caption (default)	Short image caption
detailed_caption	Longer caption
more_detailed_caption	Longest caption
ocr	All text on the image
od	Region items with `bbox_xywh` and `label`
dense_region_caption	Region items with `bbox_xywh` and `label`
region_proposal	Region items with `bbox_xywh`, no `label`
ocr_with_region	Region items with `bbox_xywh` and `text`

Each method maps to one Florence-2 task token. The task token itself (<CAPTION>, <OD>, ...) is an implementation detail and is not part of the reply: a caption is the string, and a region method is the container.

Boxes are normalized 0-1 against the image at precision decimals (default 4), and a region method with no hits returns "items": [].

Estimate cost

Input tokensOutput tokens

$0.40

Quick start

Call this model on the OpenAI-compatible gateway. The model id is already filled in.

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.vlm.run/v1/openai",
    api_key="<VLMRUN_API_KEY>",
)

response = client.chat.completions.create(
    model="microsoft/florence-2-base-ft",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://storage.googleapis.com/vlm-data-public-prod/hub/examples/image.caption/car.jpg"
                    },
                },
            ],
        }
    ],
    extra_body={"method": "caption"},
)

print(response.choices[0].message.content)

Response

The caption string alone, with no wrapper:

A cat sitting on a windowsill.