Announcing our Python and Node SDK

We’re excited to announce our VLM Run Python and Node SDKs, making it easier than ever for developers to integrate Vision Language Models (VLMs) into their applications. With just a few lines of code, you can process images, documents, and videos, extract structured data, and even fine-tune VLMs for industry-specific tasks.

The Python and Node SDK (in beta) is now available on our GitHub. You can also explore real-world examples in our cookbook repo. Be sure to follow our VLM Run Github org for more updates from us.

👩‍💻 What’s New in the VLM Run SDKs?

Our SDK simplifies how developers interact with our Visual AI APIs, reducing the complexity of handling OCR, structured document extraction, image understanding, and more.

Calling VLMs with VLM Run now takes just a few lines of code in Python — you can install our client via pip install vlmrun to get started.

🖼️ Image to JSON

Use any of our 20+ custom schemas with Pydantic data validation we support to perform visual ETL from images.

from PIL import Image
from vlmrun.client import VLMRun

client = VLMRun(api_key="...")
response = client.image.generate(
    images=[Image.open("invoice.jpg")], 
    domain="document.invoice"
)

📜 Document to JSON

Our VLMs can automatically handle PDFs and extract JSON without any additional processing on your end.

from pathlib import Path
from vlmrun.client import VLMRun

client = VLMRun(api_key="...")
response = client.document.generate(
    file=Path("test.pdf"), 
    domain="document.invoice"
)

🧪 Finetuning VLMs Simplified

We now support fine-tuning our VLMs for image and video inputs with our dead-simple API. Specialize our VLMs for your industry within minutes - check out our Colab showcase of our fine-tuned VLMs for dashcam video cataloging and analytics.

from vlmrun.client import VLMRun

client = VLMRun(api_key="...")

# Upload dataset containing images / jsonl labels
training_file = client.files.upload(Path("dataset/"))

# Train our VLMs in < 2 lines of code
client.fine_tuning.create(
	training_file=training_file.id, 
    model="vlm-1", 
    epochs=10
)

👨‍🍳 Ready-to-use Cookbooks

We provide a host of cookbooks for you to get started on your visual ETL journey. Take a look our growing list of Colab notebooks for inspiration.

• VLM Run: API QuickStart

• Features: Schema Showcase, Visual Grounding for VLMs

• Applications: Parsing Financial Presentations, TV News Analysis

🚀 Get Started

Sign up for an API key on VLM Run today, and let us know what you build! Connect with us on Discord and Twitter if you have any questions.

• Python SDK: Install via pip install vlmrun

• Node SDK: Install via npm install vlmrun

• API Docs: Explore full API reference

• Cookbooks: Step-by-step guides to get started

• Schema Hub: Pre-built schemas for structured data extraction

• VLM Run Github: Follow us for the latest updates