Databricks and Langfuse
What is Databricks? > Databricks is a unified analytics platform founded by the creators of Apache Spark. It provides an interactive workspace for collaborative data engineering, machine learning, and data analytics. With Databricks, teams can build, train, and deploy models at scale, efficiently harnessing big data and advanced analytics tools.
What is Langfuse? > Langfuse is a comprehensive platform designed to help developers monitor, trace, and evaluate their language models in production. It offers powerful insights through detailed logging and event tracing, ensuring robust performance monitoring and easier debugging of AI applications.
Tracing and Observability
Databricks serving endpoints expose an OpenAI-compatible API, so you can trace them with Langfuse in three ways: via the OpenAI SDK, via LangChain, or via LlamaIndex.
For all three approaches, configure your Langfuse and Databricks credentials as environment variables:
import os
# Get keys for your project from the project settings page: https://cloud.langfuse.com
os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..."
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..."
os.environ["LANGFUSE_BASE_URL"] = "https://cloud.langfuse.com" # 🇪🇺 EU region
# Other Langfuse data regions include 🇺🇸 US: https://us.cloud.langfuse.com, 🇯🇵 Japan: https://jp.cloud.langfuse.com and ⚕️ HIPAA: https://hipaa.cloud.langfuse.com
os.environ["DATABRICKS_TOKEN"] = "dapi-..." # Databricks personal access token
os.environ["DATABRICKS_HOST"] = "https://dbc-XXXXX-XXXX.cloud.databricks.com" # Databricks workspace URLDatabricks endpoints act as a drop-in replacement for the OpenAI API. The langfuse.openai client automatically traces your requests to Langfuse. For more examples, see the OpenAI integration docs.
pip install langfuse openaiimport os
from langfuse.openai import OpenAI
# Create an OpenAI-like client pointing to Databricks
client = OpenAI(
api_key=os.environ.get("DATABRICKS_TOKEN"),
base_url=f"{os.environ.get('DATABRICKS_HOST')}/serving-endpoints",
)
response = client.chat.completions.create(
messages=[
{"role": "system", "content": "You are an AI assistant."},
{"role": "user", "content": "What is Databricks?"},
],
model="mistral-7b", # Adjust based on your Databricks serving endpoint name
max_tokens=256,
)
print(response.choices[0].message.content)
The ChatDatabricks class wraps your Databricks Model Serving endpoint; the Langfuse CallbackHandler collects the trace data. For more examples, see the LangChain integration docs.
pip install langfuse databricks-langchainfrom databricks_langchain import ChatDatabricks
from langfuse.langchain import CallbackHandler
# Initialize Langfuse CallbackHandler for LangChain (tracing)
langfuse_handler = CallbackHandler()
chat_model = ChatDatabricks(
endpoint="mistral-7b", # Your Databricks Model Serving endpoint name
temperature=0.1,
max_tokens=256,
)
messages = [
("system", "You are a chatbot that can answer questions about Databricks."),
("user", "What is Databricks Model Serving?"),
]
chat_model.invoke(messages, config={"callbacks": [langfuse_handler]})
If you use LlamaIndex, you can replace the default LLM with a Databricks endpoint and trace calls via the OpenInference LlamaIndex instrumentation. For more examples, see the LlamaIndex integration docs.
pip install langfuse llama-index llama-index-llms-databricks openinference-instrumentation-llama-indeximport os
from langfuse import get_client
from llama_index.core.llms import ChatMessage
from llama_index.llms.databricks import Databricks
from openinference.instrumentation.llama_index import LlamaIndexInstrumentor
langfuse = get_client()
# Initialize LlamaIndex instrumentation
LlamaIndexInstrumentor().instrument()
# Create a Databricks LLM instance
llm = Databricks(
model="mistral-7b", # Your Databricks serving endpoint name
api_key=os.environ.get("DATABRICKS_TOKEN"),
api_base=f"{os.environ.get('DATABRICKS_HOST')}/serving-endpoints/",
)
messages = [
ChatMessage(role="system", content="You are a helpful assistant."),
ChatMessage(role="user", content="What is Databricks?"),
]
response = llm.chat(messages)
print(response)
Playground & Evaluations
This guide walks you through integrating Databricks language model endpoints with Langfuse. By doing so, you can quickly experiment with prompts and debug interactions using the Langfuse Playground, as well as benchmark your models systematically with Evaluations.
With Langfuse, you can:
- Experiment in the Playground: The interactive Playground lets you test your language models in real-time. You can send custom prompts, review detailed responses, and add prompts to your Prompt Library.
- Benchmark with Evaluations: LLM-as-a-Judge evaluations provide a way to benchmark your application's performance. You can run pre-defined test templates, analyze metrics like latency and accuracy, and refine your models based on measurable outcomes.
Set Up a Serving Endpoint in Databricks
Begin by setting up a serving endpoint in Databricks. This lets you query custom fine-tuned models or models served via a gateway such as OpenAI or Anthropic. For advanced configuration options, refer to the Databricks docs.
Add the Model in your Project Settings
Next, add your Databricks model endpoint to your Langfuse project settings.
Make sure you've entered the correct endpoint URL and authentication details. The model name is the name of the serving endpoint you created in Databricks.
Use the Model in the Playground
The Langfuse Playground offers an interactive interface where you can:
- Send prompts and view quick results.
- Add prompts to your Prompt Library.
Select Databricks as your LLM provider and choose the endpoint you configured earlier.
Use the Model for Evaluations
LLM-as-a-judge is a technique to evaluate the quality of LLM applications by using an LLM as a judge. The LLM is given a trace or a dataset entry and asked to score and reason about the output. The scores and reasoning are stored as scores in Langfuse.
If you want to learn more about LLM Evals, check out our blog post:
Last edited
