LangSmith Complete Tutorial 2026: How to debug, test, and monitor LLM applications
Step-by-step guide to using LangSmith for AI-powered devops workflows
LangSmith Complete Tutorial 2026: How to debug, test, and monitor LLM applications
Step-by-step guide to using LangSmith for AI-powered devops workflows
LangSmith Complete Tutorial 2026 What is LangSmith? **LangSmith** is a powerful LLM observability that enables you to debug, test, and monitor LLM applications. It has become one of the most popular tools in the AI developer toolkit in 2026. Why U
LangSmith Complete Tutorial 2026
What is LangSmith?
LangSmith is a powerful LLM observability that enables you to debug, test, and monitor LLM applications. It has become one of the most popular tools in the AI developer toolkit in 2026.
Why Use LangSmith?
Getting Started
Installation
bash
npm/yarn (Node.js projects)
npm install langsmithpip (Python projects)
pip install langsmithOr use the hosted version at langsmith.com
Configuration
yaml
config.yml
name: my-langsmith-app
version: 1.0.0integrations:
openai:
api_key: 1897628437146480647
anthropic:
api_key: undefined
settings:
timeout: 30
retry_attempts: 3
log_level: info
Core Concepts
Basic Workflow
python
Python example
from langsmith import Client, WorkflowInitialize
client = Client(api_key="your-key")Create a workflow
workflow = Workflow()
workflow.add_step("input", type="user_message")
workflow.add_step("ai_process", model="gpt-4o-mini", type="llm_call")
workflow.add_step("output", type="response")Execute
result = client.run(workflow, input="Your prompt here")
print(result.output)
JavaScript/TypeScript Example
typescript
import { LangSmithClient } from 'langsmith';const client = new LangSmithClient({
apiKey: process.env.LANGSMITH_API_KEY,
});
async function main() {
const result = await client.run({
workflow: 'my-workflow',
input: { message: 'Hello, AI!' }
});
console.log(result.output);
}
main();
Real-World Use Cases
Use Case 1: debug, test, and monitor LLM applications
python
Complete example: debug, test, and monitor LLM applications
import os
from openai import OpenAIopenai_client = OpenAI()
def create_devops_pipeline(input_data: dict) -> dict:
"""
Pipeline for debug, test, and monitor LLM applications using LangSmith.
"""
# Step 1: Process input
processed = preprocess(input_data)
# Step 2: AI analysis
response = openai_client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{
"role": "system",
"content": f"You are an expert in {t.category}. Help with debug, test, and monitor LLM applications."
},
{
"role": "user",
"content": str(processed)
}
]
)
# Step 3: Post-process
result = {
"input": input_data,
"analysis": response.choices[0].message.content,
"timestamp": datetime.now().isoformat()
}
return result
Run it
result = create_devops_pipeline({
"topic": "debug, test, and monitor LLM applications",
"context": "Building modern AI applications"
})
print(result["analysis"])
Use Case 2: Integration with Other Tools
python
Integrate LangSmith with your existing stack
import httpx
import jsonclass LangSmithIntegration:
def __init__(self, api_key: str):
self.client = httpx.AsyncClient(
base_url="https://api.langsmith.com",
headers={"Authorization": f"Bearer {api_key}"}
)
async def process(self, data: dict) -> dict:
response = await self.client.post("/process", json=data)
response.raise_for_status()
return response.json()
async def batch_process(self, items: list) -> list:
import asyncio
tasks = [self.process(item) for item in items]
return await asyncio.gather(*tasks)
Usage
import asyncioasync def main():
integration = LangSmithIntegration(
api_key=os.environ["LANGSMITH_KEY"]
)
results = await integration.batch_process([
{"input": "Item 1"},
{"input": "Item 2"},
{"input": "Item 3"},
])
for r in results:
print(r)
asyncio.run(main())
Advanced Features
Monitoring and Logging
python
import logging
from functools import wraps
import timelogging.basicConfig(level=logging.INFO)
logger = logging.getLogger("langsmith")
def with_logging(func):
@wraps(func)
async def wrapper(*args, **kwargs):
start = time.time()
logger.info(f"Starting {func.__name__}")
try:
result = await func(*args, **kwargs)
duration = time.time() - start
logger.info(f"Completed {func.__name__} in {duration:.2f}s")
return result
except Exception as e:
logger.error(f"Error in {func.__name__}: {e}")
raise
return wrapper
@with_logging
async def my_workflow(data: dict):
# Your LangSmith workflow here
pass
Error Handling
python
from tenacity import retry, stop_after_attempt, wait_exponential@retry(
stop=stop_after_attempt(3),
wait=wait_exponential(multiplier=1, min=4, max=10)
)
def reliable_api_call(data: dict) -> dict:
"""Retry on failure with exponential backoff."""
try:
return process(data)
except RateLimitError:
logger.warning("Rate limit hit, retrying...")
raise
except APIError as e:
if e.status_code >= 500:
raise # Retry on server errors
raise # Don't retry on client errors
Pricing and Plans
Comparison with Alternatives
Conclusion
LangSmith is an excellent LLM observability that makes it easy to debug, test, and monitor LLM applications. Its combination of power and usability makes it a top choice for AI developers in 2026.
Whether you're building your first AI application or scaling an enterprise system, LangSmith provides the tools you need to succeed.
*Tutorial for LangSmith latest version | May 2026*
相关工具
相关教程
Which AI coding assistant delivers the best ROI for professional developers in 2025?
o3 适合什么任务,如何在 ChatGPT 和 API 中高效使用
How HR teams use AI to hire better, reduce bias, and improve employee retention