LangSmith

Set up LangSmith observability to debug, test, evaluate, and monitor your agents.

Set up LangSmith to debug, trace, and monitor your agents from the LangSmith dashboard.

Before starting, complete the quickstart guide to have an agent running on Idun Agent Platform.

Set up LangSmith observability

1. Get your LangSmith API key

   If you do not have a LangSmith API key yet:

   1. Go to LangSmith
   2. Sign up or log in to your account
   3. Navigate to Settings (gear icon) > API Keys
   4. Click Create API Key
   5. Copy your API Key
2. Create an observability configuration

   1. On the Idun Agent Platform main page, navigate to Observability
   2. Click Add configuration
   3. Select LangSmith
   4. Enter a Configuration Name (for example, “LangSmith Prod”)
   5. Fill in the required details:
      • API Key: Your LangSmith API key (starts with lsv2-...)
      • Project Name: The name of the project in LangSmith (for example, default or prod-agent)
      • Project ID: (Optional) The specific project identifier if needed
      • Endpoint: (Optional) Custom endpoint if you are self-hosting LangSmith (for example, https://api.smith.langchain.com)
      • Tracing Enabled: Toggle to enable or disable tracing globally
      • Capture Inputs/Outputs: Toggle to log full text of inputs and outputs
   6. Click Create configuration

   After creating, you can click the Verify button on the configuration card to check that the connection works.

   Keep your API key secure. Do not commit it to version control or share it publicly.

3. Attach LangSmith to your agent

   1. Navigate to the agent you want to trace
   2. Click Edit Agent
   3. Click Next to reach the observability configuration step
   4. Select LangSmith as the observability provider
   5. Select the LangSmith configuration you want to use
   6. Click Next
   7. Click Save changes to finalize

   After saving, click Restart on your agent page to reload the configuration and enable LangSmith observability.

View observability data

Once your agent is running with observability enabled:

1. Interact with your agent through the Manager UI or API
2. Open your LangSmith dashboard at smith.langchain.com
3. Navigate to your project to view traces

You will see traces showing the execution run tree, LLM inputs/outputs, and latency. Each trace run is named after your agent by default.

Best practices

• Use distinct projects for development and production environments
• Tag runs to filter traces when investigating specific issues (configured within agent logic)
• Review error traces in LangSmith to identify and resolve problems quickly

Troubleshooting

Observability not working?

1. Check API key: Verify it is valid and has the required permissions
2. Verify project name: Traces are sent to the “default” project if the project name is not specified or is incorrect
3. Check tracing toggle: Confirm the tracing toggle is enabled in the configuration