Getting started with Root Signals

To get started, please sign up and login to the Root Signals app. Using the SDK or API downstream requires creating an API key.

With Python SDK, you can start either by attaching Root Signals to your existing OpenAI style call or by creating an LLM pipeline (skill) on Root platform.

The following instructions will get you started on the SDK path. Alternatively, you can begin building on the UI or via the REST API.

pip install root-signals

Quick test

For the best experience, we encourage you to create an account. However, if you prefer to run quick tests at this point, please create a temporary API key here.

Adding evaluators and alternative models to OpenAI chat

By wrapping an OpenAI chat objects with a Root skill, you can:

• attach it to any of the dozens of models supported by the Root proxy

• attach any validators to the chat

• monitor all chat interactions immediately via Root Signals UI.

Interaction results beyond OpenAI's ChatResponse can be retrieved separately. The two steps are:

1. Create the Root Signals skill with system prompt and model selection.

2. Pass the skill.openai_base_url as base_url and your RootSignals API key as api_key to the client.

from openai import OpenAI
from root import RootSignals
from root.validators import Validator

# Just a quick test?
# You can get a temporary API key from https://app.rootsignals.ai/demo-user 
rs_client = RootSignals(api_key="my-developer-key")

skill = rs_client.skills.create(
    name="My chatbot",
    intent="Simple Q&A chatbot",
    system_message="You are a helpful assistant.",
    model="gpt-4-turbo",
    fallback_models=["gpt-4"],
    validators=[Validator(evaluator_name="Precision", threshold=0.8)],
)

# Start chatting with the skill
client = OpenAI(base_url=skill.openai_base_url, api_key=rs_client.api_key)

completion = client.chat.completions.create(
    model=skill.models[0],  #RS skills can have multiple models for failover
    messages=[{"role": "user", "content": "How many alligators live in Florida?"}]
)
print(completion.choices[0].message.content)

Getting validation results requires retrieving execution logs from the rs_client

log = rs_client.execution_logs.get(log_id=completion.id)
log.validation_result_average
# 1.0

Creating an integration using a skill

from root import RootSignals
from root.validators import Validator

rs_client = RootSignals(api_key="my-developer-api-key")

skill = rs_client.skills.create(
    name="sdk-demo-skill",
    intent="Simple Q&A chatbot",
    model="gpt-4",
    prompt="Answer the question in markdown {{question}}",
    validators=[
        Validator(evaluator_name="Relevance", threshold=0.8),
        Validator(evaluator_name="Precision", threshold=0.6)
        ],  
)

If you do not need to use OpenAI style, you can leverage a fully managed skill with prompt templates. See Skills for more details.

We can now run the skill

response = skill.run(variables={"question": "List the states in the US"})
print(response.llm_output)

Leveraging the validators

Since we added validators when creating the skill, we can retrieve the results for each execution as a dictionary

result = skill.run(variables={"question": "List the biggest states and their largest everything in the US."})
result.validation
{
'validator_results': 
 [
   'evaluator_name': 'Relevance',
   'evaluator_id': '97730ce7-1837-43bc-a242-ae0fd1443627',
   'threshold': 0.8,
   'is_valid': False,
   'result': 0.65,
   'status': 'finished'
  },
  {
   'evaluator_name': 'Precision',
   'evaluator_id': '99622e9d-72f1-41cf-bc63-ed4fb0ae23f7',
   'threshold': 0.6,
   'is_valid': True,
   'result': 0.75,
   'status': 'finished'
   }
 ],
 'is_valid': False
 }

The result dictionary has a Boolean summary field is_valid with True or False indicating whether ALL of the validators passed their threshold values. You can easily implement your own validators and downstream conditional logic as necessary. You can remove the check altogether by not setting a threshold or setting it to 0.

The string response is available as result.llm_output:

1. **Texas**
   - Largest state by land area in the contiguous United States.
   - Known for having large cities like Houston, Dallas, and Austin.
   - Significant oil production and large ranches.

2. **California**
   - Most populous state in the U.S.
   - Home to the largest economy in the U.S. and the world's fifth-largest economy.
   - Contains the largest number of national parks and diverse landscapes.

3. **Alaska**
   - Largest state by land area overall.
   - Known for vast wilderness, largest national parks, and significant natural resources.

4. **New York**
   - Home to New York City, the largest city in the U.S. by population.
   - Major financial and cultural hub.

5. **Florida**
   - Known for having the longest coastline in the contiguous U.S.
   - Large tourism industry, with attractions like Walt Disney World and numerous beaches.

6. **Illinois**
   - Home to Chicago, one of the largest cities in the U.S. by population and a major economic center.