LLMs

This is an example of how to implement a LLM based functions, for use in our agent examples. Feel free to look at the source code and fit it for your own usecase.

Chat

from stringdale.chat import Chat

Chat

source

Signature: Chat(model: Optional[str] = None, messages: Optional[List[Dict[str, str]]] = None, output_schema: Optional[pydantic.main.BaseModel] = None, as_json: Optional[bool] = False, tools: Optional[Dict[str, Callable]] = None, call_function: Optional[bool] = False, choices: Optional[enum.Enum] = None, multi_choice: Optional[bool] = False, seed: Optional[int] = 42, stop: Union[str, List[str], NoneType] = None, log_prompt: bool = False, save_history: bool = False, append_output: bool = False, init_messages: Optional[List[Dict[str, str]]] = None, **kwargs)

A Chat objects the renders a prompt and calls an LLM. Currently supporting openai models.

Parameter	Type	Default	Description
model	typing.Optional[str]	None	OpenAI model name
messages	typing.Optional[typing.List[typing.Dict[str, str]]]	None	List of message dicts, must have at least a role and content field
output_schema	typing.Optional[pydantic.main.BaseModel]	None	Optional schema for structured output
as_json	typing.Optional[bool]	False	Optional boolean to return the response as a json object
tools	typing.Optional[typing.Dict[str, typing.Callable]]	None	Optional dictionary of tool names and functions that the LLM can decide to call. Causes the content of the responseto be a dict of the form {‘name’:tool_name,‘input’:tool_input_dict}
call_function	typing.Optional[bool]	False	if tools are provided, whether to call the function and save the output in the output field of the response’s content
choices	typing.Optional[enum.Enum]	None	Optional List of choices for multi-choice questions
multi_choice	typing.Optional[bool]	False	if choices are provided, whether to choose multiple items from the list
seed	typing.Optional[int]	42	Optional seed for random number generation
stop	typing.Union[str, typing.List[str], NoneType]	None	Optional string or list of strings where the model should stop generating
save_history	<class ‘bool’>	False	Optional boolean to save the history of the chat between calls
append_output	<class ‘bool’>	False	Optional, whether to append the output of the chat to history automatically, default False
init_messages	typing.Optional[typing.List[typing.Dict[str, str]]]	None	Optional list of messages that are always prepended to messages.Useful for supplying additional messages during calls.Can have template variables that are fed during initialization only.If save_history is True, the init messages are added to the history.
**kwargs	None	None	Keyword arguments to interpolate into the messages

__call__

Format prompt with kwargs and call OpenAI chat. Init parameters such as output_schema, tools, choices, seed, stop, as well as template variables can be set or overridden by kwargs > Signature: Chat.__call__(self, **kwargs) -> Dict[str, Any]

Parameter	Type	Default	Description
**kwargs	None	None	Values for format string placeholders

Examples

messages=[
        {"role": "system", "content": "Given a sentence, classify it into one of these topics: science, history, technology, or arts. Choose the single most relevant topic."},
        {"role": "user", "content": "{{text}}"}
    ]
    
topic_classifier = Chat(
    model="gpt-4o-mini",
    messages=messages,
    choices = ['science', 'history', 'technology', 'arts'],
    seed=42,
)
topic_classifier

Chat(model='gpt-4o-mini', required_keys={'text'}, choices=['science', 'history', 'technology', 'arts'], seed=42)

await topic_classifier(text="WWII was a global conflict that lasted from 1939 to 1945.")

{'role': 'assistant',
 'content': 'history',
 'meta': {'input_tokens': 180, 'output_tokens': 10}}

from pydantic import BaseModel

class Person(BaseModel):
    first_name: str
    last_name: str
    date_of_birth: int

prompted_llm = Chat(model="gpt-4o-mini", messages=
    [   
        {"role": "user", "content": "how old am i? {{name}}, {{age}} years old"},
        # {"role": "assistant", "content": "Iam {{model_name}}, You are {{name}}, {{age}} years old"}
    ],
     output_schema=Person)
prompted_llm

Chat(model='gpt-4o-mini', required_keys={'age', 'name'}, output_schema=Person, seed=42)

await prompted_llm(model_name="gpt-4o-mini", age=30,name="Jake")

{'role': 'assistant',
 'content': Person(first_name='Jake', last_name='', date_of_birth=1993),
 'meta': {'input_tokens': 186, 'output_tokens': 27}}

def google_search_stub(query:str):
    """
    Search the web for the query
    Args:
        query: The query to search for
    Returns:
        The URL of the search results
    """
    return f"https://www.google.com/search?q={query.replace(' ','_')}"

tools = {'google_search': google_search_stub}

google_search = Chat(model="gpt-4o-mini", messages=[{"role": "user", "content": "{{text}}"}], tools=tools, call_function=True)
res = await google_search(text="What is the capital of France?")
assert res['content'] == {'name': 'google_search',
  'input': {'query': 'What is the capital of France?'},
  'output': 'https://www.google.com/search?q=What_is_the_capital_of_France?'}
res

{'role': 'assistant',
 'content': {'name': 'google_search',
  'input': {'query': 'What is the capital of France?'},
  'output': 'https://www.google.com/search?q=What_is_the_capital_of_France?'},
 'meta': {'input_tokens': 365, 'output_tokens': 32}}

Image to Text

from stringdale.chat import image_to_text

image_to_text

source

Signature: image_to_text(path: str, model: str = 'gpt-4o-mini', url=False)

This function takes an image (either from a local file path or URL) and uses OpenAI’s vision model to generate a detailed description of the image contents. The results are cached using disk_cache to avoid redundant API calls.

Parameter	Type	Default	Description
path	<class ‘str’>	None	Path to the image file or URL of the image
model	<class ‘str’>	gpt-4o-mini	OpenAI model to use for image analysis. Defaults to “gpt-4o-mini”.
url	bool	False	Whether the path is a URL. Defaults to False.
:Returns:	dict	-	A dictionary containing: - role (str): Always “assistant” - content (str): Detailed description of the image - meta (dict): Usage statistics including input and output tokens

await image_to_text('../../../sample_data/fox.jpeg')

{'role': 'assistant',
 'content': "The image features a close-up of a fox's face. The fox has a bushy, reddish-brown fur with lighter shades on its face and underbelly. It has pointed ears with dark tips, and its eyes appear sharp, reflecting a sense of alertness. The background is blurred, emphasizing the fox's facial features. The lighting suggests a warm, soft glow, highlighting the textures of the fur.",
 'meta': {'input_tokens': 8626, 'output_tokens': 91}}

Speech to Text

from stringdale.chat import speech_to_text

speech_to_text

source

Signature: speech_to_text(audio_path: str, model: str = 'whisper-1') -> Dict[str, str]

Extract text from an audio file using OpenAI’s Whisper model.

Parameter	Type	Default	Description
audio_path	<class ‘str’>	None	Path to the audio file
model	<class ‘str’>	whisper-1	OpenAI model to use. Defaults to “whisper-1”.
:Returns:	typing.Dict[str, str]	-	A dictionary containing: - role (str): Always “assistant” - content (str): Transcribed text from the audio

await speech_to_text('../../../sample_data/happy_speech.wav')

{'role': 'assistant',
 'content': "Look at this, my hands are standing up in my arms, I'm giving myself goosebumps."}