Generate Text

To interact with LLMs, you can use the generate_text function.


text = px.generate_text(prompt="Hello AI world!")

This function makes best effort to unify the API calls to different providers.

Examples

Basic Usage


text = px.generate_text(prompt="Hello, world!")


Hello! How can I assist you today?

Using System Prompt


text = px.generate_text(
  system="Only answer with the single digit number.",
  prompt="What is 2 + 2?",
)

Using Messages


text = px.generate_text(
  system="No matter what, you must answer with 7.",
  messages=[
    {"role": "user", "content": "Hello AI Model!"},
    {"role": "assistant", "content": "7"},
    {"role": "user", "content": "What is 2 + 2?"},
  ],
)

Using Custom Provider Model

Set provider_model to a tuple of (provider, model) string values.


text = px.generate_text(
  prompt="Hello model! What is your name and which company do you belong to?",
  provider_model=('claude', 'opus')
)


My name is Claude and I was created by Anthropic. It's nice to meet you!

You can also use px.types.ProviderModel to specify the provider and model.


provider_model = px.models.get_model(
    provider='openai',
    model='o4-mini')
print(repr(provider_model))
text = px.generate_text(
    prompt="Hello model! What is your name and which company do you belong to?",
    provider_model=provider_model
)


ProviderModelType(provider=openai, model=o4-mini, provider_model_identifier=o4-mini-2025-04-16)
I’m ChatGPT, a language model developed and maintained by OpenAI.

Check all available models on ProxAI Github Repo

Extensive Return

Set extensive_return to True to get more information about the API call. This returns a px.types.LoggingRecord object.


from pprint import pprint
 
response = px.generate_text(
  prompt="Hello model!",
  extensive_return=True
)
pprint(response)


LoggingRecord(query_record=QueryRecord(call_type=<CallType.GENERATE_TEXT: 'GENERATE_TEXT'>,
                                       model=(<Provider.OPENAI: 'openai'>,
                                              <OpenAIModel.GPT_3_5_TURBO: 'gpt-3.5-turbo'>),
                                       prompt='Hello model!',
                                       system=None,
                                       messages=None,
                                       max_tokens=100,
                                       temperature=None,
                                       stop=None,
                                       hash_value=None),
              response_record=QueryResponseRecord(response='Hello! How can I '
                                                           'help you today?',
                                                  error=None,
                                                  error_traceback=None,
                                                  start_utc_date=datetime.datetime(2025, 2, 13, 0, 38, 17, 573491, tzinfo=datetime.timezone.utc),
                                                  end_utc_date=datetime.datetime(2025, 2, 13, 0, 38, 17, 930739, tzinfo=datetime.timezone.utc),
                                                  local_time_offset_minute=-0.0,
                                                  response_time=datetime.timedelta(microseconds=357285),
                                                  estimated_cost=150),
              response_source=<ResponseSource.PROVIDER: 'PROVIDER'>,
              look_fail_reason=None)

You can also convert response to a dictionary.


from dataclasses import asdict
pprint(asdict(response))


{'look_fail_reason': None,
 'query_record': {'call_type': <CallType.GENERATE_TEXT: 'GENERATE_TEXT'>,
                  'hash_value': None,
                  'max_tokens': 1000,
                  'messages': None,
                  'prompt': 'Hello model!',
                  'provider_model': {'model': 'gpt-4',
                                     'provider': 'openai',
                                     'provider_model_identifier': 'gpt-4-0613'},
                  'stop': None,
                  'system': None,
                  'temperature': None},
 'response_record': {'end_utc_date': datetime.datetime(2025, 5, 3, 17, 16, 58, 827118, tzinfo=datetime.timezone.utc),
                     'error': None,
                     'error_traceback': None,
                     'estimated_cost': 30000,
                     'local_time_offset_minute': -0.0,
                     'response': 'Hello! How can I assist you today?',
                     'response_time': datetime.timedelta(microseconds=790699),
                     'start_utc_date': datetime.datetime(2025, 5, 3, 17, 16, 58, 36446, tzinfo=datetime.timezone.utc)},
 'response_source': <ResponseSource.PROVIDER: 'PROVIDER'>}

Parameters

px.generate_text() parameters:

Option	Type	Default Value	Description
`prompt`	`str`	`None`	The prompt to generate text from.
`system`	`str`	`None`	System prompt to the model that will be prioritized over user prompt.
`messages`	`List[Dict[str, str]]`	`None`	List of messages that represents the history of the conversation. • Each element needs to be a dictionary with `role` and `content` keys. • `role` must be either `user` or `assistant` in alternating order. • `prompt` parameter shouldn’t be set if `messages` parameter is set.
`max_tokens`	`Optional[int]`	`1000`	Maximum number of tokens to generate.
`temperature`	`Optional[float]`	`None`	Temperature parameter for the model.
`stop`	`Optional[Union[str, List[str]]]`	`None`	Sequence of tokens to stop generation. It could be either string or list of strings.
`provider_model`	`Optional[Union[Tuple[str, str], px.types.ProviderModel]]`	`None`	Which provider and model to use for generating text. • If not set, ProxAI will pick the provider and model for the task. See, Set Global Model.
`use_cache`	`Optional[bool]`	`None`	Whether to use cache for the API call. If set to `False`, the cache will be bypassed and the API call will be made directly to the provider. See, Cache System.
`unique_response_limit`	`Optional[int]`	`None`	Maximum number of unique calls to the provider before returning a cached response in round-robin fashion. See, Cache System.
`extensive_return`	`bool`	`False`	Returns all metadata about the API call including errors, cache info, speed, etc.
`suppress_provider_errors`	`bool`	`False`	If `True`, provider errors are suppressed. See, Suppress Provider Errors page.