What it does:
1. Accepts query with JSON schema, this helps to solve few things at once - provides JSON structure for LLM to generate response, and hints LLM what types to use for each response element
2. Runs inference on your GPU of choice, either cloud or local GPU
3. Validates JSON response, based on query schema
No comments:
Post a Comment