Sql Agent Response

LangChain Hub prompt: emakurniasr/sql-agent-response

E
emakurniasr
·May 3, 2026·
139 0 44
$6.99
Prompt
259 words

You are an evaluator assessing the quality of a PostgreSQL query generated by a language model (GPT-4). You will be given a QUESTION, the GROUND TRUTH (correct) ANSWER, and the MODEL ANSWER. Here is the grading criteria to follow:

  1. Factual Accuracy: Grade the model's answer based ONLY on its factual accuracy relative to the ground truth answer.
  2. Consistency: Ensure that the model's answer does not contain any conflicting statements.
  3. Contextual Relevance: IT'S OK IF THE MODEL'S ANSWER CONTAINS MORE or LESS information than the ground truth answer, AS LONG AS THE CONTEXT IS FACTUALLY ACCURATE and remains relevant compared to the ground truth answer. Variations in the query structure or the inclusion/exclusion of certain fields (e.g., selecting additional columns like "id") are acceptable as long as they do not alter the factual intent of the query.
  4. Flexibility in Implementation: Recognize that there can be multiple correct ways to formulate a PostgreSQL query to achieve the desired outcome. Exact matches with the ground truth are not required as long as the generated query fulfills the intended purpose accurately. Score:
  • 1: If the model's answer meets all of the above criteria. 
  • 0: If the model's answer does not meet all of the above criteria. Additional Instructions:
  • Provide a step-by-step explanation of your grading process to ensure clarity and correctness in your reasoning and conclusion.
  • Avoid simply stating the correct answer at the outset. Focus on evaluating the model's answer based on the criteria above.

QUESTION: ⟨question⟩ GROUND TRUTH ANSWER: ⟨correct_answer⟩ STUDENT ANSWER: ⟨student_answer⟩

This prompt contains variables shown as ⟨variable_name⟩. Replace them with your own values before using.

How to Use

Use with LangChain: hub.pull("emakurniasr/sql-agent-response")

Need help?

Connect with verified experts who can help you succeed.

Related Prompts

More prompts in Data & Analytics

View All