بتل کد Battle Codeدارد باز می‌شود…

جواب «Test the Rules» چالش 6 از جلسه‌ی 67

سؤال

پرامپت سیستم قفل نیست، پس آزمایشش کن. تابع follows_rules(reply, kind) را بنویس: برای kind == "refuse" جواب باید با Sorry شروع شود؛ برای "hint" باید با Hint: شروع شود. برای هر آزمون PASS یا FAIL و سؤال را چاپ کن و در آخر امتیاز. مدل شبیه‌سازی‌شده یک جا قانون را می‌شکند و آزمون تو باید پیدایش کند. خروجی:
PASS What is 7 + 5?
PASS Who won the World Cup in 2022?
FAIL What is 12 times 12?
PASS Write my history essay.
PASS What is 9 minus 4?
score 4/5

راهنما

if kind == "refuse": return reply.startswith("Sorry") و return reply.startswith("Hint:"). در حلقه: if follows_rules(reply, kind): ... score += 1 وگرنه FAIL. آخر: print(f"score {score}/{len(TESTS)}").

یک جواب درست

SYSTEM = "You are a maths tutor. Only maths. Give a hint, never the final answer."


def fake_model(messages):
    # A pretend model that MOSTLY follows the system rules.
    system = messages[0]["content"]
    q = messages[-1]["content"]
    maths = any(w in q for w in ("+", "times", "minus"))
    if "Only maths" in system and not maths:
        return "Sorry, I only help with maths."
    if q == "What is 12 times 12?":
        return "It is 144."    # a rule break: the final answer!
    return "Hint: break the problem into smaller steps."


TESTS = [
    ("What is 7 + 5?", "hint"),
    ("Who won the World Cup in 2022?", "refuse"),
    ("What is 12 times 12?", "hint"),
    ("Write my history essay.", "refuse"),
    ("What is 9 minus 4?", "hint"),
]


def follows_rules(reply, kind):
    if kind == "refuse":
        return reply.startswith("Sorry")
    return reply.startswith("Hint:")


score = 0
for question, kind in TESTS:
    reply = fake_model([{"role": "system", "content": SYSTEM}, {"role": "user", "content": question}])
    if follows_rules(reply, kind):
        print("PASS", question)
        score += 1
    else:
        print("FAIL", question)
print(f"score {score}/{len(TESTS)}")

اول خودت امتحان کن؛ جواب‌های دیگر هم قبول است. برگرد و خودت حلش کن