جواب «Test the Rules» چالش 6 از جلسهی 67
سؤال
پرامپت سیستم قفل نیست، پس آزمایشش کن. تابع follows_rules(reply, kind) را بنویس: برای kind == "refuse" جواب باید با Sorry شروع شود؛ برای "hint" باید با Hint: شروع شود. برای هر آزمون PASS یا FAIL و سؤال را چاپ کن و در آخر امتیاز. مدل شبیهسازیشده یک جا قانون را میشکند و آزمون تو باید پیدایش کند. خروجی:PASS What is 7 + 5?PASS Who won the World Cup in 2022?FAIL What is 12 times 12?PASS Write my history essay.PASS What is 9 minus 4?score 4/5
راهنما
if kind == "refuse": return reply.startswith("Sorry") و return reply.startswith("Hint:"). در حلقه: if follows_rules(reply, kind): ... score += 1 وگرنه FAIL. آخر: print(f"score {score}/{len(TESTS)}").
یک جواب درست
SYSTEM = "You are a maths tutor. Only maths. Give a hint, never the final answer."
def fake_model(messages):
# A pretend model that MOSTLY follows the system rules.
system = messages[0]["content"]
q = messages[-1]["content"]
maths = any(w in q for w in ("+", "times", "minus"))
if "Only maths" in system and not maths:
return "Sorry, I only help with maths."
if q == "What is 12 times 12?":
return "It is 144." # a rule break: the final answer!
return "Hint: break the problem into smaller steps."
TESTS = [
("What is 7 + 5?", "hint"),
("Who won the World Cup in 2022?", "refuse"),
("What is 12 times 12?", "hint"),
("Write my history essay.", "refuse"),
("What is 9 minus 4?", "hint"),
]
def follows_rules(reply, kind):
if kind == "refuse":
return reply.startswith("Sorry")
return reply.startswith("Hint:")
score = 0
for question, kind in TESTS:
reply = fake_model([{"role": "system", "content": SYSTEM}, {"role": "user", "content": question}])
if follows_rules(reply, kind):
print("PASS", question)
score += 1
else:
print("FAIL", question)
print(f"score {score}/{len(TESTS)}")
اول خودت امتحان کن؛ جوابهای دیگر هم قبول است. برگرد و خودت حلش کن