بتل کد Battle Codeدارد باز می‌شود…

جواب «Track A: Score the Bot» چالش 2 از جلسه‌ی 79

سؤال

مسیر الف. ربات جلسه‌ی ۷۸ را روی آزمون پنهان اجرا کن. برای هر آزمون، جواب bot را با classify (همان جلسه‌ی ۷۶) دسته‌بندی کن و چاپ کن، بعد شمارش هر دسته (ترتیب الفبا) و درصد موفقیت = (right + right refusal) ÷ کل. خروجی:
right: What happens in evaporation?
right: How are clouds made?
right: What falls from heavy clouds?
right refusal: When did our school open?
over-cautious: What is a cloud made of?
over-cautious 1, right 3, right refusal 1
success 4/5 = 80%

راهنما

kind = classify(bot(question), expected)؛ counts[kind] = counts.get(kind, 0) + 1. درصد: good = counts.get("right", 0) + counts.get("right refusal", 0) و 100 * good / len(HIDDEN_TEST).

یک جواب درست

STOP = {"the", "a", "an", "is", "are", "and", "or", "of", "in", "on", "to", "it", "what", "how", "why", "does", "do", "when"}
CHUNKS = [
    "Evaporation is when the sun heats water and it turns into vapour.",
    "Condensation is when vapour cools and forms clouds.",
    "Precipitation is rain or snow falling from heavy clouds.",
]
HIDDEN_TEST = [   # (question, key word or None)
    ("What happens in evaporation?", "vapour"),
    ("How are clouds made?", "cools"),
    ("What falls from heavy clouds?", "rain"),
    ("When did our school open?", None),
    ("What is a cloud made of?", "vapour"),
]


def tokens(text):
    words = []
    for w in text.lower().split():
        w = "".join(ch for ch in w if ch.isalpha())
        if w and w not in STOP:
            words.append(w)
    return words


def jaccard(a, b):
    a, b = set(a), set(b)
    if not a or not b:
        return 0
    return len(a & b) / len(a | b)


def bot(question):
    scores = [jaccard(tokens(question), tokens(c)) for c in CHUNKS]
    if max(scores) == 0:
        return "I don't know."
    return CHUNKS[scores.index(max(scores))]


def classify(reply, expected):
    refused = reply == "I don't know."
    if expected is None:
        return "right refusal" if refused else "hallucinated"
    if refused:
        return "over-cautious"
    return "right" if expected in reply.lower() else "wrong"


counts = {}
for question, expected in HIDDEN_TEST:
    kind = classify(bot(question), expected)
    counts[kind] = counts.get(kind, 0) + 1
    print(f"{kind}: {question}")
print(", ".join(f"{k} {counts[k]}" for k in sorted(counts)))
good = counts.get("right", 0) + counts.get("right refusal", 0)
print(f"success {good}/{len(HIDDEN_TEST)} = {100 * good / len(HIDDEN_TEST):.0f}%")

اول خودت امتحان کن؛ جواب‌های دیگر هم قبول است. برگرد و خودت حلش کن