بتل کد Battle Codeدارد باز می‌شود…

Step 4: With and Without RAG چالش 4 از جلسه‌ی 72، سخت

بسنج، فرض نکن. برای هر سؤال آزمون، یک بار ask_plain(question) (مدل بدون زمینه) و یک بار ask_rag(question) (با بهترین تکه) را صدا بزن. جواب درست است اگر کلمه‌ی کلیدی آزمون را (با حروف کوچک) داشته باشد. برای هر سؤال plain ✗ rag ✓ و مانند آن را چاپ کن و در آخر دو امتیاز. خروجی:
What is condensation? plain ✗ rag ✓
Why does rain fall? plain ✗ rag ✓
What is evaporation? plain ✗ rag ✓
without RAG 0/3, with RAG 3/3

STOP = {"the", "a", "an", "is", "are", "and", "or", "of", "in", "on", "to", "into", "it", "this",
        "that", "what", "how", "why", "does", "do", "as", "when", "they", "called", "its"}
CHUNKS = [
    "Evaporation is when the sun heats water in seas and lakes and the warm water turns into vapour and rises.",
    "Condensation is when the vapour cools high in the sky and forms clouds.",
    "Precipitation is when the drops in a cloud get heavy and fall as rain or snow.",
    "Transpiration is when plants lose water through their leaves.",
]
GUESSES = {   # what the model "remembers" without notes: confident and wrong
    "What is condensation?": "Condensation is when water turns into ice.",
    "Why does rain fall?": "Rain falls because the sky gets tired.",
    "What is evaporation?": "Evaporation is when water disappears forever.",
}
TESTS = [("What is condensation?", "clouds"), ("Why does rain fall?", "heavy"), ("What is evaporation?", "vapour")]


def tokens(text):
    words = []
    for w in text.lower().split():
        w = "".join(ch for ch in w if ch.isalpha())
        if w and w not in STOP:
            words.append(w)
    return words


def jaccard(a, b):
    a, b = set(a), set(b)
    if not a or not b:
        return 0
    return len(a & b) / len(a | b)


def best_chunk(question, chunks):
    q = tokens(question)
    best, best_score = None, 0
    for i, chunk in enumerate(chunks):
        s = jaccard(q, tokens(chunk))
        if s > best_score:
            best, best_score = i, s
    return best, best_score


def ask_plain(question):
    return GUESSES.get(question, "I don't know.")


def ask_rag(question):
    index, score = best_chunk(question, CHUNKS)
    return CHUNKS[index] if index is not None else "Not in the notes."


def correct(answer, key):
    return False


plain_score = rag_score = 0
for question, key in TESTS:
    print(question)
print(f"without RAG {plain_score}/{len(TESTS)}, with RAG {rag_score}/{len(TESTS)}")

راهنما و جواب این چالش · برگشت به درس جلسه‌ی 72: RAG کوچک (۲): از روی جزوه جواب بده

بیشتر بخوان: تابع در پایتون: آموزش def، پارامتر و return با مثال