Sample 36

Evaluation Instructions

Important: Models were tasked with extracting evidence from documents. Some outputs may be low quality and should be scored accordingly.

Your task: Compare the model-generated prediction (right panel) against the ground truth criteria (left panel).

Clean view: Shows formatted, readable text for easy comparison
Diff view: Highlights differences between ground truth and prediction
Raw view: Shows original unformatted text as it appears in the dataset

Evaluation scale (0-4):

0 = No match: Prediction is unrelated or contradicts the target criteria
1 = Poor match: Minimal overlap with ground truth, major errors or omissions
2 = Partial match: Some key elements captured, but notable gaps or inaccuracies
3 = Good match: Most elements correct, minor differences or formatting issues
4 = Excellent match: Essentially identical content, fully correct capture

Consider both content accuracy and completeness. Some predictions may be technically "correct" but incomplete or out of order.

Ground Truth

INCLUSION CRITERIA

  1. For VMMC clients:are at least 18 years of age or over
  2. Possession of own phone at enrollment
  3. Willing to respond to daily text
  4. Provides contact details (phone, address)
  5. Undergoes surgical MCWilling to follow NDoH VMMC protocolsNo interoperative AEInformed consent
  6. Receives confirmed 2wT enrollment text.

EXCLUSION CRITERIA

  1. For VMMC clients:Not meeting above requirements
  2. Men without cell phones
  3. Men who chose Pre
  4. Pex device-based VMMCInter-operative AE during VMMCNo informed consent

Prediction

Amended Eligibility Criteria (Version 9)

For VMMC Clients:

 Are at least 18 years of age or older
 Possess a functioning cell phone at enrollment
 Are willing to respond to daily text messages
 Provide contact details (phone number, address)
 Undergo surgical VMMC
 Are willing to follow NDoH VMMC protocols
 Have not experienced an interoperative AE during VMMC
 Provide informed consent
 Receive a confirmed 2wT enrollment text message

For Healthcare Workers/Key Informants:

 Are employees at the site/VMMC providers at the site or NDoH VMMC district level
 Are at least 18 years of age or older
 Provide healthcare services to patients as part of the VMMC programs
 Are able to provide written informed consent
 Are willing to participate
 Are willing to be recorded

EXCLUSION CRITERIA

  1. For VMMC Clients:
  2. Not meeting the above requirements
  3. Men without cell phones
  4. Men who chose PrePex device-based VMMC
  5. Inter-operative AE during VMMC
  6. Lack of informed consent
  7. For Healthcare Workers/Key Informants:
  8. Not willing to participate
  9. Not willing to be recorded
  10. Lack of informed consent