Use English AR example and define subcategories

README.md

@@ -72,7 +72,7 @@ in `media/`.
 | | | | |
 |:--:|:--:|:--:|:--:|
 | <a href="https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/AH-AOR.mp4"><img src="assets/previews/AH-AOR.gif" width="190" alt="AH-AOR"></a> | <a href="https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MEA-RC-AR.mp4"><img src="assets/previews/MEA-RC-AR.gif" width="190" alt="MEA-RC-AR"></a> | <a href="https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VTT-UPC.mp4"><img src="assets/previews/DSLP-VTT-UPC.gif" width="190" alt="DSLP-VTT-UPC"></a> | <a href="https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MSA-PLA.mp4"><img src="assets/previews/MSA-PLA.gif" width="190" alt="MSA-PLA"></a> |
-| **AH**<br>`AH-AOR`<br><sub>She asks what the model thought of the lullaby she just sang. She never sang.</sub> | **MEA**<br>`MEA-RC-AR`<br><sub>He asks what to bring “over there”; the video identifies the court's filing hall.</sub> | **DSLP**<br>`DSLP-VTT-UPC`<br><sub>He stops mid-sentence, still thinking. The model should wait, not answer.</sub> | **MSA**<br>`MSA-PLA`<br><sub>He asks the model to put a circuit board on the rack — it has no hands.</sub> |
+| **AH**<br>`AH-AOR`<br><sub>She asks what the model thought of the lullaby she just sang. She never sang.</sub> | **MEA**<br>`MEA-RC-AR`<br><sub>He asks if “this one” suits a five-year-old; only the video identifies the painted wooden elephant.</sub> | **DSLP**<br>`DSLP-VTT-UPC`<br><sub>He stops mid-sentence, still thinking. The model should wait, not answer.</sub> | **MSA**<br>`MSA-PLA`<br><sub>He asks the model to put a circuit board on the rack — it has no hands.</sub> |
 | <a href="https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/ER.mp4"><img src="assets/previews/ER.gif" width="190" alt="ER"></a> | <a href="https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VDMC-SVMR.mp4"><img src="assets/previews/DSLP-VDMC-SVMR.gif" width="190" alt="DSLP-VDMC-SVMR"></a> | <a href="https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MEA-RC-MRR.mp4"><img src="assets/previews/MEA-RC-MRR.gif" width="190" alt="MEA-RC-MRR"></a> | <a href="https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MSA-IC-STIC.mp4"><img src="assets/previews/MSA-IC-STIC.gif" width="190" alt="MSA-IC-STIC"></a> |
 | **ER**<br>`ER`<br><sub>Disappointed after a final-round rejection, he asks whether to keep applying or change direction.</sub> | **DSLP**<br>`DSLP-VDMC-SVMR`<br><sub>The camera faces the road, not him. Left or right depends on noticing that.</sub> | **MEA**<br>`MEA-RC-MRR`<br><sub>Turn 7 of 7. She asks the colour of a vase shown earlier; only a past clip has it.</sub> | **MSA**<br>`MSA-IC-STIC`<br><sub>Late-night chat, she asks who it is. Not Siri, not GPT — its own name.</sub> |
 
@@ -124,25 +124,25 @@ exactly as it appears in the dataset, plus a small re-encode of its clip so the
 browsable without the 28 GB download. For a multi-turn subcategory the clip is the **final**
 turn, the one that gets scored.
 
-| Subcategory | Ability | Turns | Data | Clip |
-|---|---|---|---|---|
-| `AH-AOR` | AH | single | [json](examples/AH-AOR.json) | [144 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/AH-AOR.mp4) |
-| `AH-FGC` | AH | single | [json](examples/AH-FGC.json) | [223 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/AH-FGC.mp4) |
-| `DSLP-VCC-ICC` | DSLP | single | [json](examples/DSLP-VCC-ICC.json) | [393 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VCC-ICC.mp4) |
-| `DSLP-VCC-MDCC` | DSLP | multi | [json](examples/DSLP-VCC-MDCC.json) | [156 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VCC-MDCC.mp4) |
-| `DSLP-VDMC-MVMS` | DSLP | multi | [json](examples/DSLP-VDMC-MVMS.json) | [223 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VDMC-MVMS.mp4) |
-| `DSLP-VDMC-SVMR` | DSLP | single | [json](examples/DSLP-VDMC-SVMR.json) | [169 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VDMC-SVMR.mp4) |
-| `DSLP-VTT-UCR` | DSLP | single | [json](examples/DSLP-VTT-UCR.json) | [269 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VTT-UCR.mp4) |
-| `DSLP-VTT-UPC` | DSLP | single | [json](examples/DSLP-VTT-UPC.json) | [248 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VTT-UPC.mp4) |
-| `ER` | ER | single | [json](examples/ER.json) | [187 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/ER.mp4) |
-| `MEA-RC-AR` | MEA | single | [json](examples/MEA-RC-AR.json) | [323 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MEA-RC-AR.mp4) |
-| `MEA-RC-DR` | MEA | single | [json](examples/MEA-RC-DR.json) | [135 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MEA-RC-DR.mp4) |
-| `MEA-RC-MRR` | MEA | multi | [json](examples/MEA-RC-MRR.json) | [118 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MEA-RC-MRR.mp4) |
-| `MEA-SDI-MIST` | MEA | multi | [json](examples/MEA-SDI-MIST.json) | [420 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MEA-SDI-MIST.mp4) |
-| `MEA-SDI-SPSD` | MEA | single | [json](examples/MEA-SDI-SPSD.json) | [257 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MEA-SDI-SPSD.mp4) |
-| `MSA-IC-MTIC` | MSA | multi | [json](examples/MSA-IC-MTIC.json) | [259 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MSA-IC-MTIC.mp4) |
-| `MSA-IC-STIC` | MSA | single | [json](examples/MSA-IC-STIC.json) | [107 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MSA-IC-STIC.mp4) |
-| `MSA-PLA` | MSA | single | [json](examples/MSA-PLA.json) | [372 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MSA-PLA.mp4) |
+| Subcategory | Definition | Ability | Turns | Data | Clip |
+|---|---|---|---|---|---|
+| `AH-AOR` | Detect missing audio-visual evidence instead of inventing it. | AH | single | [json](examples/AH-AOR.json) | [144 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/AH-AOR.mp4) |
+| `AH-FGC` | Correct a user's false fine-grained claim using visible evidence. | AH | single | [json](examples/AH-FGC.json) | [223 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/AH-FGC.mp4) |
+| `DSLP-VCC-ICC` | Confirm the voice connection at the opening of a dialogue. | DSLP | single | [json](examples/DSLP-VCC-ICC.json) | [393 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VCC-ICC.mp4) |
+| `DSLP-VCC-MDCC` | Confirm a later connection check consistently with prior dialogue. | DSLP | multi | [json](examples/DSLP-VCC-MDCC.json) | [156 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VCC-MDCC.mp4) |
+| `DSLP-VDMC-MVMS` | Update user-relative left/right after the camera mode changes. | DSLP | multi | [json](examples/DSLP-VDMC-MVMS.json) | [223 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VDMC-MVMS.mp4) |
+| `DSLP-VDMC-SVMR` | Interpret user-relative left/right from the current camera view. | DSLP | single | [json](examples/DSLP-VDMC-SVMR.json) | [169 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VDMC-SVMR.mp4) |
+| `DSLP-VTT-UCR` | Answer a completed request despite visual distraction. | DSLP | single | [json](examples/DSLP-VTT-UCR.json) | [269 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VTT-UCR.mp4) |
+| `DSLP-VTT-UPC` | Wait or briefly acknowledge when the user pauses mid-request. | DSLP | single | [json](examples/DSLP-VTT-UPC.json) | [248 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/DSLP-VTT-UPC.mp4) |
+| `ER` | Use facial and vocal emotion cues when answering the request. | ER | single | [json](examples/ER.json) | [187 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/ER.mp4) |
+| `MEA-RC-AR` | Resolve pronouns and gestures to the correct visible referent. | MEA | single | [json](examples/MEA-RC-AR.json) | [270 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MEA-RC-AR.mp4) |
+| `MEA-RC-DR` | Resolve spoken descriptions to the correct visible referent. | MEA | single | [json](examples/MEA-RC-DR.json) | [135 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MEA-RC-DR.mp4) |
+| `MEA-RC-MRR` | Recall an earlier visible referent after it leaves the current view. | MEA | multi | [json](examples/MEA-RC-MRR.json) | [118 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MEA-RC-MRR.mp4) |
+| `MEA-SDI-MIST` | Track a change in the active speaker across turns. | MEA | multi | [json](examples/MEA-SDI-MIST.json) | [420 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MEA-SDI-MIST.mp4) |
+| `MEA-SDI-SPSD` | Identify the main user among competing voices. | MEA | single | [json](examples/MEA-SDI-SPSD.json) | [257 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MEA-SDI-SPSD.mp4) |
+| `MSA-IC-MTIC` | Maintain true model identity across turns despite misleading claims. | MSA | multi | [json](examples/MSA-IC-MTIC.json) | [259 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MSA-IC-MTIC.mp4) |
+| `MSA-IC-STIC` | State the correct model identity in one turn. | MSA | single | [json](examples/MSA-IC-STIC.json) | [107 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MSA-IC-STIC.mp4) |
+| `MSA-PLA` | Handle physical-action requests according to the model's embodiment. | MSA | single | [json](examples/MSA-PLA.json) | [372 KB](https://huggingface.co/datasets/Harland/OmniVChat/blob/main/examples/clips/MSA-PLA.mp4) |
 
 The two `_example_clip*` fields in those files are added here for convenience and are
 not part of the dataset.

examples/MEA-RC-AR.json

@@ -1,28 +1,28 @@
 {
-  "id": "0042fbd5b95b48119e67da1cd920f98d",
+  "id": "7d81ddaa0b924b758e3946d7978f38c6",
   "ability": "MEA",
   "subcategory": "MEA-RC-AR",
-  "video": "media/single_turn/0042fbd5b95b48119e67da1cd920f98d.mp4",
+  "video": "media/single_turn/7d81ddaa0b924b758e3946d7978f38c6.mp4",
   "key_points": [
     {
       "tier": 0,
-      "point": "The response is in Chinese, consistent with the user's language"
+      "point": "The response is in English, consistent with the user's language."
     },
     {
       "tier": 1,
-      "point": "The response correctly understands that \"over there\" and \"there\" refer to the court case-filing hall entrance, and answers regarding the materials required for case filing"
+      "point": "The response correctly resolves the pronouns 'this one' and 'it', identifying that the user is holding a painted wooden elephant (painted elephant / colorful wooden elephant) rather than other items on the stall."
     },
     {
       "tier": 2,
-      "point": "The response provides a specific list of materials required for case filing (e.g., complaint, ID card, evidence materials, etc., listing at least 2 items)"
+      "point": "The response provides a substantive judgment regarding suitability for a five-year-old, affirming the appeal of bright colors and animal shapes to young children."
     },
     {
       "tier": 2,
-      "point": "The response uses a friendly and gentle tone, providing reassurance or encouragement regarding the user's nervousness"
+      "point": "The response mentions safety considerations (such as sharp edges, risk of small parts detaching, or non-toxic paint) to provide practical advice for parents making a purchase."
     },
     {
       "tier": 3,
-      "point": "The response incorporates details visible in the video (e.g., mentioning the document bag in the user's hand, the court environment, etc.), demonstrating perception of the visual content"
+      "point": "The response specifically cites visual details visible in the video (such as bright colors like red, blue, and yellow), demonstrating accurate perception of the frame content."
     }
   ],
   "_example_clip": "examples/clips/MEA-RC-AR.mp4"

examples/clips/MEA-RC-AR.mp4

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9344590761baaad540855909d9f88657f9bfc267af272d752c213ee78165b026
-size 331231
+oid sha256:695c9126f80be853daf2967b087eecfa65c6287d80deea84bb2455bd4b35eaec
+size 276767

examples/index.json

@@ -84,10 +84,10 @@
   "subcategory": "MEA-RC-AR",
   "ability": "MEA",
   "turns": "single",
-  "id": "0042fbd5b95b48119e67da1cd920f98d",
+  "id": "7d81ddaa0b924b758e3946d7978f38c6",
   "json": "examples/MEA-RC-AR.json",
   "clip": "examples/clips/MEA-RC-AR.mp4",
-  "size_kb": 323
+  "size_kb": 270
  },
  {
   "subcategory": "MEA-RC-DR",