I'm wondering about submission_example_valid.json

Discussion

Home Discussion Board PST-KDD-2024 I'm wondering about submission_example_valid.json

smki

I'm wondering about submission_example_valid.json

posted in PST-KDD-2024

2024-04-11 11:37

I was investigating the details about json files about submission and validation data, and I found that the number of paper ids associated with "references" keys in`paper_source_trace_valid_wo_ans.json` is different from the number of papers associated with the paper ids in `submission_example_valid.json`.
How should we understand this difference?

---

For example,
let's say we have a paper that we need to detect the source paper(s) as follows:
```
"_id": "612c4c285244ab9dcbca22ce",
"title": "GLocal-K - Global and Local Kernels for Recommender Systems."
```
In `paper_source_trace_valid_wo_ans.json`, we have 10 references below.
```
"references": [
"599c798d601a182cd264a97a",
"558b1398e4b031bae1fb645b",
"599c78a2601a182cd25e05b1",
"5c8e25eb4895d9cbc6ca0c11",
"5d79a81b47c8f76646d446be",
"5736977f6e3b12023e66681b",
"5e5e192893d709897ce5694d",
"5da83c8447c8f766460cc39e",
"5e5e18ad93d709897ce2654c",
"57a4e91dac44365e35c9859c"
]
```

On the other hand, in `submission_example_valid.json`, we have 13 slots, in which we need to fill with 0 to 1 values.

```
    "612c4c285244ab9dcbca22ce": [
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0,
        0
    ],
```

Here, we have two numbers; 10 and 13.
I think this is weird.
I guess in `submission_example_valid.json` there should be only 10 slots, since we are supposed to predict a probability for each paper.
IF these two number (10 and 13) are different, how can we know the paper correspondences assosiciated with the same paper id (in this case, "612c4c285244ab9dcbca22ce")?
I found the same errors in quite a few cases.
Please tell us how to interpret this difference.
Thank you!

23 comments

OAG-Challenge Team

Fanjin Zhang reply nio

2024-04-17 04:13

Reply

0

Reference index is non-contiguous after parsing. Invalid entries can be regarded as 0. Please refer to baseline codes for details: https://github.com/THUDM/paper-source-trace/blob/main .

nio reply Fanjin Zhang

2024-04-16 15:48

Reply

0

However, the final n_refs is 78, but the corresponding parsed reference paper is only 70. n_refs is non-contiguous, are you using the value of n_refs to generate the padding values for submission_example_valid.json, e.g. range(n_refs)?

OAG-Challenge Team

Fanjin Zhang reply nio

2024-04-16 03:51

Reply

0

Using the final ``n_refs`` field as the number of references should work.

nio reply Fanjin Zhang

2024-04-15 14:32

Reply

0

I've tested your code, but the data still doesn't correspond to the paper id 5ff682abd4150a363cbaa623, the number of references parsed with your code is only 70, of which 9 references are filtered out due to the absence of the attribute ref.analytic, what should I do?

OAG-Challenge Team

Fanjin Zhang reply nio

2024-04-15 05:30

Reply

0

```python from bs4 import BeautifulSoup cur_pid = paper["_id"] file = cur_pid + ".xml" f = open(file, encoding='utf-8') xml = f.read() bs = BeautifulSoup(xml, "xml") f.close() references = bs.find_all("biblStruct") bid_to_title = {} n_refs = 0 for ref in references: if "xml:id" not in ref.attrs: continue bid = ref.attrs["xml:id"] if ref.analytic is None: continue if ref.analytic.title is None: continue bid_to_title[bid] = ref.analytic.title.text.lower() b_idx = int(bid[1:]) + 1 if b_idx > n_refs: n_refs = b_idx ```

nio reply Fanjin Zhang

2024-04-15 04:26

Reply

0

You can check that the paper id is 5ff682abd4150a363cbaa623, the corresponding xml parses down to 79 references, while the submission_example_valid.json file has only 78 elements corresponding to that paper id.

nio reply Fanjin Zhang

2024-04-14 15:56

Reply

0

Can you give a code for xml data processing? Because I parsed the number of references based on the xml, and there are 110 data references that don't correspond to submission_example_valid.json, or can you check parsing paper_source_trace_valid_wo_ans.json and submission_ example_valid.json to see if the number of references is the same?

smki

2024-04-12 06:18

OAG-Challenge Team

Fanjin Zhang

2024-04-12 04:09

zxc123cc

2024-04-11 11:41