Text dataset, JSONL
Native-speaker Q&A
Original question-and-answer pairs on everyday topics such as health, farming, money and schooling, written by native speakers rather than translated. Each document has one metadata row covering language, domain, length, review status and consent.
Request access
This is an example dataset. The figures and rows on this page are synthetic and shown to illustrate the format.
- 19.1
- million tokens
- 30,000
- documents
- 600
- contributors
- 7
- countries
- 95.7 MB
- total size
Million tokens by language
- Indonesian4.9
- Vietnamese2.7
- Hindi2.1
- Thai1.9
- Javanese1.9
- Tagalog1.6
- Bengali1.2
- Sundanese1.0
- Malay1.0
- Cebuano0.9
Million tokens by domain
- daily life4.4
- education3.9
- health3.4
- personal finance2.9
- agriculture2.7
- local government1.9
Metadata columns
One row per sample, stored as Parquet. Files are JSONL.
| Column | Type | What it holds |
|---|
| doc_id | Utf8 | Stable identifier for the document. |
| writer_id | Utf8 | Pseudonymous writer identifier. |
| country | Utf8 | Country where the writer lives. |
| language | Utf8 | Language the document is written in. |
| domain | Utf8 | health, agriculture, personal_finance, education, daily_life or local_government. |
| token_count | Int32 | Approximate length in tokens. |
| reviewed | Boolean | Whether a second native speaker reviewed the document. |
| consent_version | Utf8 | Version of the consent agreement the writer signed. |
| text_uri | Utf8 | Object-storage location of the text file. |
Sample rows
| consent_version | country | doc_id | domain | language | reviewed | size_bytes | text_uri | token_count | writer_id |
|---|
| contributor-consent-v1 | India | doc_d0d90537 | daily_life | Hindi | false | 1915 | s3://datachad-lake/datasets/native-speaker-qa/text/hindi/doc_d0d90537.jsonl | 383 | wrt_0258 |
| contributor-consent-v1 | Vietnam | doc_e27d85d0 | education | Vietnamese | true | 550 | s3://datachad-lake/datasets/native-speaker-qa/text/vietnamese/doc_e27d85d0.jsonl | 110 | wrt_0014 |
| contributor-consent-v1 | Philippines | doc_10539807 | personal_finance | Tagalog | true | 2840 | s3://datachad-lake/datasets/native-speaker-qa/text/tagalog/doc_10539807.jsonl | 568 | wrt_0442 |
| contributor-consent-v1 | Indonesia | doc_4bd3655c | daily_life |
Read the metadata
The metadata is plain Parquet. Filter it with PyArrow, DuckDB or Polars, then load only the files you picked.
import pyarrow.compute as pc
import pyarrow.dataset as ds
metadata = ds.dataset(
"s3://datachad-lake/datasets/native-speaker-qa/metadata/",
format="parquet",
)
rows = metadata.to_table(filter=pc.field("country") == "Indonesia")
print(rows.num_rows, rows.column_names)