Browse Source

Update README.md

main
Mahta Fetrat 2 weeks ago
parent
commit
ed508c9495
No account linked to committer's email address
1 changed files with 96 additions and 1 deletions
  1. 96
    1
      README.md

+ 96
- 1
README.md View File

@@ -1 +1,96 @@
# HomoRich-G2P-Persian
# HomoRich: A Persian Homograph Dataset for G2P Conversion

HomoRich is the first large-scale, sentence-level Persian homograph dataset designed for grapheme-to-phoneme (G2P) conversion tasks. It addresses the scarcity of balanced, contextually annotated homograph data for low-resource languages. The dataset was created using a semi-automated pipeline combining human expertise and LLM-generated samples, as described in the paper:
**"Fast, Not Fancy: Rethinking G2P with Rich Data and Rule-Based Models"**.

## Overview
The dataset contains 528,891 annotated Persian sentences (327,475 homograph-focused) covering 285 homograph words with 2-4 pronunciation variants each. Variants are equally represented (~500 samples each) to mitigate bias. The composition blends multiple sources for diversity, as shown below:

<div align="center">
<div style="display: flex; justify-content: center; gap: 20px; margin-bottom: 10px; flex-wrap: wrap;">
<!-- Distribution Plot -->
<div style="text-align: center;">
<img src="https://github.com/MahtaFetrat/HomoRich-G2P-Persian/blob/main/assets/composition-figure.png" width="400"/>
<p style="margin-top: 5px;">Distribution of data sources in HomoRich dataset</p>
</div>
<div style="text-align: center;">
<img src="https://github.com/MahtaFetrat/HomoRich-G2P-Persian/blob/main/assets/composition-table.png" width="362"/>
<p style="margin-top: 5px;">The source for different parts of the HomoRich dataset</p>
</div>
</div>
</div>


### Phoneme Representations:
Persian G2P systems use two common phoneme formats:

- Repr. 1: Used in [KaamelDict](https://huggingface.co/datasets/MahtaFetrat/KaamelDict) and [SentenceBench](https://huggingface.co/datasets/MahtaFetrat/SentenceBench) (compatible with prior studies)
- Repr. 2: Adopted by [GE2PE](https://github.com/Sharif-SLPL/GE2PE) (state-of-the-art model enhanced in this work)

The HomoRich dataset includes both formats for broad compatibility. Below is a visual comparison:

<div align="center">
<div style="display: flex; justify-content: center; gap: 20px; margin-bottom: 10px;">
<div style="text-align: center;">
<img src="https://github.com/MahtaFetrat/HomoRich-G2P-Persian/blob/main/assets/our-repr.png" width="400"/>
<p style="margin-top: 5px;">Repr. 1</p>
</div>
<div style="text-align: center;">
<img src="https://github.com/MahtaFetrat/HomoRich-G2P-Persian/blob/main/assets/ge2pe-repr.png" width="400"/>
<p style="margin-top: 5px;">Repr. 2</p>
</div>
</div>
</div>

---

## Usage
### Loading the Dataset
The dataset is available both on Hugging Face and in this repository:

**Option 1: From Hugging Face**
```python
from datasets import load_dataset
dataset = load_dataset("MahtaFetrat/HomoRich")
```

**Option 2: From this repository**
The dataset files are available in the `data` folder as:
- `part_01.parquet`
- `part_02.parquet`
- `part_03.parquet`

You can access them directly from the [data directory](./data) of this repository.

### Example Use Case: Homograph Disambiguation
```python
TODO
```
---

## Benchmarks
The dataset was used to improve:
1. **Homo-GE2PE** (Neural T5-based model): **76.89% homograph accuracy** (29.72% improvement).
2. **HomoFast eSpeak** (Rule-based): **74.53% accuracy** with real-time performance (30.66% improvement).

See [paper Table 3](#) for full metrics.

---

## License
- **Dataset**: Released under **CC0-1.0** (public domain).
- **Code/Models**: **MIT License** (where applicable).

---

## Citation
```bibtex
TODO: citation to paper arxiv
```

---

### Additional Links
- [Paper PDF](#) (TODO: link to paper)
- [HomoFast eSpeak NG](#) (TODO: link to repo)
- [Homo-GE2PE Model](#) (TODO: link to repo)

Loading…
Cancel
Save