Document Type : Research Paper
Authors
Department of Accounting, Faculty of Management and Accounting , Shahid Beheshti University, Tehran, Iran
Abstract
Given that the number of citations a research work gives to other studies is considered a measure of its scientific credibility, researchers might focus more on the quantity of citations rather than their thematic relevance to their own work. Therefore, this study aims to examine the thematic similarity between the citations and the core topic of the citing research. To achieve this objective, Natural Language Processing (NLP) was employed as one of the most significant approaches for working with textual data in machine learning. After extracting citation data from the web pages of Science Ministry journals from 1393 until 1402(Iranian calendar) using web crawling, the thematic relevance of citations and the core theme of each study were calculated using the all-mpnet model. The results indicate that the thematic cohesion between citations and the core research topic in Iranian journals is at a proper level. Furthermore, the citations by Iranian researchers are primarily to sources published in first and second-quartile journals (Q1) and (Q2), indicating their credibility. Additionally, the most highly cited sources in Iranian accounting and auditing research were identified. The novel methodology and findings of this study can significantly contribute to evaluating citation practices. They can assist journals in more accurately assessing citations, helping them move beyond a sole focus on citation count and instead utilize the degree of citation relevance as an indicator for evaluating citation quality.
Keywords: Bidirectional Encoder Representations from Transformers (BERT), Citation Analysis, Machine learning, Natural language processing, Text mining.
Introduction
The number of citations a paper receives has become one of the most visible proxies for scientific credibility in modern academia. Journals, funding agencies, promotion committees, and even national ranking systems routinely treat citation counts as evidence of impact (Dechow et al., 2020; van Raan et al., 2003; Woolgar, 1991). Yet this very pressure creates a perverse incentive: researchers may inflate reference lists with prestigious but topically distant sources simply to boost perceived legitimacy (Dai et al., 2021; Iqbal et al., 2021).
In this study, we define citation quality through a new lens: thematic cohesion, which refers to the degree of semantic alignment between the core topic of a paper and the titles of every work it cites. Although erroneous and irrelevant citations have been documented even in top-tier international journals (Evans et al., 1990), no large-scale, automated assessment of thematic cohesion has ever been conducted in Iranian accounting and auditing literature.
Using web scraping, Scopus quartile mapping, and state-of-the-art Sentence-BERT embeddings, we address three interrelated questions:
To what extent do Iranian accounting researchers cite journals of established global prestige?
How strong is the thematic cohesion between Iranian papers and the works they cite?
By moving beyond raw counts and making relevance visible, measurable, and enforceable, this study offers editors a practical tool to reward substance over symbolism and provides Iranian accounting scholarship with a roadmap to even greater international influence.
Methodology
This quantitative study adopts an applied research design that combines content analysis with large-scale text mining. The population comprises all references appearing in articles published in Iran’s Ministry of Science-approved accounting and auditing journals between 2014 and 2023. Of the 20 recognized journals, 14 provided full-text HTML archives suitable for automated retrieval, forming the final sampling frame.
Data were collected using custom Python web crawlers built with the Requests and BeautifulSoup libraries. The scripts systematically traversed journal archives, extracted article-level metadata, and downloaded full texts, yielding 3,436 published papers. After excluding 470 articles whose PDFs could not be parsed, 2,966 full texts entered the pipeline. From these, 132,543 raw references were harvested. Four exclusion filters were then applied, including Persian-language references, books, theses, conference papers, and working papers, legal documents and standards, and non-journal items without a DOI or Scopus entry. This rigorous cleaning retained 65,146 foreign journal citations from 2,938 articles.
Each cited journal was matched to its ISSN in the Scopus Source List (2023 release). Quartiles were assigned dynamically by year and subject category, producing the final distribution: Q1 (77.5%), Q2 (15.2%), Q3 (5.4%), and Q4 + unranked (2.0%).
Semantic similarity between every citing-title /and cited-title pair was computed using multilingual Sentence-BERT (sentence-transformers/paraphrase-multilingual-mpnet-base-v2), fine-tuned on more than 100 languages, including Persian and English. Cosine similarity scores were scaled to 0–100%. Inter-coder reliability was established on a random 500-pair subsample manually labelled by two accounting PhD candidates. Embedding stability was confirmed by repeating the analysis with three different seeds; the maximum variation in journal-level means was 0.12 percentage points.
By combining web-scale data acquisition with state-of-the-art NLP, this methodology delivers the first objective, replicable measure of citation relevance ever applied to an entire national accounting literature.
Results
The analysis of 65,146 international journal citations reveals a clear portrait of Iranian accounting scholarship: globally oriented, prestige-conscious, and already practising a respectable thematic discipline. Iranian papers’ citations suggest that 77 percent of the citations are made to Q1 journals. No ministry-approved journal falls below 67% Q1+Q2 combined. The leader, “Professional Auditing Research”, directs 83.1% of its citations to Q1 outlets; even the most locally oriented title, “Governmental Accounting”, still achieves 53.7% Q1.
Across all 65,146 citation pairs, the mean thematic cohesion is 45.4%. The distribution is remarkably symmetrical and bell-shaped, peaking at 45–50%. Only 9% of citations reach the high-to-excellent zone above 55%, while 10% fall into the weak territory below 35%. In other words, Iranian accounting researchers rarely cite completely irrelevant work, but they also rarely achieve perfect topical harmony.
Every single year stays within the 44.7–47.6% corridor. The absence of a strong upward or downward trend suggests that citation culture has matured but not yet entered a phase of deliberate relevance optimisation.
Taken together, the results paint Iranian accounting scholarship as internationally ambitious and methodologically disciplined. The community already cites the right journals; the next frontier is citing the right ideas inside them.
Discussion
The fact that average thematic cohesion plateaus at 45.4%—solidly “good” yet never crossing the 50%, reveals a subtle but systematic gap. Ten percent of all citations drift into the weak zone (<35%). Another 81% hover in the moderate-to-good band, useful but rarely transformative. Only 9% achieve the high-to-excellent alignment that signals a genuine intellectual dialogue. This gap is not unique to Iran. Global studies repeatedly warn that citation inflation, the Matthew Effect, and strategic referencing are widespread (Biglu, 2008; Bornmann & Daniel, 2008; Dai et al., 2021). What makes the Iranian case instructive is the stability of the 45% ceiling across ten years and fourteen journals.
The benefit of thematic cohesion is that it is cheap, instantaneous, and impossible to game. A single Sentence-BERT inference on a standard laptop takes 0.03 seconds per citation pair. An entire manuscript can be scored before the reviewer finishes the first page. Journals that adopt a modest gate would immediately filter out decorative references while preserving author freedom. Early adopters elsewhere have reported reference list shrinkage of 12–18% with no loss of scientific depth (Aljuaid et al., 2021; Anglin et al., 2021).
Conclusion
Iranian accounting and auditing scholarship has reached a pivotal moment. For a decade, the community has looked outward with confidence: 77.5% of all citations now flow to Q1 journals. Yet global engagement must now evolve into a global conversation. The 45.4% average thematic cohesion achieved across 65,146 citations tells editors that most references already serve the argument, but one in ten does not, and only one in eleven truly transforms it.
When Iranian journals lead the shift toward the new article evaluation measure provided in this paper, first, domestic papers will become sharper, more readable, and more persuasive. Second, international reviewers will notice the difference and reward it with faster acceptances and higher citations. The future of Iranian accounting research is therefore not about citing more; it is about citing better.
Keywords
- citation analysis
- natural language processing
- machine learning
- text mining
- Bidirectional Encoder Representations from Transformers (BERT)
Main Subjects
)