|Year : 2021 | Volume
| Issue : 2 | Page : 117-124
Social versus scientific-driven evidence-based orthodontics: Prediction of correlation among scientometric, altmetric, and level of evidence
Rajkumar Maurya1, Sanjeev Datana2, Sapna Verma3, Sanjay Manohar Londhe4, Sujit Kumar Bhandari5
1 Department of Orthodontics and Dentofacial Orthopaedics, Army Dental Centre (R & R), Delhi Cantt, New Delhi, India
2 Department of Orthodontics and Dentofacial Orthopaedics, Armed Forces Medical College, Pune, Maharashtra, India
3 Department of Library Science, University of Delhi, New Delhi, India
4 Department of Orthodontics, Army Dental Centre (R & R), Delhi Cantt, New Delhi, India
5 Department of Dental Surgery, Armed Forces Medical College, Pune, Maharashtra, India
|Date of Submission||01-Jun-2020|
|Date of Acceptance||18-Jun-2020|
|Date of Web Publication||17-Sep-2021|
Department of Orthodontics and Dentofacial Orthopaedics, Government Dental Centre, New Delhi - 110 010
Source of Support: None, Conflict of Interest: None
Aim: The present study was conducted to analyze the correlation between citations, altmetric score, and level of evidence of top prestigious orthodontic journals.
Materials and Methods: Data mining of selected journals American Journal of Orthodontics and Dentofacial Orthopedics (AJODO), European Journal of Orthodontics (EJO), and Angle Orthodontist (AO) was carried out using the Dimensions and the Webometric Analyst 4.1. The scores were correlated against the strength of study grades, which were interpreted from the GRADE criteria. Pearson's correlation test was used to assess the data statistically using the SPSS software version 21.
Results: AJODO (3786) produced maximum publications followed by AO (1515) and EJO (1135). Most of the publications of AJODO (3654) were under “closed access,” whereas AO (194) provided the maximum of its publications under open access in comparison to EJO (151) and AJODO. It was also found that “Clinical Sciences” category had the maximum number of publications in all. The AO had maximum “mean” of field citation ratio (FCR) and relative citation ratio (RCR) (Mean FCR = 1.78/Mean RCR = 1.68) and AJODO received maximum total attention on social platform. The EJO showed the highest Pearson correlation value of 0.39 between citations and Altmetric, whereas AJODO showed the highest value of 0.16 for citations and grade of evidence.
Conclusion: The present study found that the Altmetric score and level of evidence among all the three journals showed slight positive correlation by EJO and weak correlation by AJODO and AO in relation to citation and Altmetrics. However, weak positive correlation by AJODO and EJO in relation to citations and grade of evidence. It was found that only AO showed positive weak correlation between Altmetric score and grade of evidence.
Keywords: Altmetrics, evidence-based orthodontics, orthodontics, scientometrics
|How to cite this article:|
Maurya R, Datana S, Verma S, Londhe SM, Bhandari SK. Social versus scientific-driven evidence-based orthodontics: Prediction of correlation among scientometric, altmetric, and level of evidence. J Dent Def Sect. 2021;15:117-24
|How to cite this URL:|
Maurya R, Datana S, Verma S, Londhe SM, Bhandari SK. Social versus scientific-driven evidence-based orthodontics: Prediction of correlation among scientometric, altmetric, and level of evidence. J Dent Def Sect. [serial online] 2021 [cited 2022 May 21];15:117-24. Available from: http://www.journaldds.org/text.asp?2021/15/2/117/326221
| Introduction|| |
The role of scientific literature has immensely grown in the era of evidence-based health-care field including orthodontics. Today, there are millions of scientific data being published every year in the form of case reports, case series, cross-sectional studies, reviews, prospective, and randomized controlled trials. The abundant proof of evidence has led the debate of conflicting results on the similar research questions which can be perceived as nonconclusive meta-analysis and systematic reviews.
Evidence-based health care has proposed different tools to find out the best among the rest from the available evidence. The process of measuring the evidence from published articles goes from bibliometrics, i.e., study of quantitative aspects of recorded information in print, scientometrics, i.e., area which deals with the quantitative analyses of scientific information with an aim to understand the scientific research mechanism, informetrics, i.e., is qualitatively analysis of the information, webometrics, i.e., deals with the quantitative aspects of websites according the particular subject and altmetrics, i.e., metrics which plays a prime role in analyzing the impact of the research based on the social media. It is complementary to the traditional metrics and helps in retrieve the overall impact of research. The tools for scholarly output quantification are impact factor which measure the frequency with which the average article in a journal has been cited), citations, H-Index, Cite score. Eugene factor (Impact factor) is one of the earliest methods of determining the impact of the scholarly output which is based on the calculation of the number of citations received by a manuscript published in the journal. The “H” index, on the other hand, determines the scientific impact of researcher and Institutions itself.
With the development of Artificial Intelligence and web-based technology, the role of social media platforms such as “Facebook,” “Twitter,” “LinkedIn,” “Research Gate” “Wikipedia,” Blogs, News, and Media, have increased in the dissemination of research works in the past decade. These platforms have the advantage of dissemination of evidence to the increased number of readers; however, carries inherent bias potential of being influenced by authors and publishers of the journals for the sake of personal and commercial interest. The dissemination of scientific knowledge through these social media platforms have led to the foundation of web-based metrics such as altmetrics, impact story, plum analytics to assess the impact of the scholar and their contents based on their likes, shares, tweets, views comments, bookmarks, blogs, and recommendations.
Though not very old, these web-based metrics have found their comparable space in the scientific literature as like other impact measuring index and their badges are shown at the reputed publishers, abstract and indexing services such as Elsevier, Wiley and Blackwell, Taylor and Francis, Springer and many more. The significant differences between “Impact Factor” and “Web-based metrics” are its out-reachability and access to the apparent larger number of people which may not be restricted to only academic world. Altmetrics is the one of the form of webometrics which studies and uses social activity in online environments to measure the scientific quality. Altmetrics are quicker in accumulation in comparison to citations.
With the growing use of web-based metrics in a couple of years, few researchers have tried to find out the correlation between citations and web metrics. Knight and Barbic et al. have shown the very weak correlation of citations and altmetric scores in the field of organ transplantation and emergency medicine, respectively., Kolahi et al. conducted an almetric score study on the British Dental Journal and reported the correlation of Altmetric and citations in 2014 and 15 consecutively., To the best of our knowledge, only one study have been reported in orthodontics literature by Livas and Delli, who have reported the Altmetric score comparison of top-cited orthodontic journals and found no correlation between citations and altmetrics scores; however, the authors recommended the efforts to increase the constructive online presence of orthodontic journals to be reinforced for better dissemination among scholars and nonscholars. Since, American Journal of Orthodontics and Dentofacial Orthopedics (AJODO), European Journal of Orthodontics (EJO) and Angle Orthodontist (AO) are the one of the most influential scholarly source of evidence in orthodontic literature in present time, this study was conducted to assess the bibliometric, scientometric and altmetric data and to evaluate the correlation between citation score, altmetric score, and grade of evidence of these top three high impact factor orthodontic journals.
| Materials and Methods|| |
The present study was carried out in collaboration with the orthodontic and library and Information Science Department of University as Inter-Departmental Project. The procedure for data mining and extraction was done on March 19, 2019 and described as below [Figure 1].
- The top three journals in orthodontics subject with maximum impact factor on March 15, 2019, as reflected by the Scimago Journal and Country Ranking, included AJODO, EJO and AO, were considered for the research study
- Dimensions is a bibliographic database (adding patents, clinical trials, policy documents, and grants) produced by Digital Science and Research Solutions Inc., UK launched in January 2018 (Dimensions. ai), was used to extract the data of considered journals. Search query had been performed in the Dimensions in three parts as: (i) Criteria: Text-”American Journal of Orthodontics and Dentofacial Orthopedics” in full data; Publication Year was 2018 or 2017 or 2016 or 2015 or 2014 or 2013 or 2012 or 2011 or 2010; Source Title was “American Journal of Orthodontics and Dentofacial Orthopedics.” (ii) Criteria: Text-”European Journal of Orthodontics” in full data; Publication Year was 2018 or 2017 or 2016 or 2015 or 2014 or 2013 or 2012 or 2011 or 2010; Source Title was “European Journal of Orthodontics.” (iii) Criteria: Text – “The Angle Orthodontist” in full data; Publication Year was 2018 or 2017 or 2016 or 2015 or 2014 or 2013 or 2012 or 2011 or 2010; Source Title was The Angle Orthodontist.” The results received from each search query had been downloaded in Excel sheet for further analysis [Table 1] and [Table 2].
- The digital object identifier of all the top 500 articles from each journal was collected and run in the Webometric Analyst 4.1 (free window based program for altmetrics analysis and download data by using web Application programming interface (API)) to collect the distribution of online Attention on social media platforms, including academic (Mendeley, CiteULike, Wikipedia) and nonacademic (Facebook, twitter, and G-Plus) platforms, received by these journals [Table 3]
- Moreover, field citation ratio (FCR) which indicates the relative citation performance of a publication when compared to similarly-aged articles in its subject area and relative citation ratio (RCR) indicating the relative citation performance of a publication when comparing its citation rate to that of other publications in its area of research was tabulated in [Table 4] for top 100 publications of each journal to compare the FCR and RCR for all the considered journals for the study [Figure 2], [Figure 3], [Figure 4]
- The Altmetric Scores, Citations, for the time period of 2010–2018 was extracted of the top 500 articles of each journal form the downloaded excel sheet using dimensions [Figure 1]. Along with Altmetric scores and Citations, the level of evidence was graded with the reference from Grading of Recommendations Assessment, Development, and Evaluation (GRADE) criteria and rated accordingly. Grade 1 was considered as the strongest evidence published such as any Meta-analysis on randomized controlled trials, and subsequently, Grade 4 was considered as the weak evidence published such as case report and narratives. Grade 2a and 2b were differentiated based on the trials and prospective study on humans and laboratory based, respectively
- Finally, the data were statistically evaluated using the SPSS software version 21 (IBM Inc., Chicago, Illinois, USA). The Pearson correlation test was used to evaluate the correlation between different variables. Coefficient correlation (r) <0.3 was considered as weak association, 0.3–0.5 as slight association, 0.5–0.7 as moderate association, 0.7–0.9 as high association, and >0.9 as very high [Table 5].
|Table 4: Platforms provided Altmetric Attention Scores for the Selected Journals Articles|
Click here to view
|Table 5: Citations, relative citation ratio, field citation ratio, and altmetric scores of top 100 publications of selected journals for 2010-2018|
Click here to view
| Results|| |
[Table 1] represents the year wise publications of the selected journals. It was observed that AJODO produced maximum publications during the considered time period as well as each year followed by the AO. [Table 2] provides the details of types of access to the publications of journals. It was observed that most of the publications of AJODO were under “closed access,” whereas the AO provided maximum of its publications under open access.
[Table 3] revealed the attention received over various platforms for the selected journals and found that AJODO received maximum attention followed by the EJO. It was also found that the maximum attention was received on Mendeley. The nonacademic platforms i.e., Tweeter and Facebook also received high attention for these journals.
[Table 4] represents the citations, RCR, FCR and altmetric scores of top 100 publications of the selected journals. AJODO has topped the list for all the parameters. [Table 5] represents the total citations, altmetric score, and level of evidence among all the three journals and showed slight positive correlation by EJO and weak correlation by AJODO and AO in relation to citation and Almetric. However, weak positive correlation by AJODO and EJO in relation to citations and grade of evidence. It was also found that only AO showed positive weak correlation between Altmetric score and grade of evidence, the rest two journals showed weak negative correlation [Table 6]. Field of research of publications among the selected journals has been shown in Appendix I with their relative FCR and RCR mean.
| Discussion|| |
In the era of information technology, there is the burgeoning of research or scholarly publications in the field of orthodontics. AJODO, EJO, and AO are the premier publication of orthodontics and dentofacial orthopedics, contributing significantly to evidence-based health care. The present article was aimed to assess the impact of these journals and their scholarly output at the academic and nonacademic platform from 2010 to present. Since, there is an abundance of scholarly publications in the field of medical sciences which is ever increasing year by year, identification of the relevant and scholarly output has become imperative to distinguish the strong evidence from predatory publications. Bibliometrics and scientometrics methodologies have been used to quantify the impact of any journal by using tools such as citations, H index, and impact factor, traditionally. However, these traditional tools have the limitations of slow accumulation, self-citation, and limited to academic environment. Hence, to assess the quality of scholarly publications in a true sense, and over-come restrictions of citation-based metrics, Altmetrics and Plum analytics were proposed to measure the attention received by the research articles over various academic and nonacademic platforms.
The present study was carried out to measure the impact of three prestigious orthodontic journals at academic and nonacademic platform by correlating the citations, altmetric score, FCR, and RCR as well as the correlation between altmetric scores, Citations, and Grade of Evidence. “American Journal of Orthodontics” (AJODO) is an official publication of “American Association of Orthodontist” being published since 1915 and has a 5-year impact factor of 2.201 according to present Journal Citation Report (JCR) published by Thomson Reuters. The EJO is an official publication of European Orthodontic Society being published since 1979 having a 5-year impact factor 2.033 as per JCR. Finally, the “Angle Orthodontist” is the official publication of the “Edward H. Angle Society of Orthodontists” being published since 1931 having impact factor 1.30 as per JCR. In light of increasing interest in evidence-based orthodontics, the actual correlation of citations and altmetric scores along with the grade of evidence have become mandatory to filter predatory scholarly output from authentic research data.
The present study used citations, FCR, RCR, and altmetric scores to compare with the grade of evidence as citations are used to generate the impact factor of any scholarly journal. The recent boom of the use of social media has increased the impact of altmetric contribution toward the strength of scholarly output through the social media. The main difference between a citation and an altmetric score is the scientific and nonscientific platform for dissemination of scholarly data, respectively. Altmetrics explores the data from three main platforms: social media such as Twitter, Facebook, Google+, Pinterest and blogs, traditional print media such as mainstream news agencies such as The Wire, New York Times and science-specific such as New Scientist, Health Magazines and thirdly online reference managers such as Mendeley and Zotero. The citations have an inherent problem of delayed accumulation over a large period of time due to the cycle of research studies in comparison to the immediate change in Altmetric score at social media platforms. Hence, the present study tried to compare their correlation to each other and grade of evidence. The quantification strength of evidence in the present study has followed the reference of GRADE criteria where Grade 1 was considered as the most strong evidence published such as any meta-analysis on randomized controlled trials, and subsequently Grade 4 was considered as the weak evidence published such as case report and narratives. Grade 2a and 2b were differentiated based on trials and prospective study on humans and laboratory based respectively. The data was mined using DIMENSIONS' (Digital Science and Research Solutions Inc, UK) to carry out the study. It is a research platform, where the research can be discovered and dynamically accessed. It has linked-research data platform that re-imagines the way research can be discovered, accessed and analyzed. It provides access to 140 million grants, publications, clinical trial, and policy documents with four billion connections. Catherine reported the ability of DIMENSIONS to retrieve even the most difficult search strategies in relation to the mining of data. Its products such as Altmetrics have been very instrumental in exploring the social impact of research output. Hence, the present study used the DIMENSIONS for the conduct of the present study.
The present study also used Webometric Analyst 4.1 (Webometric Analyst, Statistical Cybermetrics Research Group, University of Wolverhampton, UK) to collect the distribution of the online attention. Patel and Parmar conducted the evaluation of seven All India Institute of Medical Sciences around country through the webometrics method in 2018 and showed that All India Institutes of Medical Science had poor webometrics indicating toward their low presence on the web pages such as Google page rank and rich files. Iroaganachi et al. reported the Webometric Analysis of Scopus Indexed Publications from 2008 to 2014 and found it as a very effective tool in scientometric studies. The present study also found above tool very effective in the extraction of distribution of social data.
Hutchins B reported the significance of “RCR” and “FCR” as an alternative method to Journal “impact factor” to assess the scholarly output of the journals. The present study also used these two citation analysis method to find out the impact of three journals. The “RCR” indicates the relative citation performance of an article in comparison to citation rate of the other articles in its area of research. A value of more than 1.0 shows a citation rate above average. Similarly, “FCR” is an article-level metric that indicates the relative citation performance of an article, when compared to similarly-aged articles in its subject area. A value of more than 1.0 indicates higher than an average citation when defined by for Subject Code, publishing year and age. The RCR is calculated for all PubMed publications which are at least 2 years old. The present study found that in all the three selected journals, clinical sciences top the list and on analysis it was found that AJODO have maximum total relevant publications, whereas the AO has maximum mean of FCR and RCR for the Clinical Sciences.
Geng et al. and Tarazona et al. conducted a bibliometric study on the top 100 cited articles from 1975 to 2011 and 1946–2012 in orthodontics., Hui et al. reported that since 1975, the articles cited 89–545 times mainly originated from the United States, and the overwhelming majority of articles were clinical. The most common study design was the case series, 40 articles were classified as level 4 and 12 as level 5 evidence. Tarazona et al. also reported that maximum cited article were from the United States. Chatterjee et al. bibliometric report of three reputed orthodontic journals from 2008 to 17 found that majority of articles published in AO (93%), AJODO (68.5%), and Journal of Indian Orthodontic Society (55.3%) were original research and nearly 92.5% contribution was in AJODO, 93.3% in AO, and 96.6% in JIOS were from educational Institutes. Present study also confirms similar Scientometrics findings that AJODO contributes 60% of the research articles as compared to other types of publications, whereas EJO and AO both contributed 85% of the research articles publications during the considered time period of the study. The present study also confirms similar findings.
Bilgiç et al. conducted scientometrics study on three different orthodontic journals at a 5-year interval and reported the contribution of the European Union was more than Asian or another continent. Their findings were exactly opposite to Baumgartner et al. who reported the rate of articles from the nonEuropean origin was twice higher than those from other origins. The difference could be due different citation databases and increasing trends of European database citation of Science Citation Index (SCI) and (SCI-expanded). Gorge Kanavakis et al. reported increased research publication in EJO in comparison to case reports and series in AJODO and AO. The present study also found the similar scientometric data distribution.
Altmetrics are nontraditional metrics proposed as an alternative or complement to conventional impact metrics such as impact factor and citations. Bansal et al. reported significant little overlap between very highly cited papers and those that received the highest Altmetric scores of the top hundred article published in 2016. Huang, et al. also reported correlation comparison between Altmetric Attention Scores and citations for six PLOS journals and mentioned the significant positive correlation between them. As far as Dentistry in concerned, Kolahi et al. conducted one of the first Almetric score correlation studies on the British Dental Journal and reported correlation in the consecutive years., Similarly, only one study have been reported in theliterature by Livas and Delli, who studied Altmetric score comparison of top-cited orthodontic journals and reported no correlation between citations and Altmetrics scores; however, the authors recommended the efforts to increase the constructive online presence of orthodontic journals need to be reinforced for better dissemination of research data among scholars and nonscholars. The present study showed slight positive correlation by EJO and weak correlation by AJODO and AO in relation to citation and Almetric, However weak positive correlation by AJODO and EJO in relation to citations and grade of evidence. It was also found that only AO showed positive weak correlation between Altmetric score and grade of evidence, rest two journals showed weak negative correlation. The positive correlation between citation and Altmetrics in the present study is contradictory in comparison to previously reported by Livas and Delli. The difference could be due to different citation database, i.e., Scopus versus DIMENSIONS. It was also concluded that the Altmetric score increases over the period of time as like citations and a 2-year difference between the present study and other only Altmetric based may have found the difference accordingly.
The citation and Altmetrics score is likely to get changed in near future due to dynamic change in web-based statistics. With the ever-growing increase in scientific publications numbers as well as web-based accessibility of academic and nonacademic social media platform, Altmetric score is expected to play a prime role in the dissemination of scholarly publications out to social platforms including academic and nonacademic. The citations and Altmetric may also help in understanding the impact of the strength of the study.
| Conclusion|| |
The present study found that the altmetric score and level of evidence among all the three journals showed slight positive correlation by EJO and weak correlation by AJODO and AO in relation to citation and altmetric. However weak positive correlation by AJODO and EJO in relation to citations and grade of evidence. It was also found that only AO showed positive weak correlation between altmetric score and grade of evidence, rest two journals showed weak negative correlation.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Shema H, BarIlan J, Thelwall M. Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics. J Assoc Inform Sci Technol 2014;65:1018-27.
Knight SR. Social media and online attention as an early measure of the impact of research in solid organ transplantation. Transplantation 2014;98:490-6.
Barbic D, Tubman M, Lam H, Barbic S. An analysis of altmetrics in emergency medicine. Acad Emerg Med 2016;23:251-68.
Kolahi J, Khazaei S. Altmetric: Top 50 dental articles in 2014. Br Dent J 2016;220:569-74.
Kolahi J, Iranmanesh P, Khazaei S. Altmetric analysis of 2015 dental literature: A cross-sectional survey. Br Dent J 2017;222:695-99.
Livas C, Delli K. looking beyond traditional metrics in orthodontics: an altmetric study on the most discussed articles on the web. Europ J Orthodontics 2017;40(2):1-7.
Mukaka MM. Statistics corner: A guide to appropriate use of correlation coefficient in medical research. Malawi Med J 2012;24:69-71.
Braam RR, Moed HF, v. Raan AFJ. Mapping of science by combined co-citation and word analysis. I: Structural aspects. Journal of the American Society for Information Science 1991;42:233–51.
Bansal SK, Banshal SK, Basu A, Singh VK, Muhuri PK. Scientific vs. Public Attention: A Comparison of Top Cited Papers in Wos and Top Papers by Altmetric Score. Erdt M, Raamkumar SA, Rasmussen E, Theng Y. Springer Singapore, editors. Altmetrics for Research Outputs Measurement and Scholarly Information Management: AROSIM; 2018. p. 81-95.
Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al
. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008;336:924-6.
Catherine W. Dimensions from digital science. Insights 2018;31:33.
Patel HA, Parmar SD. Webometrics study of All India Institutes of Medical Sciences. J Adv Lib Sci 2015;2:12-7.
Iroaganachi MA, Ilogho JE, Itsekor VO, Osinulu I. Research output and sustainable development: Webometric analysis of scopus indexed publications 2008-2014. Lib Philosophy Practice e-J 2016:13:1479.
Hutchins BI, Yuan X, Anderson JM, Santangelo GM. Relative citation ratio (RCR): A new metric that uses citation rates to measure influence at the article level. PLoS Biol 2016;14:e1002541.
Geng G, Yan W, Shao P. The 100 top-cited articles in orthodontics from 1975 to 2011. Angle Orthod 2013;83:491-99.
Tarazona B, Dominguez RL, Gallardo VP, Arroyo AA, Vidal-Infer A. The 100 most-cited articles in orthodontics: A bibliometric study. Angle Orthod 2018;88:785-96.
Hui J, Han Z, Geng G, Yan W, Shao P. The 100 top-cited articles in orthodontics from 1975 to 2011. Angle Orthod 2013;83:491-9.
Chatterjee S, Mohanty P, Sahoo N, Gowd S, Srinivas B, Gojja S. Bibliometric study of three journals of orthodontics: A comparative analysis of 10 years. J Indian Orthod Soc 2018;52:174-8. [Full text]
Bilgiç F, Küçük EB, Sözer ÖA, Ay Y, Kaya A, Kaptaç M. Analysis of six orthodontic journals in science citation index and science citation index expanded: A bibliometric analysis. Turk J Orthod 2018;31:73-8.
Baumgartner S, Pandis N, Eliades T. Exploring the publications in three major orthodontic journals: a comparative analysis of two 5-year periods. Angle Orthod 2014;84:397-403.
Kanavakis G, Spinos P, Polychronopoulou A, Eliades T, Papadopoulos MA, Athanasios E. Athanasiouc. Orthodontic journals with impact factors in perspective: Trends in the types of articles and authorship characteristics Am J Orthod Dentofacial Orthop 2006;130:516-22.
Huang W, Wang P, Wu Q. A correlation comparison between Altmetric Attention Scores and citations for six PLOS journals. PLoS One 2018;13:e0194962.
[Figure 1], [Figure 2], [Figure 3], [Figure 4]
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5], [Table 6]