Open Access
Issue
SICOT-J
Volume 3, 2017
Article Number 36
Number of page(s) 4
Section Lower Limb
DOI https://doi.org/10.1051/sicotj/2017019
Published online 17 May 2017

© The Authors, published by EDP Sciences, 2017

Licence Creative Commons
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Introduction

International conferences remain an excellent way of disseminating research and furthering evidence-based medicine, thereby potentially influencing clinical practice. Abstracts presented at conferences also provide a forum for debate, critical analysis and discussion, which in itself is a form of peer review. Previous studies have suggested up to 63% of major orthopaedic textbook chapters having included results from abstracts presented at international orthopaedic meetings [1]. However, abstracts in themselves are not always readily searchable postpresentation and this can lead to difficulties in citing any evidence presented at meetings. A peer-reviewed publication therefore remains the gold standard when judging the quality and validity of a study.

The International Society of Orthopaedic Surgery and Traumatology (SICOT) holds an annual international conference that provides a well-known and important forum to share novel research findings. There is therefore great emphasis on conferences such as this in providing a substantial portion of continued medical education for surgeons and allied health professionals. The SICOT Programme Committee is responsible for the selection of podium presentations as part of a strict, blinded and peer-reviewed process. This process has led to podium presentations at this meeting becoming increasingly well regarded, and the knowledge acquired as a result of these presentations is becoming more rapidly incorporated into clinical practice. For this reason, it is essential to question the validity of such podium presentations.

Our aim was therefore to: (1) evaluate the five-year publication rate of podium presentations of SICOT Annual International Conference, (2) to evaluate the consistency of abstracts presented and subsequent consistency with the full peer-reviewed publication and (3) to discuss our findings with other well-known orthopaedic international conferences.

Materials and methods

We reviewed all podium presentations at the 2009 SICOT XXV Triennial World Conference in Prague, Czech Republic. The abstracts from the 2009 SICOT Conference were chosen in order to give at least five years for subsequent publications and as it also allows the methods to fall in line with those used in previous similar studies [14]. All 329 podium presentations published in the 2009 SICOT Conference podium abstract book were included in the study. Poster abstracts were excluded due to a higher scientific value being placed on podium presentations.

The Bhandari methodology previously described was the main strategy used to search for abstracts in PubMed and Medline [1]. We used two separate reviewers to search for abstracts in order to increase reliability. Any abstracts, which were not agreed upon by the two reviewers, were reviewed by the first author in order to come to a majority decision. Comparing the abstract and final publication through similarity of content provided the basis for inclusion. This only comprised a small minority of the overall number.

A sizeable database was created for data input and final analysis. Each study was given a unique identification number and a variety of data were recorded for each study including study design, methodology, statistical analysis and study outcomes. All analysis was done using GraphPad Prism statistical analysis software. A median (range) value was obtained for time to publication and number of centres involved in each study. Journal impact factors were sorted accordingly in the database to rank the highest and lowest impact factor journals in which a publication occurred. The database was sorted by journal title to provide overall journal ranking in terms of number of published abstracts. A similar method was used to rank subspecialties published, type of study, country and associated level of evidence according to the Oxford Centre for Evidence-Based Medicine grading system. A goodness-of-fit test was used to assess correlation between parameters and publication, with publication set as a binary figure of either 1 (indicating publication) or 0 (indicating no publication). Differences in study parameters such as between the presented abstract and a final publication were recorded for analysis. These parameters included methodology, study size, number of centres involved as well as primary and secondary outcomes. A difference would be described as any significant change in these parameters between abstract and final publication. Consistency was therefore measured as a mean percentage with an attributed confidence interval. If there was no eventual publication then only information on the abstract was collected.

Results

Three hundred and twenty-nine abstracts were podium presentations at the 2009 International Society of Orthopaedics and Traumatology (SICOT). From these, 103 (31.3%) were published with a complete report with a median time to publication of 23.4 months (0–72 months). The 103 papers were published in 45 distinct journals. The most commonly published subspecialties were paediatric orthopaedics, hip and knee arthroplasty. A breakdown of the top 10 journals in which these papers were published and their respective impact factors are shown in Table 1. The median (range) impact factor score of the journals, which published the podium research, was 1.730 (0.181–19.967). The journal, in which an abstract was published, with the highest impact factor was The British Medical Journal (impact factor 19.967), with the lowest impact factor journal being The European Journal of Orthopaedic Surgery and Traumatology (impact factor 0.181).

Table 1.

Showing the top 10 journals in rank order of papers published.

Characteristics of abstracts published

The majority of published abstracts were submitted from single centre institutions with a median of 1.12 (1–3) with some multicentric publications. Sample sizes ranged from a minimum of two to a maximum of 15 646 with a median of 43. The largest number of studies submitted to SICOT was from India with 12.2%, while Thailand was the most published country with 16.0% of all completed publications (Tables 2 and 3, respectively). Moreover, the United States had the highest ratio of submitted to publish with a ratio of 63.6% (11 submitted to SICOT and seven publications achieved). Interestingly, both sample size and subspecialty of the abstracts submitted were not significant when tested using chi-squared goodness of fit (p > 0.05) and therefore likely played no role in successful publication of the submitted abstract.

Table 2.

Showing the top five countries with presented oral abstracts. Total N = 329.

Table 3.

Showing the top five countries with abstracts proceeding to full publication. Total N = 103.

Study methodology

We found that the dominant research design was observational comprising 81.8% of all submitted papers. Moreover, of the abstracts that were eventually published, Retrospective observational studies were also the primary research design accounting for 77.7% of all published papers and these were all level III evidence. Randomised control trials (level I evidence) consisted of only 1.5% and 1.1% of submitted and published articles, respectively. Of the presented abstracts, 78.4% mentioned clear statistical methods, while 88.3% of the abstracts that were published contained clear statistical methods signifying a potential benefactor in abstracts that proceed to full publication. The median (range) level of evidence of all studies was 3 (1–5) which corresponds to most studies being a mix of prospective and retrospective.

Consistency of abstracts with final publication

We identified seven major categories to compare the consistency between the submitted abstract and the eventual completed full publication. These categories include precision variances, change in hypothesis/study objective focus, differing study design, variation in sample size, changes in statistical analysis, and differing primary and secondary outcomes. The most consistent categories were the hypothesis and study design categories with mean consistency of 90.4% (CI 82.6%, 95.5%) and 88.3% (CI 80.0%, 94.0%), respectively. The least consistent category was sample size with a mean of 69.2% (CI 58.8%, 78.3%). Precision studies and statistical analysis remained fairly consistent with means of 81.9% (CI 72.6%, 89.1%) and 77.7% (CI 67.9%, 85.6%), respectively. Lastly primary and secondary outcomes also remained consistent with corresponding means of 87.2% (CI 78.8%, 93.2%) and 85.1% (CI 76.3%, 91.6%).

Discussion

The International Society of Orthopaedics and Traumatology (SICOT) is a highly regarded conference that is dedicated to the discussion and dissemination of all orthopaedic topics. Since its initiation in 1929 it has had 37 annual international conferences. Podium and paper presentations undergo a strict peer-review process and are considered high quality, however it is important to analyse this objectively through publication rates as this represents a permanent, accessible and validated representation of the scientific abstract.

Rates of full publication of presented abstracts range widely in the literature from 11% to 78% [1, 2, 8, 9]. Our study is the first to analyse the SICOT Annual International Conference. We have demonstrated a five-year publication rate of 31.3%, which is consistent with other high profile conferences such as AAOS (34%). Furthermore of those published, we reported mean time to publication of 23.4 months, which falls in line with other notable conferences such as AAOS and EFORT.

It is reassuring to note such a high consistency between abstract and full publication with regard to study hypothesis and study designs which is a direct reflection of aims and methods. With regard to the least consistent category of study sample size, this is often owed to the fact that studies are amended following feedback from audience members’ postpresentation [10]. Observational studies remain the most popular category of study, however, there remains a clear lack of level 1 randomised control trials presented at this meeting. Future work should include re-analysing more recent conferences to monitor any trends.

There are a number of reasons why presented abstracts may not achieve full publication. This includes failure to submit as a full article, pessimism about chances of an article being accepted and rejected by journals through the peer-review process. It is also likely that we have underestimated the publication rate of SICOT as we conducted a PubMed and Medline search and therefore publications not indexed in these databases would be missed. However, these remain highly popular databases for research articles and therefore are the databases most likely to reach the majority of our audience. One further possible limitation is that we have limited our search to a five-year period after presentation in 2009. While the authors recognise some studies could be published after five years, there is evidence to suggest that the majority of papers are published within four years of presentation [57].

To conclude, this is the first study to analyse this world renowned international conference and has demonstrated results comparable with those of other notable conferences of a similar standard. Since 2009, annual submissions to SICOT have steadily increased and we would expect podium presentations to be of an even higher scientific value and subsequently an even higher publication rate in recent years. The authors propose another analysis to confirm this hypothesis and we propose a new objective conference grading system based on the publication rate.

Compliance with ethical standards

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Funding

There is no funding source.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Not applicable.

Acknowledgments

The authors would like to thank Kamal Shihadeh for his statistical advice.

References

  1. Bhandari M, Devereaux PJ, Guyatt GH, Cook DJ, Swiontkowski MF, Sprague S, Schemitsch EH (2002) An observational study of orthopaedic abstracts and subsequent full-text publications. J Bone Joint Surg Am 84(4), 615–621. [CrossRef] [Google Scholar]
  2. Kwong Y, Kwong FN, Patel J (2007) Publication rate of trauma abstracts presented at an international orthopaedic conference. Injury 38(7), 745–749. [CrossRef] [PubMed] [Google Scholar]
  3. Donegan DJ, Kim TW, Lee GC (2010) Publication rates of presentations at an annual meeting of the American Academy of Orthopaedic Surgeons. Clin Orthop 468(5), 1428–1435. [CrossRef] [Google Scholar]
  4. Amirhamzeh D, Moor MA, Baldwin K, Hosalkar HS (2012) Publication rates of abstracts presented at pediatric orthopaedic society of North America meetings between 2002 and 2006. J Pediatr Orthop 32(2), e6–e10. [CrossRef] [Google Scholar]
  5. Riordan FA (2000) Do presenters to paediatric meetings get their work published? Arch Dis Child 83(6), 524–546. [CrossRef] [PubMed] [Google Scholar]
  6. Roy D, Sankar V, Hughes JP, Jones A, Fenton JE (2001) Publication rates of scientific papers presented at the Otorhinolarygological Research Society meetings. Clin Otolaryngol 26(3), 253–256. [CrossRef] [Google Scholar]
  7. Scherer RW, Dickersin K, Langenberg P (1994) Full publication of results initially presented in abstracts: a meta-analysis. JAMA 272(2), 158–162. [CrossRef] [PubMed] [Google Scholar]
  8. Okafor L, Frost C, Mesfin A (2015) Publication rate of podium presentations from CSRS Annual Meeting. Spine 40, 699–702. [CrossRef] [PubMed] [Google Scholar]
  9. Frost C, Rubery P, Mesfin A (2015) Publication rate of podium presentation at two international spine meetings: SRS and IMAST. Spine Def 3, 528–532. [CrossRef] [Google Scholar]
  10. Bakkum BW, Chapman C, Johnson C (2014) Publication rates of abstracts presented at the Association of Chiropractic Colleges Educational Conference/Research Agenda Conference from 2002 to 2008. J Chiropr Edu 28, 32–40. [CrossRef] [Google Scholar]

Cite this article as: Al-Hourani K, Al-Aref R, Ley-Greaves R, Ballout F & Mesfin A (2017) Five-year publication rate of podium presentations at SICOT Annual Conference: an observational study and new objective proposal of conference power. SICOT J, 3, 36

All Tables

Table 1.

Showing the top 10 journals in rank order of papers published.

Table 2.

Showing the top five countries with presented oral abstracts. Total N = 329.

Table 3.

Showing the top five countries with abstracts proceeding to full publication. Total N = 103.

Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.

Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.

Initial download of the metrics may take a while.