Revista PODIUM, 2020, January-April 2020; 15(1): 142-152

 

Translated from the original in spanish

 

Evidence-based sports sciences

 

Ciencias del deporte, basadas en la evidencia

 

Ciências do esporte baseadas em evidência

 

Hakan Yaman* https://orcid.org/0000-0001-6077-9851

1Former Faculty of Akdeniz University, Faculty of Medicine, Department of Family Medicine, Antalya, Turkey.

 

*Correspondence author: hakanyam@yahoo.com

 

Received: October 31th, 2019.
Approved: January 21th, 2020.

ABSTRACT

Evidence-based practice is a useful and necessary tool for sports science. The experience of evidence-based medical science could be useful for the implementation of this new approach. Evidence-based practices in sport sciences could also affect, in addition to the fields of training practice, such as management, legislation and education. This will allow decision making to be based on the best available scientific evidence. This review summarises approaches and models of how to apply evidence-based practice in sport science. It also shares a five-step approach with the aim of providing a practical tool. Certain conditions require starting with the practice of evidence. First, the scientific literature must answer the question. Access to scientific databases, knowing how to walk through the jungle of knowledge, mastery of English (since most papers are published in English) is mandatory. A commitment by sport science stakeholders to evidence-based practice and the removal of barriers to implementation would be a promising approach to evidence-based practice in sport science in a short period of time.

Keywords: evidence-based practice; sports; athletes.

RESUMEN

Las prácticas basadas en la evidencia son una herramienta útil y necesaria para las ciencias del deporte. La experiencia de las ciencias médicas, basadas en la evidencia podría ser útil para la implementación de este nuevo enfoque. Las prácticas basadas en datos empíricos, en las ciencias del deporte, también podrían afectar además de los campos de la práctica de la formación, como la gestión, la legislación y la educación. Esto permitirá que la toma de decisiones se base en las mejores pruebas científicas disponibles. En esta revisión, se resumen los enfoques y modelos de cómo aplicar la práctica basada en la evidencia, en las ciencias del deporte. También se comparte un enfoque de cinco pasos, con el objetivo de proporcionar una herramienta práctica. Ciertas condiciones requieren comenzar con la práctica de la evidencia. En primer lugar, la literatura científica debe responder a la pregunta. Acceso a bases de datos científicos, saber caminar por la selva del conocimiento, dominio del inglés (ya que la mayoría de los trabajos se publican en inglés) es obligatorio. Un compromiso de las partes interesadas en las ciencias del deporte, con la práctica basada en la evidencia y la eliminación de los obstáculos a la aplicación, sería un enfoque prometedor para las prácticas basadas en la evidencia, en las ciencias del deporte, en un corto período de tiempo.

Palabras clave: práctica basada en la evidencia; deportes; atletas.

RESUMO

A prática baseada em evidência é uma ferramenta útil e necessária para a ciência do esporte. A experiência da ciência médica baseada em evidência poderia ser útil para a implementação desta nova abordagem. As práticas baseadas em evidência em ciências do esporte também poderiam afetar, além das áreas de prática de treinamento, como gestão, legislação e educação. Isto permitiria que a tomada de decisões fosse baseada nas melhores evidências científicas disponíveis. Esta revisão resume abordagens e modelos de como aplicar a prática baseada em evidência na ciência do esporte. Também compartilha uma abordagem em cinco etapas com o objetivo de fornecer uma ferramenta prática. Certas condições exigem que se comece com a prática da prova. Em primeiro lugar, a literatura científica deve responder à pergunta. O acesso às bases de dados científicos, saber caminhar pela selva do conhecimento, o domínio do inglês (já que a maioria dos artigos são publicados em inglês) é obrigatório. Um compromisso das partes interessadas na ciência do esporte com a prática baseada em evidências e a remoção de barreiras à implementação seria uma abordagem promissora para a prática baseada em evidência na ciência do esporte em um curto período de tempo.

Palavras-chave: prática baseada em evidências; esportes; atletas.

INTRODUCTION

Sport sciences have been historically analyzed from various dimensions and edges. Noguera, M. A. D., & Medina, J. (1997), made a significant contribution to the studies of this science from the research. Within his training, this researcher illustrated the contributions of scientific research to the development of this science, his viewpoint was based on the components of this area of knowledge but he did not emphasize the historical and social elements of the time. Devís-Devís, J., et al., (2010) is an example of the studies of Sports Sciences that have been carried out based on the analysis of various themes and disciplines. This research showed the gradual evolution of sports from the integrality of the areas with which they have been related. That is the strength of this study, however it left gaps, in which the specialties that enriched this science were not deepened. These authors started their studies not only from internal factors within the sport, but from environmental factors that affected it, as it is already exposed in their work Casterad, J. Z., et al., (2006), who related the patterns of physical activity of the adolescents with a certain environmental influence of the environment where they were. Other methods have been incorporated into the study of this science, including mixed methods, as is the case with Anguera, M. T., & Hernández-Mendo, A. (2016) who prepared a notebook that gathered together fundamental aspects of this science from a psychological perspective.

Among other aspects, some contributions have been included in this evolution of sports sciences, such as Campos-Izquierdo, A., & Martín-Acero, R. (2016) who approach it from the perspective of professional competitions. On the other hand, Gavira, J. F., & Fernández, J. G. (2017) reiterate the use of research for the study of sport science, but not from the structural analysis of Noguera, M. A. D., & Medina, J. (1997), but this time from the research as a catalyst or facilitator of the development of these sciences.

With the advance of psychology, sociology and health sciences in the field of sport, work has been undertaken such as that of Cepeda Quintanar, S. et al., (2019). This author included in the study of Sports Science factors such as body image, lifestyle and other aspects related to routines, eating habits and physical exercise. All these studies have enriched the theory of sports science, however, none of them have approached this issue from the actual facts, that is, from the evidence. These are the premises with which the author of this research starts to propose as a central objective of study: to base the development of sport sciences based on evidence.

 

DEVELOPMENT

The exponential increase in scientific knowledge and the increase in the number of information providers pose a challenge to professionals in the field to access up-to-date, reliable and accurate information. It is recommended to develop information literacy in order to find the way out of the information jungle. To master information, it is essential to prioritize evidence-based practices. This approach started in medicine and then divided into other fields of science as a motivation to access the best available information and the increase of qualified research has evolved. It is possible to see the same trend in sports science. In sport science, this approach is called evidence-based practice (EBP), in sport science.

EBP deals with the use of the most qualified scientific findings as well as the management of large amounts of information. Discussions within sport science continue and cover issues such as that sport science is separate from the field and practice or that practitioners, not academics on site, are not eligible to practice what sport science recommends and provides as new scientific knowledge Holt, Pankow, Camiré, Côté, Fraser-Thomas, MacDonald, Strachan & Tamminen KA, (2018).

Within this framework, there is a possibility that the PEB (Evidence-Based Practice) will bring these discussions to an end.

The origins of the EBP concept date back to the 1980s. In 1992, Guyatt et al., (1992) and then Sackett proposed a three-phase model in evidence-based medicine (Sackett, Rosenberg, Gray, Haynes & Richardson, (1996); Haynes et al., 2015). It is claimed that it is essential to have three components in the medical decision process. In addition to evidence-based knowledge, the patient's preferences and the physician's clinical experience in managing the patient's health problem are critical. If sport science is adopted, the model appears in Figure 1 Haynes, et al., 2015; Geisler, McKeon & McKeon, (2017).

The evidence-based medicine model is based primarily on the arguments of the Cochrane attempt to use the best clinical research findings in clinical decision-making in the 1970s. Haynes, et al., 2015; Hjorland, (2011). Although some aspects of this model have been criticized, it is now being widely used in the clinical field. In recent years, it has gained importance due to the increase in the number of qualified investigations, including meta-analyses or systematic reviews in sports science.

The initial criticism was that complaint based medicine was highly elitist and did not take into account clinical experience and patient expectations. However, the experts who responded to these criticisms claimed that this was not true and that the evidence presented in the literature could only exist with the opinion and experience of a clinician. In other words, they have argued that a balanced, organic and contextual integration of the elements of the diagram shown in Figure 1 may be possible with a healthy evidence-based approach. Holt et al., (2018); Sackett et al., (1996); Geisleret al., (2017); Faulkner, Taylor, Ferrence, Munro and Selby, (2006); Wilkerson and Denegar, (2018); Rosner, (2012); Dizon, (2011); Jutte and Walker, (2010); Raab and Craig, 2019; Slade, Philip and Morris, (2018) (Figure 1).

Fig. 1. - Sackett's Three-Circle Diagrams (Adapted from Sports Science)

Sports-related professions, such as athletic training, began to be included in PBS in the early 2000s. In their studies, they took the principles of the Institute of Medicine. The qualifications to be acquired in this framework are expressed as patient-centred care, working as interprofessional teams, requesting evidence-based medicine, focusing on quality improvement and the use of information technology. Jutte & Walker, (2010); Welch, Hankemeier, Wyant, Hays, Pitney & Van Lunen, (2014).

If this is adapted to sports science, we can talk about the following capabilities:

However, it has been reported that there were problems in the field of athletic training during the implementation phase. Although this paradigm shift is significant, problems with time, source, lack of knowledge and training have delayed the implementation of the PEB, Jutte & Walker, (2010); Welch, Hankemeier, Wyant, Hays, Pitney & Van Lunen, (2014).

Although the above scenario, which is likely to be experienced, is unlikely to occur for every athlete; the coach must predict this situation and take proactive measures. This approach is an athlete-centered approach. In addition, as a coach who has adopted PBS, you need to have access to the scientific literature. As a result of the literature search, a large number of publications need to be removed in order to identify the articles that best fit the coach's needs. Raab& Craig, (2019). In addition to the appropriateness of the title of the article and the issue of research, the quality of the article should also be considered. The quality of evidence could be recovered from meta-analysis and systemic review primarily, and if studies with lower levels of evidence are not available (e.g., cohort study, case-control study, narrative review, expert opinion reports, case series, case reports, animal experiments, etc.), they should be considered. Randomised controlled trials will not always be of the same quality and power. When examining randomized controlled studies, some points should be considered (Raab & Craig, 2019):

The two points above are also expected to include the coach's experience. It is possible to examine the literature based on the experiences. Given these three points, the requirements of the PBS will be met. However, it should never be forgotten that PBS is athlete-centered and that the athlete-centered approach is the most crucial (Raab & Craig, 2019).

It is not always possible to transfer the scientific results and the results obtained after the experiments to sports. Controlled conditions will not tell athletes which training conditions, which conditions, which training phase or training period should be used during a training process, while test results reveal the criteria for success. Research results generally report the outcome rather than the result or the effectiveness rather than the outcome. In other words, efficiency refers to what works in the experimental setting and what is useful in the application setting Faulkner, Taylor, Ferrence, Munro & Selby, (2006); Wilkerson & Denegar, (2018); Rosner, (2012); Roth & Parry, (1997); Rothwell, (2005).

Undoubtedly, the very well designed works are beautiful. However, it would not be possible for elite athletes to use the results of the experiments that elite athletes undergo as a scientific study in training programmes. Here the skill of the trainer will come into play. Sometimes the coach's experience is as valuable as the scientific evidence. The ideal combination of PBS with application-proof testing (UBK) in elite sport will provide a very satisfactory result. However, achieving this ideal relationship and ensuring harmony between sport science and sport practice is not always easy in elite sports Dizon, (2011); Ross, Gupta & Sanders, (2018).

A five-step approach has been proposed for the implementation of the above-mentioned three-phase PBS model. This approach includes the identification of current findings in the literature, critical synthesis and monitoring of the criteria on athletes. Raab, et al., (2019; English, Amonette, Graham & Spiering, (2012):

Step 1: before reviewing the literature, a question should be identified that best describes your problem. The question should not be too broad or too narrow. In the case of evidence-based medicine, the PICO format is recommended. The search question considers information about the patients, the population or the problem, the intervention (intervention, prognostic factor or exposure), the comparison and the outcome criteria or measurement. In addition, the question (diagnosis, treatment, therapy, etiology/damage, prognosis, prevention issues) and the type of study (design and most appropriate methodology) are also essential (TarihiE. (1), 2019). If we take this approach to sports science, we must first decide which athletes to look for (P). Then we must determine the pattern of work. The intervention (training, dietary supplement, medication, etc.) must be appropriate to the answer we are looking for (I). It must be determined which intervention (different training methods, etc.) should be compared (C) and what type of change is expected to occur with the intervention at the end (strength, speed, maxVO2, etc.) (O). Therefore, what is measured with this change must be known.

Step 2: after creating the PICO question, we try to find the best evidence in the databases. Decide which databases to use before searching the databases. Many of the databases are free to access. When evidence-based medicine is mentioned, the Cochrane Database comes to mind first. In addition, databases such as Pubmed, Clarivate, Scopus, Cinahl, EBSCO are used. The Sportdiscuss database is available for sports.

Step 3: after searching for appropriate titles and abstracts, after scanning the database, the full texts of these studies are available. As mentioned above, all the expected points, which are in the material method part, are taken into consideration when examining the randomised controlled studies.

Step 4: review the bibliographic information obtained at this stage. The trainer compares this information with the available knowledge. He looks for answers to the unknowns. There are solutions that fit the expectations of the athlete.

Step 5: the coach, following the athlete's development in the training and competition environment, makes comparisons and improvements with the available bibliographic evidence; increases the experience. He runs steps 1-4 continuously and continuously monitors the athlete's performance and success criteria.

Evidence-based practices have been mentioned so far to support the training of an experienced coach in the science and practice of sport and to support their decision making during training activities. The evidence-based decision-making process of a coach or sport scientist is fundamental in evidence-based sport science, but other disciplines and areas related to sport science should also be involved. Again, it is possible to take evidence-based sport science (KDSB) in three ways, taking the work done in the field of athletic training Geisler et al., (2017):

PEBs are discussed in detail above. The three essential components of EBP, which derive their origin from evidence-based medicine, are evidence of athletic values, coach experience and evidence-based practice. KDR-M, as in the five-stage approach of the EBP model, involves the process of obtaining information in five stages, comparing and synthesizing the information and monitoring its efficiency and effectiveness in implementation, while seeking answers to the problems that will form the basis of regulation and legislative development. In this case, the source of knowledge is not only scientific research. Expert opinion includes the results of the stakeholder questionnaire: statistics, review of previous legislation and Internet scans Spector, (2010).

Evidence-based education also consists of three elements. The first consists of the concept of educator-student; time, space and complex dimensions of the educational environment are taken into account, as well as the values of the learner and educator. The second has to do with the concept of professional wisdom. This is also the intrinsic evidence that the trainer has contributed to this process. In addition to the expertise and experience of the trainer, it is the consensus and practices that trainers and experts form among themselves Geisler et al., (2017).

Once again, based on the health care model and evidence that Spector has proposed, it is possible to develop an Evidence Based Policy (SPKDP) in Sports Science by interacting with Evidence Based Practice, Legislation (KDR-M) and Evidence Based Education (EBE) in their sport science Spector, (2010).

A group called PRECISION states that four different types of evidence are needed in the decision-making process, in evidence-based training and that this evidence should also guide the business model. Accordingly, the evidence that will affect the decisions of optimal and professional trainers is (TarihiE. (2), 2019).

 

CONCLUSIONS

In conclusion, evidence-based practices are a useful and necessary approach to sport science. The evidence-based approach has been applied in the medical field for over 30 years and its focus is on the area of application and remains widespread in many areas. The fact that evidence-based practice and sport science takes place in the field of management and legislation as well as in education will ensure that existing scientific knowledge is taken as a basis for decision making and that it paves the way for more qualified approaches. Although not discussed because it is not the subject of this review, there is a need for scientists and field practitioners to have access to the scientific resources available to implement evidence-based practices.

Due to the fact that the current literature is published primarily in English, people who will benefit from these studies must have sufficient knowledge of the foreign language to read the literature. Individuals need to be educated in this field in order to evaluate the existing evidence-based literature. For the evidence-based approach to practice to be resolved, all stakeholders in sport science must reach a consensus on this issue and pave the way for evidence-based practice. If the above obstacles are provided and removed, it will be possible to feel the positive effects of evidence-based practice in sport science in a short time.

 

BIBLIOGRAPHICAL REFERENCES

Anguera, M. T., & Hernández-Mendo, A. (2016). Avances en estudios observacionales de Ciencias del Deporte desde los mixed methods. Cuadernos de psicología del deporte16(1), 17-30. Recuperado de: https://revistas.um.es/cpd/article/view/254261

Campos-Izquierdo, A., & Martín-Acero, R. (2016). Percepción de las competencias profesionales de los graduados en Ciencias de la Actividad Física y del Deporte. Revista de psicología del deporte, 25(2), 339-346. Recuperado de: https://dialnet.unirioja.es/servlet/articulo?codigo=5650787

Casterad, J. Z., Puyal, J. R. S., Gurrola, O. C., Lanaspa, E. G., Ostariz, E. S., & Clemente, J. A. J. (2006). Los factores ambientales y su influencia en los patrones de actividad física en adolescentes. RICYDE. Revista Internacional de Ciencias del Deporte, 2(4), 1-14. Recuperado de: https://www.cafyd.com/REVISTA/art1n4a06.pdf

Cepeda Quintanar, S., Sánchez Castillo, S., Díaz Suárez, A., & López Sánchez, G. F. (2019). Imagen corporal y estilo de vida (actividad física, dieta, alcohol y tabaco) de estudiantes de ciencias del deporte. Sportis, 5(3), 484-512. Recuperado de: https://ruc.udc.es/dspace/handle/2183/23919

Devís-Devís, J., Valenciano Valcárcel, J., Villamón, M., & Pérez Samaniego, V. (2010). Disciplinas y temas de estudio en las ciencias de la actividad física y el deporte. Rev.int.med.cienc.act.fís.deporte. 10(37). Recuperado de: http://cdeporte.rediris.es/revista/revista37/artdisciplinas147.htm

Dizon, J. M. (2011). Evidence Based Practice in Sports Medicine and Rehabilitation: Easy, Pragmatic or Nonsense? Journal of Sports Medicine & Doping Studies, 01(01). Recuperado de: https://doi.org/10.4172/2161-0673.1000e101 Recuperado de: https://www.hilarispublisher.com/open-access/evidence-based-practice-in-sports-medicine-and-rehabilitation-easy-pragmatic-or-nonsense-2161-0673.1000e101.pdf

English, K. L., Amonette, W. E., Graham, M., & Spiering, B. A. (2012). What is "Evidence-Based" Strength and Conditioning? Strength and Conditioning Journal, 34(3), 19-24. https://doi.org/10.1519/SSC.0b013e318255053d Recuperado de: https://journals.lww.com/nsca-scj/fulltext/2012/06000/what_is__evidence_based__strength_and.2.aspx

Evidence-Based Medicine Working Group. (1992). Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA, 268(17), 2420-2425. https://doi.org/10.1001/jama.1992.03490170092032 Recuperado de: https://www.ncbi.nlm.nih.gov/pubmed/1404801

Faulkner, G., Taylor, A., Ferrence, R., Munro, S., & Selby, P. (2006). Exercise science and the development of evidence-based practice: A "better practices" framework. European Journal of Sport Science, 6(2), 117-126. https://doi.org/10.1080/17461390500528568 Recuperado de: https://www.tandfonline.com/doi/abs/10.1080/17461390500528568?journalCode=tejs20

Geisler, P. R., McKeon, P. O., & McKeon, J. M. M. (2017). On the Need to Move Toward an Evidence-Based Athletic Training (EBAT). Athletic Training Education Journal, 12(2), 81-94. https://doi.org/10.4085/120281 Recuperado de: https://www.academia.edu/33379319/On_the_Need_to_Move_Toward_an_Evidence-Based_Athletic_Training_EBAT

Haynes, R. Brian., et al. (2015) Clinical Epidemiology How to Do Clinical Practice Research. Wolters Kluwer Health. Recuperado de: https://epdf.pub/clinical-epidemiology-how-to-do-clinical-practice-research.html

Hjørland, B. (2011). Evidence-Based Practice: An Analysis Based on the Philosophy of Science. JASIST, 62, 1301-1310. https://doi.org/10.1002/asi.21523 Recuperado de: https://onlinelibrary.wiley.com/doi/abs/10.1002/asi.21523

Holt, N. L., Pankow, K., Camiré, M., Côté, J., Fraser-Thomas, J., MacDonald, D. J., Tamminen, K. A. (2018). Factors associated with using research evidence in national sport organisations. Journal of Sports Sciences, 36(10), 1111-1117. https://doi.org/10.1080/02640414.2017.1357830 Recuperado de: https://www.tandfonline.com/doi/full/10.1080/02640414.2017.1357830

Jutte, L. S., & Walker, S. E. (2010). Incorporating Foundational Evidence-Based Practice Concepts and Skills Across an Athletic Training Education Program. Athletic Training Education Journal, 5(3), 119-125. https://doi.org/10.4085/1947-380X-5.3.119

Raab, S.,& Craig, D. (2019). What is Evidence-Based Practice Anyway? (This is an excerpt from Evidence-Based Practice in Athletic Training.). Eriþim: https://uk.humankinetics.com/blogs/excerpts/what-is-evidence-based-practice-anyway. Eriþim Tarihi: 02.02.2019. Recuperado de: https://us.humankinetics.com/blogs/excerpt/what-is-evidence-based-practice-anyway

Rosner, A. L. (2012). Evidence-based medicine: revisiting the pyramid of priorities. Journal of Bodywork and Movement Therapies, 16(1), 42-49. https://doi.org/10.1016/j.jbmt.2011.05.003 Recuperado de: https://www.ncbi.nlm.nih.gov/pubmed/22196426

Ross, E., Gupta, L., & Sanders, L. (2018). When research leads to learning, but not action in high performance sport. Progress in Brain Research, 240, 201-217. https://doi.org/10.1016/bs.pbr.2018.08.001 Recuperado de: https://www.ncbi.nlm.nih.gov/pubmed/30390832

Roth, A., & Parry, G. (1997). The implications of psychotherapy research for clinical practice and service development: Lessons and limitations. Journal of Mental Health, 6(4), 367-380. https://doi.org/10.1080/09638239718699 Recuperado de: https://www.researchgate.net/publication/232073074_The_implications_of_psychotherapy_research_for_clinical_practice_and_service_development_Lessons_and_limitations

Rothwell, P. M. (2005). External validity of randomised controlled trials: «to whom do the results of this trial apply?» Lancet (London, England), 365(9453), 82-93. https://doi.org/10.1016/S0140-6736(04)17670-8 Recuperado de: https://www.ncbi.nlm.nih.gov/pubmed/15639683

Sackett, D. L., Rosenberg, W. M., Gray, J. A., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: what it is and what it isn't. BMJ (Clinical Research Ed.), 312(7023), 71-72. https://doi.org/10.1136/bmj.312.7023.71 Recuperado de: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2349778/

Slade, S. C., Philip, K., & Morris, M. E. (2018). Frameworks for embedding a research culture in allied health practice: a rapid review. Health Research Policy and Systems, 16(1), 29. https://doi.org/10.1186/s12961-018-0304-2

Spector, N. (2010). Evidence-Based Nursing Regulation: A Challenge for Regulators. Journal of Nursing Regulation, 1(1), 30-36. https://doi.org/10.1016/S2155-8256(15)30364-1 Recuperado de: https://www.sciencedirect.com/science/article/abs/pii/S2155825615303641

Tarihi Eriþim (1) (2019) The P.I.C.O. Model for Clinical Questions. The University of Illinois Library. Recuperado de: https://researchguides.uic.edu/c.php?g=252338&p=3954402

Tarihi Erisim (2)( 2019) What Is "Evidence-Based Training"? Precision Sport Science. Recuperado de: http://precisionsportscience.com/what-is-evidence-based-training/.

Welch, C. E., Hankemeier, D. A., Wyant, A. L., Hays, D. G., Pitney, W. A., & Van Lunen, B. L. (2014). Future Directions of Evidence-Based Practice in Athletic Training: Perceived Strategies to Enhance the Use of Evidence-Based Practice. Journal of Athletic Training, 49(2), 234-244. https://doi.org/10.4085/1062-6050-49.2.15 Recuperado de: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3975779/

Wilkerson, G., & Denegar, C. (2018). A Growing Consensus for Change in Interpretation of Clinical Research Evidence. Journal of Athletic Training, 53, 320-326. https://doi.org/10.4085/1062-6050-8-17 Recuperado de: https://www.researchgate.net/publication/324267344_A_Growing_Consensus_for_Change_in_Interpretation_of_Clinical_Research_Evidence

 

Conflict of interests:

The authors declare not to have any interest conflicts.

 

Authors' contribution:

The authors have participated in the writing of the work and analysis of the documents.

 


This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International license.
Copyright (c) 2020 Hakan Yaman