Bibliometrics

Bibliometrics – “measuring books” – is about making quantitative studies of, above all, scientific publication. That which can be measured and compared are, alone or in combination, authors, articles, terms, subject, university, country, journal, publisher, and citations.

Modern bibliometrics is based on the basic assumption that the attention research should be reflected in subsequent research citations. Often cited research has thus received a great deal of attention and influenced subsequent research. Such attention is usually a sign of quality. Bibliometric studies of citations are therefore often indicative of the distribution of funds for research.

Bibliometric concepts and definitions

When a researcher in his publication refers to another publication, this is called a citation. In addition to bibliographic information about scientific articles, the Scopus and Web of Science databases also contain the articles’ reference lists, making it possible to calculate how many citations a given article has received from other articles in the database.

A research article often has more than one author, and the authors frequently come from more than one university. After summing up the number of articles from a university, the question remains how co-authored articles should be handled? It is customary for the various higher education institutions to share the article. This is called fractionalised counting.

Articles are usually similarly fractionalised for individual authors. Each of the five authors of an article is credited with one-fifth of the article.

Co-citation means that two specific publications or authors are quoted in subsequent publications. Co-citation analysis involves the examination of the interrelationships of publications. The connection means that the cited publications in the respective subject area are likely similar. The more shared citations publications receive, the greater the similarity.

Scientific publications have a mutual relationship or connection, if they have one or more common sources – they cite the same work. More common sources in a publication indicate a stronger connection. With bibliographic links, one can group scientific publications thematically and reflect the networks that describe the publications' mutual relationship.

The Top 5% indicator signifies the proportion of the analysed unit’s publications belonging to the 5% most cited and related to the world average in the same subject area and with the same document type and age. The top 5% indicator is written as a decimal number related to the world average, 1. A value higher than 1 indicates that the unit has more publications than the average among the 5% highest cited and a value less than 1 indicates that it has fewer.

The H-index, or Hirsch index, is a metric showing how many publications a researcher has produced over time and the impact of the publications expressed in the number of citations.

The H-index can also be calculated for a journal or an institution. The H-index increases with time. It also depends on the subject area. H-index is defined as the number of articles X that have been cited more than X times.

How to calculate an h-index

When generating an h-index, it is essential to know how the database is calculated and the calculation period. The following three databases contain different journals and index them differently and therefore, do not yield the same h-index.

  • Web of Science – search for author, ORCID or ResearcherID. Make sure all WoS publications are included. Click “Create Citation Report” in the upper right corner of the list. The results page contains citation statistics for all the included publications as well as the h-index.  
  • Scopus – search for all articles per author or ORCID. Make sure all Scopus publications are included. Click “View Citation Overview” in the upper left corner of the publication list.
  • Google Scholar – create a Google Scholar Citations Account. Click on “My profile” to view the Google Scholar h-index. The quality of inbound publications can vary. This also includes citations in “predatory journals”. 

There are also more databases – often subject-dependent – for calculating an h-index.

 

Criticism and problems

The simplicity and impact of quantitative analyses of research have in recent years increased interest in bibliometrics. From being only a neutral analytical tool, the use of bibliometrics for research evaluation and distribution of funds has had some unplanned consequences:

  • An incipient adaptation towards increased formalisation of scientific publishing in the humanities is a consequence of the difficulties of measuring publication.
  • Problems with manipulative adaptation to bibliometric requirements and conditions.
  • Concerns about a shift from research quality to citations.
  • Concerns about “salami slicing” – that research results are published in small pieces to collect more citations.

Other criticisms:

  • Subject coverage in citation databases – social sciences and humanities are inadequately covered.
  • The citation and quality relationship – citations for reasons other than demonstrating previous knowledge, such as criticism.
  • Reliability of bibliometric comparisons – especially at the personal level.

Benefits of Bibliometrics

  • An inexpensive way to assess large amounts of research.
  • A method to detect developments, patterns, and deviations over time.
  • Comparatively free from problems related to inequality, nepotism, prejudice, gender issues and competition.

What to consider when interpreting and performing analyses

Standardisation: Compare equals. Units with different conditions do not make for a fair analytical comparison. Always compare within the same subject, the same annual interval, and the same publication type.

Bibliometrics are best suited for large units, such as journals, universities, or countries. Analyses of smaller units, such as articles, researchers, or research groups, require additional knowledge.

Alternatives to bibliometrics

Panel assessment

Peer review has an older history than bibliometrics and is generally considered a more appropriate assessment of disciplines poorly represented in citation databases. The assessment model for allocating resources that the Swedish Research Council has is primarily regarded as assessment with expert panels, even if bibliometric indicators are included.

Norwegian model

A hybrid model and a mixture of panel assessment and bibliometrics. The model is based on an assessment of publishing channels: journals, series, publishers, and websites. Publications such as articles, books or book chapters are then assigned a value based on the assessment of the journal or publisher. This value is then used to analyse larger units. As the name suggests, the model is used in Norway to evaluate research and the distribution of funding. The Norwegian expert’s assessment of publishing channels is freely available.