Child pages
  • 4.2 Data Acquisition and Preparation

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.


ISI Web of Science (WoS) is a leading citation database cataloging over 10,000 journals and over 120,000 conferences. Access it via the "Web of Science" tab at (note: access to this database requires a paid subscription). Along with Scopus, ISI WoS provides some of the most useful datasets data sets for scientometric analysis.
To find all publications by an author, search for the Publications may be obtained by querying the Web of Science database by a variety of fields within the database, such as topic, author and journal names, date ranges, etc. There are two methods for exporting data from Web of Science: direct data exporting from the search interface, and through adding records to a marked list. Direct data exporting from searches provides the fastest method to gather the most used data fields in a search, and provides less room for user selection bias of a group of records. Web of Science Marked Lists allow users to select only the records that interest them and select specific data to be exported from the various databases that a user's institution subscribes to. ISI Web of Science - Direct Exporting

In this tutorial, we will download publication data using a query with an author name. Author names should be searched using the last name and the first initial followed by an asterisk wildcard in the author field. To find papers by Eugene Garfield, enter Garfield E* in the author field. The search yielded 1,551 results on January yielded 725 results on April 23, 20142016, 500 of which can be downloaded at a time, see Figure 4.22a.

Image Modified

Figure 4.22a: ISI Web of Science search interface and ISI Web of Science search results

To download the first 500 article records with citations, go to the bottom of the page, and from the drop down menu next to the printing and email icons, select "Save to Other File Formats". A box will appear on the screen where you can manually select records '1' to '500', select 'Full Record and Cited References', select 'Save to Plain Text' in the drop down menu, and then click send, see figure 4.3a2b. This process can be repeated to obtain records and citations by adjusting the manually selected records by units of 500, e.g. select records "501" to "1000725" for the Garfield example.

Image Modified
Figure 4.3a2b: Saving records from Web of Science

Wait for the processing to complete, and then save the file as GarfieldE.isi. The resulting file can be seen in Figure 4.3b2c.

Figure 4.3b2c: Saving Records from Web of Science

Figure 4.3a2d: View of saved GarfieldE.isi ISI Web of Science - Marked List Data Exporting

Downloading WoS data from a marked list is very similar to the process of downloading data directly from the search page. With this tutorial, we will skip the instructions on how to run a basic WoS search, which is described in the section above.

After you have refined your search query to the WoS database, you will want to add records to your Marked List. To do this, you may select individual articles using the check boxes next to a publication record and then selecting the button, "Add to Marked List", see figure 4.2e.


Image Added

Figure 4.2e: Locating the Marked List button on Web of Science

You may also add records to the Marked List using a range of publication records in a search result. To do this, select the "Add to Marked List" button in the search result page without adding marks to individual publication records. Next, a box will pop up in the screen that allows you to enter a range for the publication records that you are interested and then select the "Add" button, see figure 4.2f for a picture of this screen.

Image Added

Figure 4.2f: Specifying a the range of records in a Marked List on Web of Science

You will notice that the publications you have selected in the search page will now have orange checks next to them, and the Marked List tab on the Navigation bar for the site will be updated to include the total number of records in your list. After you've completed adding publication records to your marked list you will want to visit the Marked List page. Click on the link in the navigation bar named "Marked List" that has an orange count box next to it, see figure 4.2g.

Image Added

Figure 4.2g: Click the Marked List link in the navigation bar to access the Marked List page on Web of Science

The Marked List page provides you with a list of publications selected and various means of exporting publication data, including limiting exports to specific databases that are part of the WoS database. To export the most amount of data from WoS and to ensure the most replicable results, it is best to choose the Web of Science Core Collection export tool. The export tool outlines the various steps you will need to take: from selecting the exported records, selecting the content for exporting, and the format that the data will come in, see figure 4.2h for a view of this export tool.

The Web of Science Core Collection export tool allows you to select a variety of fields, including correspondence addresses of researchers, funding information, cited reference lists, keywords and WoS categories and research areas, etc.

Like direct data exporting, make sure to select the format option "Save to Other File Formats" and Plain Text.  You will also be limited to exporting 500 records at a time, which means that you will have to combine data sets manually after downloading all publications (see for instructions here).

Image Added

Figure 4.2h: Selecting data fields and exporting data from a Marked List

ISI files are loosely based on the RIS file format, and data in this format can be used for the following types of analyses:


Elsevier's Scopus, like ISI Web of Science, has an extensive catalog of citations and abstracts from journals and conferences. Subscribers to Scopus can access the service via Scopus provides a multiple methods to search and analyze citations and abstracts: by document, authors, institutional affiliations, and advanced Boolean search.

To find all articles whose abstract, title, or keywords include the terms 'Watts Strogatz Clustering Coefficient', simply enter those terms in the Article/Abstract/Keywords field. ThirtyForty-nine results were found as of February 3April 29, 20142016. You can download the references (up to 2,000 full records, and 20,000 "Citations Only" in a CSV file) by checking the 'Select All' box and clicking on the down arrow next to 'RIS Export' and choosing the file type you wish to export the references as. Image RemovedYou can export in RIS Format (this is for EndNote, or other Reference Managers), CSV as mentioned earlier, BibTeX, or as a .txt file. Once you have chosen the desired format, click on the 'Export' button on the bottom right of the drop down menu.

Image Added
Figure 4.42i: Scopus search interface and Scopus search results

At the export screen, select 'Comma separated file, .csv' (e.g. Excel) and select the types of information that you will need. For our purposes, select 'All available information' from the drop-down menus and choose 'Export'.
Save the file as WattsStrogatz.scopus. The resulting file can be seen in Figure 4.52j.

Figure 4.52j: Saving records in Scopus and viewing WattsStrogatz.scopus


Google Scholar data can be acquired using Publish or Perish (Harzing, 2008) that can be freely downloaded from A query for papers by Albert-László Barabási run on Sept. 21, 2008 April 29, 2016 results in 111 280 papers that have been cited 1492,343 213 times, see Figure 4.62k.

Image RemovedImage Added
Figure 4.62k: Publish or Perish search results for Albert-László Barabási and viewing barabasiPoP.csv

To save records, select 'File > Save' from menu and then choose the appropriate file format (.csv, *.enl, or *.bib) in the 'Choose File' pop-up window. All three file formats can be read by the Sci2 Tool. The result in all three formats named 'barabasi.' is also available in the respective subdirectories in 'yoursci2directory /sampledata/scientometrics/' and will be used later in this tutorial.
Data from Google Scholar can be used for the following types of analyses:


Please note that before any of the following algorithms can be run, the Google Citation User ID Search algorithm must be run to retrieve the Google Citation User IDs.

Attach citation Citation Table from Google Scholar


To download your Facebook data, open the Sci2 Tool and select the "Facebook" option in the file menu. In the next submenu, select the "Access Token" option. You may be asked to log into your account; afterwards, you will be redirected to a webpage that provides you with an access token. In your browser, right click on the grey text, choose "Select All", and then copy this text to your clipboard. Figure 4.7 2l demonstrates this process.

Fig. 4.72l: Use Sci2 to gain access and retrieval of Facebook Access Token

After you have copied your access token, return to the Sci2 Tool and the file menu. In the "Facebook" sub-menu, select either "Facebook Friends Data" or "Mutual Friends". A window will pop up, like in figure 4.82m. Paste your access token into the text field and hit the "OK" button.

Fig 4.82m: Sci2 Tool's Facebook Friends Data and Mutual Friends data load windows with access tokens


Funding data provided by the National Science Foundation (NSF) can be retrieved via the Award Search site ( Search by PI name, institution, and many other fields, see Figure 4.9.

Figure 4.92n: NSF 'Award Search' interface and search results page

To retrieve all projects funded under the Science of Science and Innovation Policy (SciSIP) program, simply select the 'Program InformationAdvanced Searcg' tab, do an 'Element Code Lookup', enter '7626' into the 'Element Code' field which is under the 'Program Information' section, and click the 'Search' button. On Sept 21stApril 23, 20082016, exactly 50 124 awards were found. Award records can be downloaded in csv, Excel, XML, or XML .txt format. Save file in csv format, and change the file extension from .csv to .nsf. A sample .nsf file is available in 'yoursci2directory /sampledata/scientometrics/nsf/BethPlale.nsf'. In the Sci2 Tool, load the file using 'File > Load File'. Select "NSF csv format" in the "Load" pop-up window. A table with all records will appear in the Data Manager. View the file in Excel.
Data in NSF files can be used for the following types of analyses:


Funding data provided by the National Institutes of Health (NIH), and associated publications and patents, can be retrieved via the NIH RePORTER site ( The database draws from eRA, Medline, PubMed Central, NIH Intramural, and iEdison. Search by location, PI name, category, etc., see Figure 4.102o.

Figure 4.102o: NIH RePORTER search interface and search results page


4.2.4 Datasets: Scholarly Database
4.2.4 Datasets: Scholarly Database
4.2.4 Datasets: Scholarly Database

Figure 4.112p: Graph of the numbers of records published each year by various organizations

Medline, U.S. patent, as well as funding data provided by the National Science Foundation and the National Institutes of Health can be downloaded from the Scholarly Database (SDB) at Indiana University. SDB supports keyword based cross-search of the different data types and data can be downloaded in bulk, see Figures 4.12 2q and 4.13 2r for interface snapshots.

Register to get a free account or use 'Email:' and 'Password: nwb' to try out functionality.
Search the four databases separately or in combination for 'Creators' (authors, inventors, investigators) or terms occurring in 'Title,' 'Abstract,' or 'All Text' for all or specific years. If multiple terms are entered in a field, they are automatically combined using the Boolean operator 'OR.' Entering 'breast cancer' will match any record with 'breast' or 'cancer' in that field. Using the Boolean operator AND (for example, 'breast AND cancer') would only match records that contain both terms. Double quotations can be used to match compound terms, e.g., "breast cancer" retrieves records with the phrase "breast cancer," but not records where 'breast' and 'cancer' are present in isolation. The importance of a particular term in a query can be increased by putting a ^ and a number after the term. For instance, 'breast cancer^10' would increase the importance of matching the term 'cancer' by ten compared to matching the term 'breast.'

Figure 4.122q: Scholarly Database 'Home' page and 'Search' interface

Results are displayed in sets of 20 records, ordered by a Solr internal matching score. The first column represents the record source, the second the creators, third comes the year, then title and finally the matching score. Datasets can be downloaded in different subsets and formats for future analysis.

Figure 4.132r: Scholarly Database search results and download interfaces