The Positive and Negative Influence of Search Results on People’s Decisions about the Efficacy of Medical Treatments Dataset

Amira Ghenai

University of Waterloo

This site provides the dataset including the list of webpages used in the study "The Positive and Negative Influence of Search Results on People’s Decisions about the Efficacy of Medical Treatments".

See also: The full paper is available here.

The user study dataset consists of 158 Web pages collected manually about medical treatment's efficacy. We used Bing, Yahoo, and Google to collect the webpages relevant to determining the efficacy of the ten medical treatments. For each medical treatment, we created pools of 8-10 correct and 8-10 incorrect webpages. A correct webpage (document) contains information about the efficacy of the medical treatment that supports the truth. An incorrect webpage (document) contains information that contradicts the true efficacy of the medical treatment. The set of webpages may be downloaded here:

The user_study_material.zip file will have 11 folders for different 11 medical treatments. Inside every medical treatment folder, there is a folder of correct (containing all the list of correct webpages) and incorrect (containing the list of incorrect pages) folders.
The words-definitions.txt contains the list of words with corresponding definitions as displayed to the participants during the study.
The webpages_snippets.txt lists all the webpages snippets used during the user study.
The Topics_and_URLS.txt lists the topics with corresponding list of URLs.

Note 1: The topic "Do cranberries prevent urinary tract infections" is used only in the tutorial section and not during the actual user study.
Note 2: The user_study_material.zip decompressed file is 145.3MB.
Note 3: The webpages are screenshots of the web page rather than to the actual web page. We did this because we wanted to make sure that the participant was not able to click on any links and view any pages outside the scope of the study. In addition, this allowed us to be certain that each participant was exposed to the same version of the web page, and we did not have to fear the loss of pages during the study. Further, the image names correspond to the URL string.

For more details, please read the paper.

Please cite the original publication when using the dataset:
Pogacar, Frances A., Amira Ghenai, Mark D. Smucker, and Charles LA Clarke. "The Positive and Negative Influence of Search Results on People's Decisions about the Efficacy of Medical Treatments." Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval. ACM, 2017.