Illinois Data Bank

Expert assessment of RobotReviewer data extraction performance on 10 articles

The PDF describes the process and data used for the heuristic user evaluation described in the related article “Evaluating an automatic data extraction tool based on the theory of diffusion of innovation” by Linh Hoang, Frank Scannapieco, Linh Cao, Yingjun Guan, Yi-Yun Cheng, and Jodi Schneider (under submission).

Frank Scannapieco assessed RobotReviewer data extraction performance on ten articles in 2018-02. Articles are included papers from an update review: Sabharwal A., G.-F.I., Stellrecht E., Scannapeico F.A. Periodontal therapy to prevent the initiation and/or progression of common complex systemic diseases and conditions. An update. Periodontol 2000. In Press.

The form was created in consultation with Linh Hoang and Jodi Schneider. To do the assessment, Frank Scannapieco entered PDFs for these ten articles into RobotReviewer and then filled in ten evaluation forms, based on the ten Robot Reviewer automatic data extraction reports. Linh Hoang analyzed these ten evaluation forms and synthesized Frank Scannapieco’s comments to arrive at the evaluation results for the heuristic user evaluation.

Social Sciences
RobotReviewer; systematic review automation; data extraction
CC BY
U.S. National Institutes of Health (NIH)-Grant:R01LM010817
Jodi Schneider
631 times
Version DOI Comment Publication Date
1 10.13012/B2IDB-8274875_V1 2018-07-25

1.11 MB View File

Contact the Research Data Service for help interpreting this log.

RelatedMaterial destroy: {"material_type"=>"Code", "availability"=>nil, "link"=>"https://github.com/ijmarshall/robotreviewer3", "uri"=>"", "uri_type"=>"URL", "citation"=>"Byron Wallace, Iain Marshall, Joël Kuiper and Frank Soboczenski. RobotReviewer [github code].", "dataset_id"=>616, "selected_type"=>"Code", "datacite_list"=>"IsSupplementedBy", "note"=>"", "feature"=>false} 2025-01-08T23:48:50Z
RelatedMaterial destroy: {"material_type"=>"Article", "availability"=>nil, "link"=>"", "uri"=>"", "uri_type"=>"", "citation"=>"Linh Hoang, Frank Scannapieco, Linh Cao, Yingjun Guan, Yi-Yun Cheng, and Jodi Schneider. Evaluating an automatic data extraction tool based on the theory of diffusion of innovation. Under submission.\r\n", "dataset_id"=>616, "selected_type"=>"Article", "datacite_list"=>"IsSupplementTo", "note"=>"", "feature"=>false} 2025-01-08T23:48:39Z
RelatedMaterial destroy: {"material_type"=>"Article", "availability"=>nil, "link"=>"", "uri"=>"", "uri_type"=>"", "citation"=>"Sabharwal A., G.-F.I., Stellrecht E., Scannapeico F.A. Periodontal therapy to prevent the initiation and/or progression of common complex systemic diseases and conditions. An update. Periodontol 2000. In Press.", "dataset_id"=>616, "selected_type"=>"Article", "datacite_list"=>"", "note"=>"", "feature"=>false} 2025-01-08T23:48:39Z
RelatedMaterial destroy: {"material_type"=>"Code", "availability"=>nil, "link"=>"https://robot-reviewer.vortext.systems", "uri"=>"", "uri_type"=>"", "citation"=>"Byron Wallace, Iain Marshall, Joël Kuiper and Frank Soboczenski. RobotReviewer [demo site]. ", "dataset_id"=>616, "selected_type"=>"Code", "datacite_list"=>"", "note"=>"", "feature"=>false} 2025-01-08T23:48:38Z
RelatedMaterial update: {"note"=>[nil, ""], "feature"=>[nil, false]} 2024-02-01T17:42:30Z
RelatedMaterial update: {"note"=>[nil, ""], "feature"=>[nil, false]} 2024-02-01T17:42:30Z
RelatedMaterial update: {"note"=>[nil, ""], "feature"=>[nil, false]} 2024-02-01T17:42:30Z
RelatedMaterial update: {"uri_type"=>["", "URL"], "datacite_list"=>["", "IsSupplementedBy"], "note"=>[nil, ""], "feature"=>[nil, false]} 2024-02-01T17:42:30Z
Research Data Service Illinois Data Bank
Access and Use Policies Web Privacy Notice Contact Us