Skip to main Content

Crowd-sourced assessment of technical skills (C-SATS): Validation through the basic laparoscopic urologic surgery (BLUS) curriculum - Interview

A properly-sized and qualified crowd can accurately score laparoscopic skill performances on par with faculty experts. Crowd-based ratings may be an efficient method for assessing passing/failing performances, and for measuring change in performance after training.

Lendvay, T., et al.. Crowd-sourced assessment of technical skills (C-SATS): Validation through the basic laparoscopic urologic surgery (BLUS) curriculum - Interview. 32nd World Congress of Endourology & SWL - September 3 - 7, 2014 - Taipei, Taiwan.

World Congress of Endourology Meeting

Lendvay T, Comstock B, Averch T, Box G, Knudsen B, Brand T, Ferrandino M, Kaouk J, Landman J, Lee B, McDougall E, Menhadji A, Schwartz B, Sweet R, Kowalewski T. Crowd-Sourced Assessment of Technical Skills (C-SATS): Validation Through the Basic Laparoscopic Urologic Surgery (BLUS) Curriculum. World Congress of Endourology Meeting, Taipei, Taiwan, September 6, 2014 [Moderated Poster].

Crowd-Sourced Assessment of Technical Skills: a novel method to evaluate surgical performance

Journal of Surgical Research

Validated methods of objective assessments of surgical skills are resource intensive. We sought to test a web-based grading tool using crowdsourcing called Crowd-Sourced Assessment of Technical Skill.

Chen C, White L, Kowalewski T, Aggarwal R, Lintott C, Comstock B, Kuksenok K, Aragon C, Holst D, Lendvay T. Crowd-sourced assessment of technical skills: a novel method to evaluate surgical performance. Journal of Surgical Research. 2014 Mar 31;187(1):65-71.

Surgical Skill and Complication Rates after Bariatric Surgery

The technical skill of practicing bariatric surgeons varied widely, and greater skill was associated with fewer postoperative complications and lower rates of reoperation, readmission, and visits to the emergency department. Although these findings are preliminary, they suggest that peer rating of operative skill may be an effective strategy for assessing a surgeon's proficiency.

Birkmeyer, J.D., et al. for the Michigan Bariatric Surgery Collaborative, Surgical Skill and Complication Rates after Bariatric Surgery The New England Journal of Medicine. Published online October 10, 2013. N Engl J Med 2013; 369:1434-1442

Crowd-Sourcing Assessment of Technical Skills (C-SATS): Faculty Experts vs. Amazon.com Mechanical Turk Project™ vs. Facebook™

We discovered that crowd-sourced surgery-naïve graders provided assessments equivalent to trained expert surgeon evaluators, and at a faster rate.

Chen C., et al.  Crowd-Sourced Assessment of Technical Skills (C-SATS): Faculty Experts vs. Amazon.com Mechanical Turk Project™ vs. Facebook™. University of Washington School of Medicine, 2012. Medical Student III Poster Session.