Review Process
BUCLD 2011 Review Process
- We send each abstract to 5 reviewers.
- Reviewers rate each abstract independently on a scale of 1-10 (double-blind procedure), and optionally submit comments for the authors.
- We calculate two scores for each abstract: a. Mean raw score b. Mean z score
- We rank each abstract by raw score and z score, and calculate a composite rank.
- We select 81 abstracts to be presented as papers, 12 as alternates, and 72 as posters.
- Acceptance rates for recent BUCLDs
ASSIGNMENT OF ABSTRACTS TO REVIEWERS
Abstracts are individually assigned to reviewers by the BUCLD faculty advisors, with the help of an automated computer program, on the basis of information indicated by authors and reviewers. The submitting author of each abstract selects codes for the content area of the abstract, the types of learners represented, and the languages studied. Each reviewer similarly indicates his/her expertise in content areas, types of learners, and languages. Also taken into account are the following criteria:
- Ensure that the reviewer is sufficiently familiar with the content of the abstract.
- Ensure that the reviewer is not unfriendly to the theoretical perspective of the abstract.
- Don’t assign abstracts to reviewers who are colleagues, students, advisors, close friends, or enemies of the authors (insofar as we know this).
- Each reviewer gets between 7 and 20 abstracts.
REVIEWERS FOR BUCLD 36
| Nameera Akhtar Shanley Allen Ben Ambridge Inbal Arnon Richard Aslin Jessica Barlow David Barner Edith Bavin Misha Becker Heike Behrens David Birdsong Gerard Bol Patrick Bolger Ellen Broselow Nancy Budwig Ann Bunger Helen Cairns Catherine Caldwell-Harris Kyle Chambers Molly Collins Erin Conwell Peter Coopmans Stephen Crain Suzanne Curtin Barbara Davis Cecile De Cat Kamil Deen Laurent Dekydtspotter Heiner Drenhaus Ken Drozd Nigel Duffield Catherine Echols Neiloufar Family Anne Fernald Michael Frank Maria Joao Freitas Alison Gabriele Anna Gavarro Lisa Gershkoff-Stowe Judit Gervain Heather Goad Adele Goldberg |
Roberta Golinkoff Janet Grijzenhout John Grinstead Theres Gruter Ayse Gurel Martin Hackl Cornelia Hamann Makiko Hirakawa Kathy Hirsh-Pasek Miren Hodgson Barbara Hoehle Bart Hollebrandse Yi Ting Huang Felicia Hurewitz Nina Hyams Tania Ionin Elizabeth Johnson Rene Kager Dorit Kaufman Nina Kazanina Evan Kidd Grzegorz Krajewski Tanja Kupisch Usha Lakshmanan Laura Lakusta Donna Lardiere Thomas Lee Beth Levin Casey Lew-Williams Juana Liceras Jeffrey Lidz Heather Littlefield Conxita Lleo Molly Losh Theo Marinis Lori Markson Danielle Matthews Rachel Mayberry Corrine McCarthy Tamara Nicol Medina Luisa Meroni Toben Mintz |
NMaria Mody Silvina Montrul James Morgan Alan Munn Julien Musolino Letitia Naigles Thierry Nazzi Elissa Newport Claire Noble Rama Novogrodsky Cathy O’Connor William O’Grady Mitsuhiko Ota Seyda Ozcaliskan Anna Papafragou Johanne Paradis Lisa Pearl Sharon Peperkamp Ana Perez-Leroux William Philip Colin Phillips Bernadette Plunkett Philippe Prévost Clifton Pye Jennie Pyers Marnie Reed Mabel Rice Judith Rispens Tom Roeper Jason Rothman Monika Rothweiler Caroline Rowland Phaedra Royle Tetsuya Sano Lynn Santelmann Teresa Satterfield Cristina Schmitt Petra Schulz Carson Schutze Bonnie D. Schwartz Nuria Sebastian Galles Amanda Seidl |
Ann Senghas Joan Sereno Valerie Shafer Rushen Shi Yasuhiro Shirai Leher Singh Roumyana Slabakova William Snyder Melanie Soderstrom Hyun Joo Song Antonella Sorace Rex Sprouse Jeffrey Steele Carol Stoel-Gammon Kristen Syrett Kriszta Szendroi Helen Tager-Flusberg Anne-Michelle Tessier Margaret Thomas Rosalind Thornton Ruth Tincoff John Trueswell Ianthi Maria Tsimpli Sharon Unsworth Sigal Uziel-Karl Elena Valenzuela Heather van der Lely Angeliek van Hout Spyridoula Varlokosta Joshua Viau Laura Wagner Daniel Weiss Juergen Weissenborn Lydia White Elizabeth Wonnacott Charles Yang Chen Yu Andrea Zukowski Kie Zuraw Barbara Zurer Pearson |
2. Reviewers rate each abstract independently on a scale of 1-10 (double-blind procedure), and optionally submit comments for the authors.
RATING GUIDELINES
Reviewers are asked to use the following criteria, as appropriate, for the abstracts they evaluate. Note that not all criteria will apply equally well to each abstract.
- Is the question or issue clearly stated?
- Is the significance of the work clearly stated? Is relevant previous work appropriately cited?
- If relevant, are the method, data collection, and analysis procedures well-designed and appropriate to the question addressed?
- Is the conceptual framework coherent? If relevant, is the theoretical analysis well-argued?
- Is the work original? Does it present new data (if relevant), particularly from less-studied languages?
- Is the work completed, or does it show very strong promise of being completed in time for the conference?
- Are the conclusions justified in relation to the data and/or analyses?
- Is the abstract written clearly and organized well?
- Is the topic of scientific, methodological or theoretical importance?
- Is the paper timely in terms of current issues of interest in the field of language development?
- Is the paper likely to be of interest to a reasonable number of attendees at BUCLD?
3. We calculate two scores for each abstract:
a. Mean raw score b. Mean z score
RAW SCORE
| Definition: | Score out of 10 from a reviewer. |
| Assumption: | Every reviewer’s use of a particular score category is equivalent. |
| Problem: | May be misleading if a reviewer is particularly lenient or stringent in their ratings. |
Z SCORE
| Definition: | Standard score indicating how far, and in what direction, a given raw score deviates from the mean of all the raw scores assigned by a given reviewer. |
| Assumption: | Every reviewer’s use of a particular score category may NOT be equivalent. Some reviewers may be more demanding or lenient than others, or may use a restricted range. |
| Problem: | It may be misleading if a reviewer receives a set of unusually excellent or unusually terrible papers. (The z score effectively forces the ratings from a given reviewer to fit a bell curve.) |
4. We rank each abstract by raw score and z score, and calculate a composite rank.
SAMPLE ABSTRACT RANKING DATABASE
5. We select 81 abstracts to be presented as papers, 12 as alternates, and 72 as posters.
PAPER SELECTION PROCESS
From the set of abstracts designated as “paper only” or “either paper or poster”:
- We select the top 55 abstracts from the raw score list, and the top 55 abstracts from the z score list. This totals 70-75 abstracts (there is a lot of overlap between the two sets).
- We create a pool of the next 40 abstracts based on composite rank.
- We select up to 11 abstracts from the pool to complete the program of 81 papers, based as much as possible on composite rank, with the goal of forming coherent sessions.
ALTERNATE SELECTION PROCESS
From the set of abstracts designated as “paper only” or “either paper or poster”:
We select 12 alternate abstracts from the remaining abstracts in the pool, based as much as possible on composite rank, with the goal of getting a good distribution of content areas.
POSTER SELECTION PROCESS
From the set of abstracts designated as “poster only” or “either paper or poster”:
- We eliminate all abstracts already selected as papers.
- We select the top 72 remaining abstracts based on composite rank.
6. Acceptance rates for recent BUCLDs
| Abstracts Submitted | Abstracts Accepted* | Acceptance Rate | |
| 2001 | 298 | 90 (90 papers, 0 posters) | 30% |
| 2002 | 277 | 90 (90 papers, 0 posters) | 33% |
| 2003 | 314 | 133 (87 papers, 46 posters) | 42% |
| 2004 | 386 | 133 (87 papers, 46 posters) | 34% |
| 2005 | 390 | 133 (87 papers, 46 posters) | 34% |
| 2006 | 526 | 153 (87 papers, 66 posters) | 29% |
| 2007 | 466 | 153 (87 papers, 66 posters) | 33% |
| 2008 | 479 | 153 (87 papers, 66 posters) | 32% |
| 2009 | 519 | 153 (81 papers, 72 posters) | 29% |
| 2010 | 423 | 153 (81 papers, 72 posters) | 36% |
| 2011 | 479 | 153 (81 papers, 72 posters) | 34% |
*Does not include the 12 alternate papers. BUCLD began having posters in 2003.