Review Process
BUCLD Review Process
 We send each abstract to 5 reviewers.
 Reviewers rate each abstract independently on a scale of 17 (doubleblind procedure), and optionally submit comments for the authors.
 We calculate two scores for each abstract: a. Mean raw score b. Mean z score
 We rank each abstract by raw score and z score, and calculate a composite rank.
 We select about 69 talks, 108 posters, and 11 alternates.
 Acceptance rates for recent BUCLDs
ASSIGNMENT OF ABSTRACTS TO REVIEWERS
Abstracts are individually assigned to reviewers by the BUCLD faculty advisors, with the help of an automated computer program, on the basis of information indicated by authors and reviewers. The submitting author of each abstract selects codes for the content area of the abstract, the types of learners represented, and the languages studied. Each reviewer similarly indicates his/her expertise in content areas, types of learners, and languages. Also taken into account are the following criteria:
 Ensure that the reviewer is sufficiently familiar with the content of the abstract.
 Ensure that the reviewer is not unfriendly to the theoretical perspective of the abstract.
 Don’t assign abstracts to reviewers who are colleagues, students, advisors, close friends, or enemies of the authors (insofar as we know this).
 Each reviewer gets between 7 and 20 abstracts.
2. Reviewers rate each abstract independently on a scale of 17 (doubleblind procedure), and optionally submit comments for the authors.
RATING GUIDELINES
Reviewers are asked to use the criteria as outlined in the guidelines section and as appropriate for the abstracts they evaluate. Note that not all criteria will apply equally well to each abstract.
3. We calculate two scores for each abstract:
a. Mean raw score b. Mean z score
RAW SCORE
Definition:  Score out of 7 from a reviewer. 
Assumption:  Every reviewer’s use of a particular score category is equivalent. 
Problem:  May be misleading if a reviewer is particularly lenient or stringent in their ratings. 
Z SCORE
Definition:  Standard score indicating how far, and in what direction, a given raw score deviates from the mean of all the raw scores assigned by a given reviewer. 
Assumption:  Every reviewer’s use of a particular score category may NOT be equivalent. Some reviewers may be more demanding or lenient than others, or may use a restricted range. 
Problem:  It may be misleading if a reviewer receives a set of unusually excellent or unusually terrible papers. (The z score effectively forces the ratings from a given reviewer to fit a bell curve.) 
4. We rank each abstract by raw score and z score, and calculate a composite rank.
SAMPLE ABSTRACT RANKING DATABASE
5. We select 72 abstracts to be presented as papers, 12 as alternates, and 72 as posters.
PAPER SELECTION PROCESS
From the set of abstracts designated as “paper only” or “either paper or poster”:
 We select the top 50 abstracts from the raw score list, and the top 50 abstracts from the z score list. This totals 6070 abstracts (there is a lot of overlap between the two sets).
 We create a pool of the next 40 abstracts based on composite rank.
 We select abstracts from the pool to complete the program of 72 papers, based as much as possible on composite rank, with the goal of forming coherent sessions.
ALTERNATE SELECTION PROCESS
From the set of abstracts designated as “paper only” or “either paper or poster” which were indicated by the authors as possible alternates:
We select 12 alternate abstracts from the remaining abstracts in the pool, based as much as possible on composite rank, with the goal of getting a good distribution of content areas.
POSTER SELECTION PROCESS
From the set of abstracts designated as “poster only” or “either paper or poster”:
 We eliminate all abstracts already selected as papers.
 We select the top 72 remaining abstracts based on composite rank.
6. Acceptance rates for recent BUCLDs
Abstracts Submitted  Abstracts Accepted*  Acceptance Rate  
2001  298  90  30% 
2002  277  90  33% 
2003  314  133  42% 
2004  386  133  34% 
2005  390  133  34% 
2006  526  153  29% 
2007  466  153  33% 
2008  479  153  32% 
2009  519  153  29% 
2010  423  153  36% 
2011  479  153  34% 
2012  514  153  30% 
2013  447  154  34% 
2014

555  188  34% 
2015

490  188  38% 
*Does not include the 12 alternate papers. BUCLD began having posters in 2003.