About Us

News & Events

Dr. Naomi Caselli, along with researchers at San Diego State University and Tufts University, and a team of software engineers at Boston University’s Rafik B. Hariri Institute for Computing and Computational Science & Engineering are preparing to launch ASL-LEX 2.0, the second-generation interactive visualization of the lexical and phonological properties of signs in American Sign Language. ASL-LEX has interdisciplinary applications for research in linguistics, education, and computer science, and also is widely used by educators and people interested in learning ASL.

The database has been built and maintained by Dr. Caselli, Dr. Zed Sevcikova Sehyr and Dr. Karen Emmorey from San Diego State University and Dr. Ariel Cohen-Goldberg from Tufts University. The first iteration debuted in 2016 as the first ever web-based ASL lexicon. It featured 993 ASL signs and was awarded the PopSci People’s Choice award for its interactive website visualizations in the National Science Foundation Vizzies Visualization Challenge in 2017.

ASL-LEX 2.0: A bigger, more detailed picture of the lexicon

ASL-LEX 2.0 will contain 2727 ASL signs, tripling the number of signs originally mapped in 1.0. Though there is currently no way of confirming how much of the ASL lexicon this database actually accounts for, Dr. Caselli estimates that ASL-LEX 2.0 brings the platform “much closer to having full coverage of the lexicon.”

The team built out 2.0’s expanded reach by cross-referencing peer projects that seek to document ASL. Dr. Sevcikova Sehyr and Dr. Emmorey drew on past in-house experiments, the ASL Sign Bank project, and other online ASL dictionaries, while Dr. Caselli identified further dialectical signs via focus groups held at Boston University.

Throughout the process, the team held regular meetings in which they assessed existing research for gaps in the field. One gap had to do with phonology, which Dr. Sevcikova Sehyr notes “may have been inconsistently captured due to the researchers’ own theoretical views” in prior projects. To address that lack, Caselli’s team, including Chelsea Hammond (Wheelock’17’19), worked to layer in more detailed phonological information for all 2727 signs, adding descriptive depth to visual structure of the lexicon.

“With 1.0, you could only see a sign’s starting location,” says Dr. Caselli. “Now you can see where the sign ends up.” This is valuable because it provides more points of connection between signs made up of visually-similar components, similar to how spoken-language words that share an onset or a rhyme or other structural components might relate.

The team also saw a gap in documentation of sign iconicity (how much a sign “looks like” the object or concept it represents). “One of the neat things about sign languages is that the form of signs and their meanings are sometimes closely related, they are ‘iconic.’ Tree is a good example: in the sign the arm looks like a tree trunk and the fingers look like branches,” says Dr. Caselli.

Dr. Sevcikova Sehyr collected iconicity ratings by deaf ASL signers for all signs documented in ASL-LEX 1.0. By building them into ASL-LEX 2.0, the researchers aim to establish a standardized data set that peer researchers can use as a starting point for their own questions and assessments. Dr. Caselli has used the iconicity data already in her work on sign-language acquisition, examining how sign iconicity relates to how easily—and at which age—ASL learners typically learn it in publications in Psychological Science and the Journal of Experimental Psychology: Learning Memory and Cognition.

“The iconicity ratings provide valuable information about deaf signers’ intuitions about the resemblance between the sign form and meaning,” says Dr. Sevcikova Sehyr. In an upcoming article, currently in press with the journal Language & Cognition, she and Dr. Emmorey use the data in ASL-LEX 2.0 in their exploration of how a person’s knowledge of ASL influences their intuitions about the mappings between meaning and form.

Working with SAIL: The results of a unique cross-BU partnership

Accessibility was another major priority for ASL-LEX 2.0, explains Dr. Caselli. “We are trying to make it more accessible to educators,” she says. “As we reduce the amount of English text in the interface, we lower the barrier to access to people whose native language is ASL or another language.”

Another big step toward increased accessibility happened on the technical level. Engineers from the Boston University Hariri Institute’s Software & Application Innovation Lab (SAIL) built new graphing and visualization features into ASL-LEX 2.0, making it possible for researchers and educators who have collected and coded the lexicons of other sign languages to now use ASL-LEX’s visualization tools to explore their data.

“ASL-LEX 2.0 has two visualization modes that allow for a deeper dive into the lexicon,” explains lead engineer Shreya Pandit. “The first shows the lexical database as a network graph, where users can see how signs group together based on a set of phonological and lexical properties including selected fingers, major location, flexion and so on. This offers a panoramic view of the data and allows us to view relationships between signs that are not otherwise apparent.”

“The other visualization shows the relationships between pairwise combinations of sign properties. This provides an in-depth understanding of how signs vary in different subspaces. Users can select a subset of signs and switch between the two visualization modes.”

To get from ASL-LEX 2.0’s initial prototype, assembled by the research team, to a full-featured and launch-ready version, Shreya and her peers at SAIL pushed themselves to understand the specifics of the subject at hand.

“Working with sign language is inherently different from working with spoken language,” she says. “In sign language, words are expressed using a combination of the hands, arms, face, and head. This allows sign language to be expressed in a multitude of ways, and determines how signs are related.”

“For example, rhyming signs may be spatially related (where they are signed on the body) instead of being auditorily related. Similarly, to indicate pluralization, a sign is often reduplicated instead of adding suffixes, like many spoken languages do.”

Shreya, who recently earned her Master’s in Artificial Intelligence from BU, conducted much of the work on ASL-LEX 2.0 alongside two graduate students completing internships at SAIL: Megan Fantes (GRS’20) and Arezoo Sadeghi (GRS’20). “This project has helped them to foster a better understanding of working on an active research project,” she says. “It also provided them with an opportunity to come up with a systematic way of organizing and developing the data transformations that lie at the heart of this project.”

“Via its interactive online interface, ASL-LEX 2.0 provides accessible training and learning opportunities for deaf and hearing students, scientists and educators,” notes Dr. Sevcikova Sehyr. “The database can assist researchers in conducting systematic, reliable and reproducible research, something that is absolutely crucial in the age of increased demands for scientific replicability.” This could lead researchers to probe questions about how language works in general, and test theories of language that have been historically limited to spoken languages only.

Dr. Sevcikova Sehyr also points out that ASL-LEX 2.0 may be used by educators and early intervention specialists seeking to develop benchmarks for assessing vocabulary development in signing children (e.g., do children know the most frequent signs?) and to support literacy development (e.g., to find sign-based “rhymes”).

For Shreya, working with the ASL-LEX 2.0 research team exemplified how SAIL can support data-driven research. “This project brought researchers working in computer science, cognitive neuroscience and psychology from across the nation together,” she says. “The depth of this project has allowed us to think outside the box, teaching us important lessons on how research should be made more accessible and understandable.”

On the horizon: Adding meaning to the lexicon

By the time ASL-LEX 2.0 officially launches, the research team will be well on their way toward developing its next iteration, which was recently funded by the National Science Foundation. “3.0 will go beyond phonological mapping by including semantic relationships between signs,” says Dr. Caselli. “Even though signs for ‘cat’ and ‘dog’ don’t have much of a visual similarity, their meanings are related, and these relationships will be mapped in the next iteration of ASL-LEX.”

Dr. Sevcikova Sehyr hopes develop ASL-LEX 3.0 into a “lexicon-wide network of how signs are interconnected via their meaning as well as their physical form, and how meaning and form possibly influence each other.” She has proposed a study where the team will ask deaf ASL signers to think of signs that are associated by meaning to the signs in ASL-LEX. “This will tell us how many close or distant neighbors or ‘friends’ each sign has, how strong and diverse their neighborhoods are, or what attracts them to some friends but not others,” she says. Caselli adds that “ASL-LEX 3.0 will also help us know not only what these signs mean, but what parts of meaning show up in the sign forms.”

Lexical databases which document these relationships have been critical to the development of the natural language processing technology that makes applications like Siri and Alexa possible for spoken languages. Sign language recognition technology has lagged behind, but by documenting the same relationships in ASL ASL-LEX opens the door for similar Artificial Intelligence developments sign recognition and automatic translation software.

A research team that included Dr. Caselli and authors from Microsoft, Gallaudet University, Rochester Institute of Technology, Paris-Saclay University, University of Maryland, and Leiden University examined the current state and future challenges of successful sign language recognition, generation, and translation systems in their paper, “Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective,” which won the ASSETS 2019 Best Paper Award.

“ASL-LEX 1.0 was our attempt at getting basic information about a large portion of ASL signs into one spot, and document about six phonological features per sign,” says Dr. Caselli. “2.0 expands the number of signs we include and the number of phonological features we track — up to 15 now. What is special about 3.0 is that it will add meaning.”

With funding secured and participation from Co-PIs at San Diego State University and Tufts University on board, ASL-LEX 3.0 should become public in 2022.