CALL Resources | A Guide for Evaluating Language Learning Software

John de Szendeffy, 11/5/2007

This guide presents factors to consider in evaluating the practicality of tutorial CALL software for purchase and implementation. It is decision-based and is intended to be used in light of specific environments and populations in which the software will be used. There are other guides based on principled approaches for evaluating the pedagogical value of a program (see References).

 

Technical considerations

Platform. What operating system will the software run on, Windows, Macintosh, Linux? Which version of each of these: Windows XP, Vista, etc.; Macintosh OS 9.x, OS X, etc.?

Hardware. What are the hardware requirements for the computers?

• processor type and speed (in Mhz or Ghz)
• hard drive space needed
• memory (RAM)
• network (Ethernet—10Mb/s, 100Mb/s, or 1,000Mb/s)
• CD, DVD, or floppy drive
• special or high-performance video card or sound card
• speakers, headset, or microphone (line level input or USB?)

Installation. What technical expertise is required of the person installing, configuring, and supporting it?

Compatibility. What upgrades to the existing hardware, system software, or network configuration is required? What impact will this have on the use of existing programs?

 

Content

Level: Is the level of the material appropriate for the users’ levels? Is there a range of levels possible? Is the language used in the instructions and reference material consistent with the level of the exercises?

Organizing principle: What is the organizing principle: lexical, grammatical, thematic, functional, other? Does this fit with your language teaching approach?

Skills. What particular skills does the program exercise?

Activities. Is the program interactive enough to maintain student interest, such as with a variety of exercise and quiz types, or is it primarily a one-way flow of pedantic information, such as in reference material?

Vocabulary: Does it focus on high-frequency or low-frequency words (see word frequency tool at wordcount.org)? Concrete or abstract vocabulary? In meaningful groups or clusters?

Methodology: Does it follow a particular language teaching method, such as audio-lingual, communicative, natural?

Culture. Is cultural information (history, literature, arts, pop-culture) included in the content? Does the program make any cultural assumptions about the context of its use or the background of its user? Would any of these be offensive, inappropriate, or misleading to the intended users?

 

Usability

Ease of use: Do faculty and students need to be trained in the use of the software or is it more straightforward and self-explanatory?

Interface. How intuitive is the interface design? Does it use common symbols for functional icons?

Navigation. Are there navigation tools on every screen or quick access to the main menu?

Feedback: If it offers instant feedback to questions, does the student have a second chance to get the question right instead of getting an instant “right” or “wrong”?

Score access: Do students have access to their scores on test portions and can they be saved to turn into the teacher?

Workbooks. Can students pause their work and resume it later, such as through saving their progress in a workbook file?

Reference. Is context-sensitive reference material linked to the exercises (material specifically related to the grammar point of the question)?

 

User

Profile. Is the program designed for a specific user profile? Does it match your students closely enough? Can it be used by pairs or small groups? Consider:

• age
• gender
• native language (bilingual or TL only?)
• education
• environment (foreign language, second language, language for specific purposes)
• interests
• computer experience and skills

Teachers: Does the program reflect teacher preferences for software?

Mode of use: Will it be used for classes, self-access, or both?

 

Cost

Licenses. How many computers will the software run on? Unless otherwise stated, software is sold with a license to be legitimately installed on only one computer. Multiple-use licenses include “lab packs,” a network license, and a site license.

Upgrades. Is this software version the most recent? Does this purchase entitle you to free or reduced-price upgrades in the future?

Meeting requirements. Does the software require upgrades to the existing system software used on the computers or the hardware they are currently configured with?

 

General installation and operation options for software

CD/DVD. Most commercial programs are distributed on CD, a media that holds up to 700MB of data. Some software is also offered on DVD, which holds much more data—from 4.7GB to 8.3GB (dual-layer)—particularly relevant for additional video. Some of these programs can be installed on the hard drive and used without the CD or DVD, while others require the presence of the disk in the drive to run. CD- and DVD-based programs benefit from easy installation and use, a critical factor for small language programs with little technical expertise or support. They can also be deployed in non-networked labs or the students’ own computers. Drawbacks to CD- or DVD-based use include the possibility of equipment damage (to the trays) and loss due to theft or misplacement.

Local. A program is local if it is installed from CD or DVD or Web download onto each individual computer’s hard drive (known as the local drive) and run thereafter without the CD or DVD inserted in the computer.

Network. Some programs are installed onto or copied to a server and run from there. Student computers in a lab (clients), connect to this server and launch the application, though it is not installed on the individual client computes. Large media files, especially video, can make network access to these programs problematic and may lead to a disappointing user experience when compared to local or CD access. The installation and configuration of networked programs also requires more technical expertise, additional hardware (server) and capabilities (local network infrastructure), and limiting access to users according to the site license agreement with the publisher.

 

Other References

Calico Software Review Guidelines

Language Learning Software Evaluation: Top-down or Bottom-up? (Asian EFL Journal)

Software Evaluation Guide (National Center for Literacy Education)

Courseware Evaluation, Development, and Implementation (Philip Hubbard)

A Place to Start in Selecting Software, Deborah Healey, Oregon State University and Norman Johnson, Lane Community College.