Evaluating Technology Using...Checklists?

Last week, I discussed evaluating technology and came to the conclusion that I really had no idea what it means to do so.  In fact, much of my post came back to the idea that I--as well as the students we teach--do not even put much thought into what goes into evaluating, as we take our technology for granted. Indeed, we can evaluate so unconsciously that when we are asked to describe what makes a good website for anything (including learning English), we blank.

When it comes to websites, tools, applications, and so on for learning English as a second language, there are actually many approaches we can use, but I would like to discuss one in detail: checklists. When they first came up in class, again, my immediate reaction was to think that it seemed obvious, but yet again, there is more to it.

We were given an example in class, and it was then that I realized that these checklists, such as Son's model, are extensive. They can include 15, 30, and sometimes more criteria, each of which is ranked on a scale rather than the typical boxes.


Source: http://www.ncvps.org/index.php/students/student-checklist/

There are some obvious benefits. Checklists are straightforward. They're a well-organized way to evaluate. As we see with technology, we tend to like things that are easy to use.

However, there is also a downside: checklists are not exclusive. That is, each institution will need a different checklist that matches up with the objectives of that program. Sure, there are some checklists that are marketed as being usable by different institutions, but those tend to be general, sometimes including questions like: Does the website match the objectives of the course? While that is always good to ask, the question itself ignores the most critical element: the objectives themselves.

Another downside to checklists is that, save for the ones that use broad questions, checklists are static; they do not change as technology changes. In a world where even your laptop can be obsolete in a matter of months, there needs to be a consistent effort to evaluate and reevaluate technology often.

As such, checklists are useful, but need to be used with careful planning. Criteria needs to be up-to-date so that programs are relevant to students. Otherwise, a poor evaluation--one that does not keep students in mind--can lead to a less than helpful learning experience.