ECONOMIC & SOCIAL RESEARCH COUNCIL

CAQDAS Comparison

CAQDAS Comparison / Stability & Speed

One of the greatest advantages of CAQDAS are their efficiency. Instead of cumbersome searches through filing cabinets or myriads of loose cards, you can search for your data or notes at the ease of a mouse click. However, CAQDAS are not always the fastest or even the most stable programs. Here we document our experiences with the stabilty and speed of the different programs.

We measured the stability and speed of the programs using a 2GHz Pentium 4 computer with 512MB RAM running the Windows 2000 operating system with several, typical Office programs running in the background: A mail program (Qualcomm Eudora), an inter browser (Mozilla Firefox), a Word processor (Microsoft Word), a statistical package (SPSS), and Adobe Acrobat. Each CAQDAS was supplied with 2,626 plain text (in the case of MAXqda, which does not allow for plain text: rich text) documents, which ranged from 30 Bytes to 10 Kilobytes in size. All features of the programs were then tried out over a period of several days and the number of crashes were noted down. Since the differnt programs have different ranges of functions, the crash figures are not fully comparable: A program might have a bug in a function some of the other programs might not even offer. Likewise, different versions of Windows might yield very different results. Yet, we feel that a program should be bug free to ensure a smooth working process. Using the standard Windows Task Manager, the CPU time was measured. Finally, we ran simple, Boolean and proximity text searches and measured the time it took to perform these searches.

Except for MAXqda, all CAQDAS frooze at least once, i.e. did not respond to user input after a waiting period of 3 hours. It therefore seems advisable to always create backup copies of the project one is working with. Kwalitan was the least stable program. Most of its bugs did not crash the program entirely, but often just displayed error messages — some of them in Dutch — which require a subsequent restart of the program. While there is little loss of information, this can become quite a nuisance. NVivo, which to be sure is not intended for the amount of data we used in the test, tended to use up so much CPU time, that it quasi frooze, when some functions (such as the display of coding stripes during coding or editing) were performed. Much of the search and coding functions moreover became unusable, once a project surpassed a few hundred files. Below that limit, the program worked fine. ATLAS.ti was considerably more stable, but it too ran into time problems, when searches and codings were performed over several hundred files. Qualrus improved yet again over ATLAS.ti, both in terms of stabilty and in terms of speed for those functions it offered an equivalent to ATLAS.ti. Finally, QDA Miner and MAXqda both were the most stable and the fastest CAQDAS.


  Stability CPU Usage Simple Search Complex Search Autocoding
ATLAS.ti 5.0 medium (4) high 6'20 12'40 12'20
E6
HyperRESEARCH 2.6
Kwalitan 5.09 very low (19)1 low 24'30" 27'30" n/a
MAXqda 2 stable (0) low 0'10" 0'19" 0'17"
N6
NVivo 2 medium (6) extremely high 20" >3h n/a3
QDA Miner 1.0.15 high (1) low 0'14" 0'17" 0'40"4
Qualrus 2.0.4.0 high (2) low to medium 0'23" 1'04 n/a5
TAMS 2.50b5
Non-CAQDAS benchmark programs
InfoRapid Cardfile stable (0) very low 0'20" 0'26" n/a
Textpack
TCAS

  1. Additionally, Kwalitan frequently produced error messages, which did not correspond to actual errors.
  2. When displaying coding stripes, NVivo frequently freezes.
  3. As seaches could not be completed, autocoding time could not be measure.
  4. QDA Miner does not show, when coding is finished.
  5. Autocoding in Qualrus requires prior definition of segments.

media methods  > research >  software evaluation > caqdas > comparison > stability & speed contact