citrus bulletin of information technology research
a citrus project - www.citrus.ac.nz
National Advisory Committee on Computing Qualifications


this issue  home  back issues  about BITR 

A Review of Managing IT Research on the Internet in New Zealand
Bulletin of Information Technology Research. Vol 1, Issue 1 (June 2003). ISSN 1176-3108.

Claudius Benedikt Hildebrand, Sebastian Simmer & Kay Fielden
School of Computing and Information Technology
UNITEC
Auckland, New Zealand
research@a66.de, S.Simmer@gmx.net & kfielden@unitec.ac.nz

Abstract

This paper contains a literature review of the research in the field of managing information technology in New Zealand and Australia published since 1998. Searching skills were required to locate New Zealand papers in international publications. These research papers are categorised both by research method used and by domain area within the field of managing information technology. Articles were gleaned from conference papers, journal articles and online publications, 57% of which were peer-reviewed. The research articles are split into three sub-domains: Knowledge Management (KM), e-Markets and Supply Chain Management (SCM). Strategic Issues dominate the KM sub-domain, implementation and performance measurement are the dominating topics in e-markets and implementation is the main topic in SCM.

Conferences are identified as the most valuable sources of research, while university websites have only a disappointingly low amount of research on-line. The general focus of the research in this relatively new field concentrates mostly on problem definition and understanding that is explored mostly via secondary research. Validated implementation tools and models are identified as one of the main fields where additional research is required. This sample of papers was found to utilise qualitative more than quantitative methods. More quantitative and mixed method research could improve the validity of most of the frameworks and models. The predominance of case studies can be understood as they are a useful tool to investigate the complex systems, but broader data basis, achieved for example by surveys, would be desirable.

Keywords

Literature Review, Research methods, Managing Information Technology (MIT)

Introduction

This literature review covers three areas of managing information technology: Knowledge Management (KM), e-Markets and Supply Chain Management (SCM). Methods used to locate research articles are discussed first including which resources were used to retrieve the articles. Problems with retrieving articles are then discussed and how these problems were overcome. Possible biases in this particular sample of research articles influenced by the sampling method in this domain of managing information technology are discussed next. The classification of research methods found is described, analysed for applicability and timeliness. An overview of the topics covered in this sample of research articles is shown in the literature map (figure 7). This paper concludes with a critique of content, gaps, research methods presentation and style of the retrieved papers.

Literature Review Methodology Resources Used

The forty articles included in this literature review were located in full-text databases, scholarly journals available at UNITEC and resources available on the Internet. Because management magazine articles influence the state of the field they are writing about some of these non-peer reviewed articles have been included.

Retrieval of Material

Retrieving material from New Zealand or Australian authors turned out to be a rather complicated task. This was mainly due to the following reasons: full-text databases do not support searching for the country of origin of the publication; the academic background of the author is normally not a searchable field; research conducted in New Zealand and Australia does not necessarily have the word New Zealand or Australia in its searchable fields, its abstract or even its body; and

it is not straightforward to identify research conducted by Australian or New Zealand's authors if this research is published in overseas journals without any specific link to New Zealand or Australia.

The following list contains some of the methods used to retrieve research: manual scanning of available New Zealand journals in the UNITEC library; keywords of New Zealand and Australian cities as well as the countries' names itself in database searches;scanning universities of both countries for downloadable material or links to downloadable material; and downloading research reports of major New Zealand and Australian universities to obtain a list of available publishing authors. Initial focus was not on specific types of research within managing information technology so that the sample was not unduly biased. The preference was for peer- reviewed articles.

Limitations and Possible Biases

One of the biggest problems with this literature review was narrowing the scope to New Zealand and Australian research without creating major biases in the retrieved research. International databases do not allow the possibility to limit searches to New Zealand authors or magazines. Articles published by New Zealand authors in non-New Zealand magazines without any explicit relationship to New Zealand were difficult to locate. The limited availability of overseas journals (free of charge) either online or within the UNITEC library increased the chances of biases. It appears that this sample of research articles may be biased towards case study oriented, practical research published in New Zealand. Mainly theoretical research without any association to New Zealand that was conducted by New Zealand authors in overseas journals may be under-represented.

Research Methodologies

Only three papers (6%) of the sample located used a quantitative method and no experimental research articles were found. All of the quantitative research articles found used statistical methods that gathered data with a survey or questionnaire (for example, Mackay, Altmann, and McMichael (2003) researching the efficiency of Australian e supply chains and e-businesses).

89% (36 out of 40) research papers found used a qualitative approach. Only two of the qualitative research papers used an ethnographic approach (for instance Birkenkrahe, 2002) who described his long-time studies of knowledge management as an insider in the consulting industry to identify likely changes and future prospects of knowledge management.)

13 out of the 36 qualitative research articles (36%) used case studies to investigate real-world implementations of knowledge management and supply chain management systems. Moody and Shanks (1999) for example study a knowledge management project that was perceived to be very successful by its participants.

A narrative approach was adopted by 2 of the 36 qualitative research papers. For instance, Bland (2001a) interviewed experts in the field of implementing knowledge management systems and found ten general recommendations for knowledge management system implementations.

Surveys in qualitative research normally have smaller numbers of participants as in quantitative research. They usually contain open-ended and hard-to-quantify questions. 3 of the 36 qualitative research papers in this sample used surveys. A good example is Steins and Hawking's (2002) research paper that surveys e-commerce drivers and barriers of Australian companies. Rather than providing the companies with specific quantifiable categories, surveys were sent out that asked the companies to provide the main drivers and barriers to e-commerce themselves.

Secondary Research

Much of the qualitative research is based on secondary research. Prior research is reviewed and refined. Many secondary research papers combine existing theories and the development of theories in the literature with actual cases and developments. They indicate weaknesses in the existing theories and by using their observation and data that has not been used for the original theories and construct new models for these cases. Secondary research is the biggest subgroup within qualitative research. 16 of the 36 (44%) qualitative research papers have been categorised in this subgroup. A good example is Boven's (1999) research paper. He investigated the increasing environmental instability of companies by providing a conceptual framework that describes how companies can use these changes in order to gain strategic advantage. This method of challenging the assumptions of existing models, modifying the assumptions and deriving changed models was found to be very common in this ample of research articles. The distribution of these subgroups is shown in figure 1.

Figure 1. Distribution of the subgroups within qualitative research

Mixed Methods Research

Only the article of Bland (2003) could have been assigned to this category.

Distribution of Research Methods

The majority of the research found was qualitative (89%). Only 3 research papers follow a quantitative approach (8%) and only one paper had a mixed methods approach (figure 2).

Figure 2. Types of Research Types of Publication Review Process

57% of the research in this literature review was from peer-reviewed sources. In addition to that, many papers have been retrieved from conference websites that gave no insight into the reviewing process and therefore the articles could not have been categorised "peer reviewed". However it can be assumed that a certain degree of reputation in the field of study is necessary in order to be able to publish for conference reviews. Figure 3 shows the distribution of peer-reviewed articles in contrast to non-peer reviewed articles.

Figure 3. Relation of peer reviewed to non-peer reviewed articles Form of Publication

All sources of the annotated bibliography were classified into the following categories: periodical articles published in journals and magazines; conference papers presented and published by conference organisers; university working research papers made available on a university's website (these papers have not usually been published in any other form); government publications; and On-line publications. This distribution is shown in figure 4.

Figure 4. Distribution of the Types of Publications Types of Resources

One method of resource distinction is distinction into primary and secondary resources. "A primary source gives the words of the witnesses or the first recorders of an event. Primary sources include manuscripts, archives, letters, diaries, and speeches. [...] Secondary sources are descriptions of the event derived from and based on primary sources" (Lorimer, 1996). The distinction between primary and secondary resources appeared to be fuzzy. As a general rule, primary resources can be considered more valuable as new data is collected.

Normally every research paper contains secondary research that is used as a basis for the following primary research. 60% of these research papers was classified as primary research and 40% as secondary. Figure 5 shows this distribution.

Figure 5. Relation between primary and secondary research Timeliness of the Resources

Articles in this literature review have an average age of 2.15 years. The Publication date range was between 1998 and 2003 whereas publications from 2003 were categorized to be 0 years old. Figure 6 shows the distribution by publication year.

Figure 6. Distribution of the publication dates Content Analysis

The purpose of this literature review is to give an overview of the "ongoing dialogue in the literature about a topic" (Creswell, 2003, p.30). The literature map shown in Figure 7 provides an overview over the topics of the research in New Zealand and Australia in the chosen domain.

Critique

This section critically reflects and critiques the retrieved research for this literature review. The purpose of this section is to identify strong and weak points in the research work in New Zealand and Australia and to identify opportunities for new research.

Availability of Research

The richest source of information for this literature review was journals and conferences. Conference websites were found to be a rich resource for up-to-date research in this domain area. New Zealand and Australia institutions have hosted a number of Computing and Technology conferences during the past 5 years.

Periodicals, especially scholarly journals, publish most of universities' research that is conducted in New Zealand. However, there are very few scholarly journals in New Zealand and most authors seem to prefer to publish overseas. The websites of universities did not provide many published full-text research outputs. Almost no university published their research on its website. Usually a list of all publications was retrievable but full-text availability was not available. Often the list of publications was retrieved from a sub-page of the publishing school or faculty. It is acknowledged that authors who publish research periodicals cede their copyrights; therefore they are normally not allowed to publish their work full-text on university websites. However, as academic creditability of a university is mainly reached by publishing recognisable research, it is hard to understand why the universities do not show more interest in making their research available to the public. The only real exception found was the Business Faculty of the University of Deakin, Australia (www.deakin.edu.au) that publishes most of their working papers online.

Large consultancy companies were also active in this research domain. However, due to the fact that they are global, they prefer to perform research in their country of origin or in big markets like the USA, Japan or Europe. Smaller, local New Zealand consultancies do not appear to publish research online (free of cost).

Content of the Research

Managing information technology, particularly KM, e-markets and supply chain management are relatively new research domains that are evolving rapidly. This fact holds challenges for the research community and research appears to fall behind the developments in practice. Topics, assumptions and models change so fast that they often become outdated several times before research is completed.

The retrieved research articles show research activity in defining the benefits of KM, e-markets and CRM and the impacts they have on strategy. Analysing the practical implementation of knowledge management systems was another major domain.

Large consultancies and the research academic community are very active in researching the benefits and success factors for knowledge management and how it can meet the challenge of increased environmental uncertainty and high rates of change. More practically oriented studies conducted in businesses show problems with knowledge management and SCM. These studies audit the benefits of the implementations and compare them against the promised benefits. There are problems that are identified very commonly, for example most knowledge management systems fail because technology is the driving factor and not the business needs.

However, there appear to be few frameworks for successful implementation of knowledge management and SCM systems. The understanding of the principal benefits of knowledge management systems and the identification of common problems do not appear to have been transferred into models for leveraging the benefits yet. No models were present in this sample that closely investigate the required properties a company must have and the processes it must use in order to benefit from implementing the discussed systems. This appears to indicate that the research community is in a state of problem definition rather than a state of constructing theories in this field.

Research Methodology

Traditionally, management sciences employ qualitative research methodologies. Managing IT is not a technical field and the research concentration should be on the special properties of technology as a manageable resource. Hugh (1997) defines operational research and management science as follows:

Management science […] is the science that (1) observes the phenomena of operations (or functions) consisting of people, material, artifacts of technology and society, and money, these operations being given coherence by an objective, a need, or a problem, and (2) gives due consideration to the known, unknown, and implicit perceptions of the human actors in these operations. It represents those phenomena in models (or theories), and manipulates them to create understandings of the phenomena. With the intent of achieving desired ends, OR/MS workers then apply these results to problems in the reality from which the observations emerged.

The complexity of the studied field and aspects like the "implicit perceptions of the human actors" (Hugh, 1997) imply that a great amount of research has to be qualitative as it is hardly possible to research these factors with quantitative methods.

It is understandable and very reasonable that most of the research uses qualitative methodologies However it was very surprising to discover the very limited amount of quantitative research being done in this sample of forty papers. It seems that the promised benefits of SCM and KM systems predicted by theoretical and perceptive research are not being realised in practise. Quantitative studies could help testing hypothesises, for example that the strengthening of communication links (independent) improves the customer response time (dependent).

Mixed method approaches would be especially valuable for this purpose: Qualitative research methods could be used for obtaining a holistic view of the problem. A theory could be build of how the proposed system benefits influence specific company variables and quantitative methods could be applied to test the theory.

However it must be acknowledged that mixed method or quantitative approaches in this field of study are hindered in a variety of ways. Firstly, the proposed systems bring mostly intangible benefits. Quantifiable variables could be found that express these non-quantifiable benefits (e.g. customer response time, amount of site views in the electronic knowledge repository), however this must be based on the assumption that these quantifiable variables are really an appropriate expression of the intangible benefits. This assumption lowers the significance of the quantitative studies greatly.

Mixed method approaches are generally more time-consuming. The pace of change and the novelty of the field greatly hinder mixed-method approaches.

Qualitative Research Approaches

Case studies are the second biggest qualitative research subgroup. Case studies in this literature review normally investigate specific implementations of KM, SCM or e-market systems. Case studies are especially useful for examining implementation of these systems as the systems are designed to build complex structures in complex environments. The systems' benefits are often intangible and hard to measure. Case studies are useful, unobtrusive tools for measuring the implementations in detail in their natural environment.

The academic value of the studies is however limited. A case study can provide the researcher with ideas and possible reasons for problems and solutions. Because of their very limited data basis (which is one) however the results cannot be generalized. The case study of Moody and Shanks (1999) is very interesting, but the findings in a non-governmental institution in the medical sector can hardly be used for generalization (e.g. it will hardly apply to Ford). The number of case studies in this literature overview can provide a researcher in this subject with ideas about what may happen when implementing a KM or SCM system. Case studies provide an ever-increasing database in this area upon which research trends can be studied.

Conclusion

This literature review has analysed research in the field of Managing Information Technology in New Zealand and Australia. This analysis shows a predominance of qualitative research in the papers reviewed. Secondary research and case studies dominate the qualitative research. Retrieval of literature specifically from Australia and New Zealand proved difficult, not necessarily because of the amount of literature published but more because of retrieval problems. It would be desirable if future databases became more powerful with additional retrieval fields to reference specific countries.

There appear to be many research opportunities within the field of study. Quantitative and mixed methods research is certainly under-represented in this sample and the opportunity to evaluate many of the given frameworks is still to be taken. Evaluations of successful system implementations appear to be rare or non-existent.

New Zealand and Australia has a very active research community that frequently publishes research in Information Technology Management. There appears to be many opportunities for additional research in this domain. Considering the importance of KM, e-markets and SCM (many researchers claim that knowledge is the only lasting competitive advantage) and the obvious uncertainties within these fields, good additional research can make a substantial contribution to the development of New Zealand's and Australia's economy.

Figure 7. Literature Map of the retrieved research

References



this issue home | back issues index about bitr  | about citrus

Copyright © 2003 The CITRUS Charitable Trust. All rights reserved.
Individual articles remain the property of the authors.