Research in Engineering Design

Referencing Practice

Description: red_image

Motto: A journal is a product; one that requires careful design, design with input from all its stakeholders.

 

Consider yourself invited to take part in the continuous design and improvement of this journal by providing me input, for example, by commenting on this page.

Yoram Reich, PhD 

Professor

Editor-in-Chief

 

 

The goal of this page is to help researchers improve their literature review performance. It aims to cover basic ideas that could help experienced but more importantly, young researchers or students.

The difficulty to conduct quality literature review arises since we live in an era of information explosion and it is impossible for researchers to review all published information related to their subject of inquiry. Rather, researchers have to conduct a careful, selective, reading of the literature to make them aware of published material that matters.

 

 

Content:

(1)  Status

(2)  Locating material

(3)  Evaluating material

(4)  Consolidating material

(5)  Concluding from material

(6)  Summary

 

Status of referencing practice

Engineering design is a difficult subject to study because it deals with many topics that could be spread over multiple traditional disciplines such as management, economics, art, engineering, etc. Compared to other classic disciplines, we experience that there is poor practice of referencing to previous work. This leads to re-creating previously known knowledge and consequently, wasting valuable research time.

 

Locating material

The first stage of a literature review involves locating relevant sources of information. There are several ways to do this.

 

Keyword search. This is the most popular approach. Keyword search could be done in diverse databases with varying degrees of quality and scope such as ISI, Google scholar, INSPEC, EI COMPENDEX, SCIRUS and many more. For each discipline and perhaps specialization, these databases will provide different results with different degrees of relevance and quality for a particular search.

As in many activities, locating material is an art and should be practiced carefully and developed continually. An important way to develop your skills is learning to recognize which database if valuable to your needs. All search engines have simple and advanced search facilities that might need to be consulted to focus the search on more relevant results.

Always look for keywords that are synonyms or that represent different topics but related to the requested material. Relevant ideas could be found in neighboring topics. These might even turn out to be most interesting and innovative.

 

Aside from searching for material whenever it is, focused search could lead to more quality and relevant material. Such strategy usually marks an experienced researcher who is knowledgeable in the discipline and knows who are the journals that might contain relevant information; who are the leading researchers in the particular topics, in which conferences do they meet, and to which professional organizations they belong. Each such entity, whether a journal, a researcher, laboratory, conference, organization, may have a website that could be searched to find relevant information.

 

Journals. Significant material could be found related to the diverse topics covered by engineering design in journals that publish design-related papers. Some of the journals in the following list publish few design-related papers and some focus on engineering design. Some of the following journals are new and have not established themselves as mainstream, or quality academic journals and some have a long tradition of quality. Some of these journals exercise rigorous review process and some much less. It is again the task of each researcher to develop the knowledge for siting through this continually growing list.

While not an objective mark of quality, and I know some journals that are exceptions, journals listed in ISI and have impact factor, would be considered more important even only from the standpoint of promotion committees and other bodies that evaluate, rank, or score researchers or universities. There is a debate on the value of the Impact Factor in general and also specifically to engineering design, but nevertheless, researchers should be aware of this measure.

The following (incomplete) list is alphabetic aside from RED which appears first. The list length is a reflection of the difficult task:

 

Research in Engineering Design

Advanced Engineering Informatics

AI for Engineering Design, Analysis and Manufacturing

CIRP Annals

CoDesign

Computer Aided Design

Computers in Industry

Concurrent Engineering: Research and Applications

Creativity Research Journal

Design Issues

Design Management Journal

Design Management Review

Design Studies

Engineering Optimization

Engineering with Computers

Environment and Planning B - Planning & Design

IEEE Transactions on Engineering Management

IEEE Transactions on Systems Man and Cybernetics

International Journal of Design

International Journal of Design Creativity and Innovation

International Journal of Design Engineering

International Journal of Design Sciences & Technology

International Journal of Mass Customization

International Journal of Sustainable Design

International Journal of Technology Management

International Journal on Interactive Design and Manufacturing

Journal of Advanced Mechanical Design Systems and Manufacturing

Journal of Computing and Information Science in Engineering

Journal of Design & Manufactures

Journal of Design Research

Journal of Engineering and Technology Management

Journal of Engineering Design

Journal of Engineering, Design and Technology

Journal of Mechanical Design

Journal of Mechanisms and Machine Theory

Journal of Product Innovation Management

Journal of Sustainable Product Design

Materials & Design

Mechatronics

Research Policy

SDPS Transactions: Journal of Integrated Design & Process Science

Technovation

The Design Journal

 

See related note on referencing on ASME JMD website.

 

 

Second phase. Locating references through keyword search (including more subtle synonym search) as well as through focused browsing of sources will leave you with the first cut of relevant references. However, this is not enough; it is just the first phase in the search!

Valuable references appear in the bibliography of the papers found this far. In this way, papers not found in the direct search are located by going to the past. But one can easily go into the future of each paper to find which references cite it. Some of these would be exactly on the topic of interest and would make the collection of retrieved papers much more up-to-date. The process of going back and forth could be done several times thus expanding the search significantly, making sure that the chance to miss important work is reduced. This is critical doing research and finding out that the same idea has been researched before and even published is not pleasant.

 

If you do research today, your list of references should include very recent papers and a mix of others that start from the time the first paper on the subject appeared. If your list does not include papers from the last five years, for example, it could point to poor search or that the topic has lost its appeal or even worse, its importance. If you mention an idea and 5 references to it, check whether all are necessary or who is the true source; try to give credit to the actual source and not to those who cite it or that review it or that incrementally improved it.

Evaluating material

Not all references found in the search stage are valuable to understand the state-of-the-art and not all need to be cited when writing on some topic. There are papers that are published whose quality is poor and while they claim to do X, there is no evidence that indeed they accomplish their aim. Consequently, they should not appear in the review. After collecting the material in the previous stage, it needs to undergo quality check and selection to locate only those references that truly make a contribution. If a paper on X makes a statement on Y without further discussion or evidence, referring to this paper as a source for Y is flawed. If that paper refers to a third paper to back statement Y, you might as well study the third paper to see whether it needs to be cited.

Do not assume that because a paper is cited by others that it is a quality paper. There are cases of serial referencing when author A refers to a paper B and another author C copies the sentence from A regarding B without even reading B only because C needs to do a literature survey. In hot topics, some papers could collect significant referencing without been seriously evaluated. It is the responsibility of each author to carefully evaluate, select, and retain only a handful of papers relevant to the present research that truly make a difference.

Consolidating material

So we have a handful of references, quality papers that are relevant to a particular research topic. The worst thing that can happen, and it happens often is that the review reads like A said A1, B said B2, etc. extracting sentences or phrases from each paper to compose the review. Such summary hardly adds up to anything comprehensible or valuable to readers. To worth something, the papers reviewed have to be placed on a common foundation with similar language. The researcher has to interpret each such source and make sure that their similarities, differences, or other relations are made explicit and clear with one set of concepts. If done properly, this set of concepts and the consolidated review is a contribution in itself!

Concluding from material

We are often reluctant to criticize a piece of research. This leads to saying what each paper proclaims but not much else. A real review requires critical analysis of the content of each paper, identifying its scope and limitations. Such analysis is critical to identify a true gap in our knowledge one that deserves the present research or another. Failing to clearly present a gap in the literature casts doubt in the need for the present research.

Summary

Proper reviewing practice is a sign of scholarship and knowledge of the field. Present referencing practice in engineering design research is quite poor and it has serious impact on the perception of the field externally and internally. It creates an outside image that our journals are inferior to others in the classic engineering disciplines and it hurt our effort to establish credibility among our peers in engineering departments and faculties. It also makes it harder to build on previous knowledge properly and avoid re-inventing the wheel.

Proper referencing in a paper is relatively easy to check. Do it before you submit a paper to make sure, it is you who find the issues and not your reviewers.

 


Page URL: http://www.eng.tau.ac.il/~yoram/RED/redReferencingPractice.html
Last modified: 11/15/2012 8:15:49 AM