|
Research in
Engineering Design Referencing Practice |
|
Motto: A journal is a product; one
that requires careful design, design with input from all its stakeholders.
Consider yourself invited to take part in the continuous
design and improvement of this journal by providing me input, for example, by
commenting on this page.
|
Yoram Reich, PhD Professor Editor-in-Chief |
The goal of this page is to help
researchers improve their literature review performance. It aims to cover basic
ideas that could help experienced but more importantly, young researchers or
students.
The difficulty to conduct quality literature review arises since we live in an era of information explosion and it is impossible for researchers to review all published information related to their subject of inquiry. Rather, researchers have to conduct a careful, selective, reading of the literature to make them aware of published material that matters.
Content:
(1) Status
(6) Summary
Status of referencing practice
Engineering design is a difficult
subject to study because it deals with many topics that could be spread over
multiple traditional disciplines such as management, economics, art,
engineering, etc. Compared to other classic disciplines, we experience that
there is poor practice of referencing to previous work. This leads to
re-creating previously known knowledge and consequently, wasting valuable
research time.
Locating material
The first stage of a literature review involves locating relevant sources of information. There are several ways to do this.
Keyword search. This is the most popular approach. Keyword search could be done in diverse databases with varying degrees of quality and scope such as ISI, Google scholar, INSPEC, EI COMPENDEX, SCIRUS and many more. For each discipline and perhaps specialization, these databases will provide different results with different degrees of relevance and quality for a particular search.
As in many activities, locating material is an art and should be practiced carefully and developed continually. An important way to develop your skills is learning to recognize which database if valuable to your needs. All search engines have simple and advanced search facilities that might need to be consulted to focus the search on more relevant results.
Always look for keywords that are synonyms or that represent different topics but related to the requested material. Relevant ideas could be found in neighboring topics. These might even turn out to be most interesting and innovative.
Aside from searching for material whenever it is, focused search could lead to more quality and relevant material. Such strategy usually marks an experienced researcher who is knowledgeable in the discipline and knows who are the journals that might contain relevant information; who are the leading researchers in the particular topics, in which conferences do they meet, and to which professional organizations they belong. Each such entity, whether a journal, a researcher, laboratory, conference, organization, may have a website that could be searched to find relevant information.
Journals. Significant material could be found related to the diverse topics covered by engineering design in journals that publish design-related papers. Some of the journals in the following list publish few design-related papers and some focus on engineering design. Some of the following journals are new and have not established themselves as mainstream, or quality academic journals and some have a long tradition of quality. Some of these journals exercise rigorous review process and some much less. It is again the task of each researcher to develop the knowledge for siting through this continually growing list.
While not an objective mark of quality, and I know some journals that are exceptions, journals listed in ISI and have impact factor, would be considered more important even only from the standpoint of promotion committees and other bodies that evaluate, rank, or score researchers or universities. There is a debate on the value of the Impact Factor in general and also specifically to engineering design, but nevertheless, researchers should be aware of this measure.
The following (incomplete) list
is alphabetic aside from RED which appears first. The list length is a
reflection of the difficult task:
Research in Engineering Design
Advanced Engineering Informatics
AI
for Engineering Design, Analysis and Manufacturing
CIRP Annals
CoDesign
Computer Aided Design
Computers in Industry
Concurrent Engineering: Research and
Applications
Creativity Research Journal
Design Issues
Design Management Journal
Design Management Review
Design Studies
Engineering Optimization
Engineering with Computers
Environment and Planning B - Planning & Design
IEEE Transactions on Engineering Management
IEEE Transactions on Systems Man and Cybernetics
International Journal of Design
International Journal of Design
Creativity and Innovation
International Journal of Design
Engineering
International Journal of Design
Sciences & Technology
International Journal of Mass
Customization
International Journal of Sustainable Design
International Journal of Technology Management
International Journal on
Interactive Design and Manufacturing
Journal of Advanced Mechanical Design Systems and
Manufacturing
Journal of Computing and
Information Science in Engineering
Journal of Design & Manufactures
Journal of Design Research
Journal of Engineering and Technology
Management
Journal of Engineering Design
Journal of Engineering, Design and Technology
Journal of Mechanical Design
Journal
of Mechanisms and Machine Theory
Journal of Product Innovation Management
Journal of
Sustainable Product Design
Materials & Design
Mechatronics
Research Policy
SDPS Transactions: Journal of Integrated
Design & Process Science
Technovation
The Design Journal
See related note on referencing on ASME JMD website.
Second phase. Locating references through keyword search (including more subtle synonym search) as well as through focused browsing of sources will leave you with the first cut of relevant references. However, this is not enough; it is just the first phase in the search!
Valuable references appear in the bibliography of the papers found this far. In this way, papers not found in the direct search are located by going to the past. But one can easily go into the future of each paper to find which references cite it. Some of these would be exactly on the topic of interest and would make the collection of retrieved papers much more up-to-date. The process of going back and forth could be done several times thus expanding the search significantly, making sure that the chance to miss important work is reduced. This is critical – doing research and finding out that the same idea has been researched before and even published is not pleasant.
If you do research today, your list of references should include very recent papers and a mix of others that start from the time the first paper on the subject appeared. If your list does not include papers from the last five years, for example, it could point to poor search or that the topic has lost its appeal or even worse, its importance. If you mention an idea and 5 references to it, check whether all are necessary or who is the true source; try to give credit to the actual source and not to those who cite it or that review it or that incrementally improved it.
Evaluating material
Not all references found in the search stage are valuable to understand the state-of-the-art and not all need to be cited when writing on some topic. There are papers that are published whose quality is poor and while they claim to do X, there is no evidence that indeed they accomplish their aim. Consequently, they should not appear in the review. After collecting the material in the previous stage, it needs to undergo quality check and selection to locate only those references that truly make a contribution. If a paper on X makes a statement on Y without further discussion or evidence, referring to this paper as a source for Y is flawed. If that paper refers to a third paper to back statement Y, you might as well study the third paper to see whether it needs to be cited.
Do not assume that because a paper is cited by others that it is a quality paper. There are cases of serial referencing when author A refers to a paper B and another author C copies the sentence from A regarding B without even reading B only because C needs to do a literature survey. In hot topics, some papers could collect significant referencing without been seriously evaluated. It is the responsibility of each author to carefully evaluate, select, and retain only a handful of papers relevant to the present research that truly make a difference.
Consolidating material
So we have a handful of references, quality papers that are relevant to a particular research topic. The worst thing that can happen, and it happens often is that the review reads like A said A1, B said B2, etc. extracting sentences or phrases from each paper to compose the review. Such summary hardly adds up to anything comprehensible or valuable to readers. To worth something, the papers reviewed have to be placed on a common foundation with similar language. The researcher has to interpret each such source and make sure that their similarities, differences, or other relations are made explicit and clear with one set of concepts. If done properly, this set of concepts and the consolidated review is a contribution in itself!
Concluding from material
We are often reluctant to criticize
a piece of research. This leads to saying what each paper proclaims but not
much else. A real review requires critical analysis of the content of each
paper, identifying its scope and limitations. Such analysis is critical to
identify a true gap in our knowledge – one that deserves the present research
or another. Failing to clearly present a gap in the literature casts doubt in the
need for the present research.
Summary
Proper reviewing practice is a sign of scholarship and knowledge of the field. Present referencing practice in engineering design research is quite poor and it has serious impact on the perception of the field externally and internally. It creates an outside image that our journals are inferior to others in the classic engineering disciplines and it hurt our effort to establish credibility among our peers in engineering departments and faculties. It also makes it harder to build on previous knowledge properly and avoid re-inventing the wheel.
Proper referencing in a paper is relatively easy to check. Do it before you submit a paper to make sure, it is you who find the issues and not your reviewers.
Last modified: 11/15/2012 8:15:49 AM