Article Text
PDF
Abstract
Study objective: To develop a cross disciplinary literature search methodology for conducting systematic reviews of all types of research investigating aspects of the built environment and the health of the public.
Design: The method was developed following a comprehensive search of literature in the area of housing and injuries, using 30 databases covering many disciplines including medicine, social science, architecture, science, engineering, environment, planning and psychology. The results of the database searches, including the type (or evidence) of research papers identified, were analysed to identify the most productive databases and improve the efficiency of the strategy. The revised strategy for literature searching was then applied to the area of neighbourhoods and mental health, and an analysis of the evidence type of references was carried out. In recognition of the large number and variety of observational studies, an expanded evidence type classification was developed for this purpose.
Main results: From an analysis of 722 citations obtained by a housing and injuries search, an overlap of only 9% was found between medical and social science databases and only 1% between medical and built environment databases. A preliminary evidence type classification of those citations that could be assessed (from information in the abstracts and titles) suggested that the majority of intervention studies on housing and injuries are likely to be found in the medical and social science databases. A number of relevant observational studies (10% of all research studies) would have been missed, however, by excluding built environment and grey literature databases. In an area lacking in interventional research (housing/neighbourhoods and mental health) as many as 25% of all research studies would have been missed by ignoring the built environment and grey literature.
Conclusions: When planning a systematic review of all types of evidence in a topic relating to the built environment and the health of the public, a range of bibliographical databases from various disciplines should be considered.
- systematic review
Statistics from Altmetric.com
There is a growing and conclusive body of work relating socioeconomic inequality to health.1–3 The Acheson report highlighted the increasing health problems being caused by the rapidly widening gap in living standards4 and the government's green paper, Our healthier nation, emphasised the need to tackle issues associated with the physical and social environment.5 Housing and the built environment are principal components of this inequality in industrialised countries6 and it is generally accepted that there is an association between the built environment and the health of the public.7,8 It is also clear that health is influenced by both physical conditions and features, such as cold7 or hazards in the home9–11 and, possibly, by psychological factors including the satisfaction of the dweller with their housing type and their sense of control over conditions.6,7 There is therefore some recognition that built environment and socioeconomic factors interact to produce effects on the health of the public.
Given the potential benefits of a multidisciplinary approach to research, and the implementation of findings, in this area12 one might expect a common integrated approach among built environment and public health professionals. However, from a search of the literature and discussion with professionals of both disciplines, it became clear that there was a lack of an agreed framework by which the various professions could systematically search for, and classify the available evidence.
For example, a study of 14 systematic reviews that examined aspects of housing and health 7,9,10,13–23 revealed that certain types of databases were not represented in the reviews. Reviews were generally directed from individual disciplines associated with specific diseases or conditions, and did not encompass the wider sources of information on the built environment. Of the 14 reviews, none searched the extensive environmental/built environment bibliographical databases, and only seven searched both medical and social science databases.7,9,10,17–20 The remaining seven reviews were based on medical databases alone.13–16,21–23 The effect of excluding built environment and, in some cases, social science databases from reviews in this area is, as far as we know, untested.
An agreed system for classifying the research or evidence type of studies relevant to this subject area is also lacking. Health care professionals are increasingly basing their practice on the systematic weighting of evidence from health care research using internationally agreed criteria.24 The development of a consensus of the hierarchy or “strength” of evidence has been an important part of this work. Most evidence hierarchies are derived from the Canadian Task Force on the Periodic Health Examination25 and are becoming increasingly sophisticated as they are adapted for differing purposes.26–28
The randomised controlled trial is, rightly, at the top of these hierarchies of evidence but there are considerable practical and ethical difficulties in undertaking these trials in the context of the environment and public health. While experimental studies should be carried out whenever feasible, well designed and controlled observational studies have been found to be as reliable as randomised controlled trials in many cases.29–31 High quality observational studies of interventions can cover a wider population than can be achieved by randomised controlled trials (for reasons of cost and organisation). They are likely to be dominant in the identification of potentially negative effects and particularly valuable when randomised controlled trials would be unethical or impractical,32 as is likely to be the case for many areas of the built environment and health. In the absence of controlled studies, surveys and case studies are also of value for generating hypotheses and informing research programmes. In recognition of the relative merits of different research methods, the Health Evidence Bulletins Wales Project has adapted the Bandolier hierarchy of evidence28 to refer to evidence “type” rather than strength33 (table 1). As there is only one category for observational studies (type IV), however, this system is of limited value for a subject area in which a large number of observational, but few experimental studies have been carried out.
The Health Evidence Bulletins Wales classification of evidence type
In response to these issues, a study was carried out with two main objectives:
(1) To expand the existing Health Evidence Bulletins Wales evidence type classification to account for the large number and variety of observational research designs;
(2) To examine the types of evidence found in a range of healthcare, social science, `grey' literature and built environment databases to assess the most useful databases for a cross disciplinary systematic review in the area of built environment and public health.
The overall objective was to develop a practical methodology for carrying out systematic reviews of all types of research examining the effect of the built environment on the health of the public.
METHOD
A list of databases and other information sources, relevant to the two disciplines, and available to the authors, was drawn up through discussion with the professionals involved (table 2). The databases covered many disciplines including medicine, social science, architecture, science, engineering, environment, planning and psychology.
Information sources used for the housing and injuries search
A proposed model for classifying the evidence was adapted from the Health Evidence Bulletins Wales type I to V classification system for evidence33 (table 1).
The category of observational studies was expanded to allow for the consideration of the substantial body of built environment literature, both quantitative and qualitative, and including cross sectional surveys and case studies, which provide the evidence base for building sciences and building regulations.
In the adapted model (the STOX system) the evidence types are divided into four distinct segments:
Systematic reviews.
Trials and other experimental studies.
Observational studies.
eXpressions of opinion.
The order reflects the potential power, within but not between, each category of evidence to support a causal relation (table 3).
The STOX classification system
Because the range of possible links between the built environment and health was so wide, and a full review was planned on this topic, the method was initially tested in the discrete area of housing and injuries. The search strategy is given in table 4.
The search strategy for the housing and injuries search
The relevance of publications was determined from the abstract (or title where no abstract was available) according to a list of inclusion/exclusion criteria for each discrete topic (table 5).
Inclusion/exclusion criteria for the housing and injuries search
The results of all of the databases used for the housing and injuries search were analysed to identify overlap and efficiency using a reference management database. The aim was to determine the most valuable databases for interdisciplinary reviews while accounting for the type of evidence found within them. In order to select a group of citations that could be classified according to the STOX criteria from information contained in the abstracts and/or titles, these results were narrowed down using the search terms in table 6.
Search terms to identify potentially classifiable references in the housing and injuries search
Where possible, these references were given a preliminary classification according to the STOX system (table 3). As information available from abstracts was limited, an O3 classification was given where the abstract mentioned quantitative data relating to groups of people (for example, statistics or the use of a recognised measurement tool like the SF-36). Category X was assigned for literature reviews/overviews that may have been systematic reviews but where there was no sign of a systematic approach from the information provided in the abstract. Models based on reviews of the literature were also given this classification. Attempts were made to classify references without abstracts by their title. The types of evidence found in each database were then analysed.
Given that housing and injury was only one of many relevant issues, it was decided to conduct a further assessment of the range of research found in each database for another area, neighbourhoods and mental health. A set of papers from a preliminary literature search of evaluation studies was analysed for this subject area, which was characterised by a dearth of experimental studies. The search strategies, databases searched and inclusion/exclusion criteria for the mental health searches are given in tables 7 and 8.
The search strategy for the housing/neighbourhoods and mental health search
Inclusion/exclusion criteria for the housing/neighbourhoods and mental health search
RESULTS
Housing and injuries
Using the search strategy detailed in table 4 and the databases listed in table 2, 36 094 references were identified as potentially relevant to the area of housing and injuries. Titles and abstracts, where available were examined and 722 (2%) of the papers identified were found to be relevant to the housing and injuries enquiry using the inclusion/exclusion criteria (table 5). Only 19 of the databases contained relevant references and 10 of these databases together retrieved 685 (95%) of the 722 relevant references (table 9). Thirty seven publications (the remaining 5%) were only found in the other nine databases (Cochrane Library, INSPEC, General Science Index, Sociofile, UKOP, ASSIA, Environmental Abstracts, RCN-Nurse ROM and PsycLIT).
Databases where 95% of relevant housing and injury publications were identified
Overlap between databases was analysed and very little was found between architectural/environmental and medical databases. More overlap was found between medical and social science databases. For example, 62 of 722 papers were found in both a medical and a social science database (9%) compared with just seven papers being found in a medical and architectural/environmental database (1%). Furthermore, 95% of references in ICONDA (a construction database), 84% of references in Urbadisc (a planning database) and 77% in APID (an architectural database) were not found in any other database. Twelve per cent of all papers would have been missed by excluding built environment databases.
Key points
-
Systematic reviews in the cross disciplinary area of the built environment and health of the public tend to rely solely on the medical or medical/social sciences literature.
-
Where intervention studies exist in a subject area (for example, housing and injuries), the majority are reported in the medical/social science literature, therefore a number of databases from these disciplines should be consulted to maximise the sensitivity of the search.
-
Where there are few if any intervention studies (for example, housing and mental health) a range of research types (qualitative and quantitative) may be of relevance to illuminate the complex relation between environment and health, and guide further research.
-
Where all types of research evidence are considered in a review, the built environment and grey literature offer a significant contribution.
The evidence type analysis from the sample of 352 housing and injury references (selected as potentially classifiable) showed that most research studies are cross sectional in design (tables 10 and 12). Almost all interventional studies and 91% of all research studies (including those with an observational design) were found in the medical and social science databases (table 11). Embase was the most valuable database (retrieving 55% of putative research studies) while the addition of Medline increased this number to 77% of the total (table 11). Although the Social Science Citation Index added only a further 7% of the research studies, three intervention studies and one systematic review would have been missed if this database had been excluded (table 12). In this subject area approximately 5% of all research references were found in the built environment and grey literature databases and were mainly observation based studies (tables 11 and 12). While there is clearly a contribution from the built environment and grey literature databases, table 12 highlights the importance of medical and social science databases for identifying a range of references that are not duplicated in any other database.
Evidence type contained within each database, from a sample of 356 housing and injury references that could be classified using STOX from their titles and abstracts
Cumulative effect of including additional databases on the number of publications retrieved from a sample of 356 housing and injury references that could be classified using STOX from their titles and abstracts
Evidence type contained within each database not duplicated in any other, from a sample of 356 housing and injury references that could be classified using STOX from their titles and abstracts
Housing/neighbourhoods and mental health
From an original list of around 1600 citations retrieved by the search strategy (table 7), the abstracts (or titles) of 289 papers met the inclusion/exclusion criteria (table 8) and included terms that suggested they could be classified by the STOX categorisation (table 6). These were then classified according to the STOX criteria. Tables 13, 14 and 15 show the types of evidence found in each database.
Evidence type contained within each database, from a sample of 289 housing/neighbourhoods and mental health references that could be classified using STOX from their titles and abstracts
Cumulative effect of including additional databases on the number of publications retrieved from a sample of 289 housing/neighbourhoods and mental health references that could be classified using STOX from their titles and abstracts
Evidence type contained within each database, not duplicated in any other, from a sample of 289 housing/neighbourhoods and mental health references that could be classified using STOX from their titles and abstracts
The majority of research studies were cross sectional in design, as in the housing and injury search, but there were also a substantial number of qualitative and case studies (table 13). The literature search strategy was designed to look for evaluation/assessment studies and would favour observational designs. However, it is clear from this and other literature searches that experimental studies examining this complex issue are lacking. In contrast with the housing and injuries search, less than 7% of all research studies were retrieved from Embase and the four medical databases accounted for only 32% of the total (table 15). A substantial contribution was made by a specialist database (PsycINFO) and the Social Science Citation Index. In addition, a large research contribution (29% of the total) came from the built environment and grey literature although much of this may have been of a qualitative or case study nature (table 14 and 15).
DISCUSSION
The analysis of publications in two subject areas has confirmed the relative paucity of experimental, and high quality observational research, in the arena of built environment and health. It has also shown the value of a broad based approach, including databases from a variety of disciplines in a comprehensive systematic review covering all types of research. Sole reliance on medical databases is likely to exclude a significant number of relevant research studies.
The extension of the evidence type classification proposed in the STOX scheme allows for an expanded class of observational studies to include a range of research types from the more powerful cohort and case-control designs through to cross sectional surveys and case reports. A range of epidemiological, behavioural and social research can be used to illuminate important issues. This research (both quantitative and qualitative) can then be synthesised to identify the scope and feasibility of a potentially successful intervention where ethical.
The preliminary classification method for evidence type used in this study (based on the use of abstract/title information only) will have missed some important papers. The key words chosen to select potentially classifiable research studies, are well used in the medical but less so in the built environment professions. There is also a move in medical journals towards more structured abstracts containing information on the research design. Information is particularly limited in the built environment journals, particularly in APID where titles only are available. These factors may well have resulted in a preferential selection of publications from the medical literature. Despite this, the results provide an indication of the types of evidence found in different databases. It is clear that there is a limited overlap between databases from different disciplines, and that these provide a range of research types.
On the basis of these results, it is recommended that anyone undertaking a comprehensive systematic review of intervention studies in this subject area should consider, as a minimum, medical and social science databases. The major medical databases (Medline and Embase) should both be included. The Social Science Citation Index contained a large number of relevant papers, not found in other databases, particularly in the area of neighbourhoods/housing and mental health. Although ASSIA and Sociological Abstracts were of limited value for the housing and injuries search, it is quite possible that they will be relevant in other subject areas. We propose to examine the relative contributions of these social science databases in more detail. For a subject area in which little if any experimental research is available, the built environment and grey literature databases include a number of cross sectional surveys and case studies with quantitative and qualitative research results. This research can provide insights into complex issues like satisfaction and quality of life. An overview of these findings can then inform the development of hypotheses to be tested in further controlled observational research, and in interventional research where ethical and practical.
If a subject area is covered by a specialist database (for example, PsycINFO) this should also be included. For the housing and injuries search only seven relevant references were found in the Cochrane Library so it was excluded from the core list. Four of these, however, were not found in other databases. The Cochrane Library database is increasing rapidly in size and moving beyond its traditional work on reviews of therapy only. It is thus likely that the Cochrane Collaboration and its affiliated organisation the Campbell Collaboration (with a remit for reviews of social and educational policy and practice)35 will produce reviews of increasing relevance to environmental influences on health.
A further finding from our experience of searching for systematic reviews in this area, is the value of personal contact to find relevant evidence-based books, reports and research reviews that would not be readily located via database searching. Two particularly useful recent reviews of the housing and health literature (including one systematic review) were located via this route.7,8
On the basis of these findings, a preliminary core list of databases might be considered for a scoping literature search (using a few relevant key words), before carrying out a systematic review (table 16). Taking into account the results of this preliminary search, and the types of evidence that the researcher proposes to include, this list could then be amended and finalised for the review.
Core list of recommended information sources for a scoping search, in preparation for a cross-disciplinary systematic review on housing/neighbourhoods and health
In conclusion, the review of systematic reviews showed that current relevant systematic reviews on public health and the built environment do not usually search built environment databases, and only half search beyond medical ones. The detailed review of housing and injury, where medical, social science and built environment databases have been searched, showed little overlap in papers identified through built environment and public health searches. Furthermore, the analysis of evidence types found within databases for the search areas of injury and mental health showed that research of a variety of evidence types is being missed by using databases from single disciplines alone. While the majority of intervention studies may currently be retrieved by the medical and social science databases, a large number of observational studies are available in the built environment and grey literature. A broad-based approach, which considers a large range of evidence types, could be of value in a complex area like public health.36 Our methodology is designed to search for and classify all types of evidence via a cross disciplinary approach.
The framework that has been developed for the assessment of evidence of the environmental influences on the health of the public draws heavily on the rigorous methods of the Evidence Based Health Care movement. The cross disciplinary methodology is easy to transport into areas like social and mental health, healthy environments and healthy living and can be used to assist in the formulation of evidence-based social and environmental policies. It is hoped that this study will stimulate debate among public health, social science and built environment professionals as to the types of research to carry out and use, to inform practice in this area. The methodology will be further tested and developed with a view to carrying out detailed systematic reviews on injury, mental health and indoor respiratory health for a recently funded study, the Housing and Neighbourhoods and Health (HANAH) Project.37
Acknowledgments
Conflicts of interest: none.
Funding: none.