The PD-Net Approach to Supporting Ethics Compliance in Public Display Studies

Research involving public displays often faces the need to study the effects of a deployment in the wild. While many organizations have institutionalized processes for ensuring ethical compliance of such human subject experiments, these may fail to stimulate sufficient awareness for ethical issues among all project members. Some organizations even require such assessments only for medical research, leaving computer scientists without any incentive to consider and reflect on their study design and data collection practices. Faced with similar problems in the context of the EU-funded PD-Net project, we have implemented a step-by-step ethics process that aims at providing structured yet light-weight guidance to all project members, both stimulating the design of ethical user studies, as well as providing continuous documentation.

This web page describes our process and our 3 years of experience using it. All materials are publicly available and we encourage other projects in the area of public displays, and beyond, to adopt them to suit their particular needs and to eventually re-share their experiences and material through this website. If you adopt any of the documents presented here in your project we would appreciate if you reference our corresponding publication as follows:

Marc Langheinrich, Albrecht Schmidt, Nigel Davies, and Rui José (2013): A Practical Framework for Ethics – the PD-Net Approach to Supporting Ethics Compliance in Public Display Studies. In: Proceedings of the Second International Symposium on Pervasive Displays (Mountain View, CA, June 4-5, 2013). PerDis 2013. ACM, New York, NY. DOI=

Process Overview

From the ethical workflow point of view, the project is divided into 3 phases: the preparatory phase, the research phase, and the closing phase (cf. Figure 1).

1. During the preparatory phase , the project partners jointly form an “Ethics Advisory Board” (EAB). Advisory board members (typically one per partner) are ethics experts from each of the partners, e.g., members of the organization’s legal team or institutional review board, or the data protection officer. EAB members are asked to review a small number of process documents (see below) over the course of the project.
2. During the research phase , project members follow the ethics process described below for each human subject study that is planned. Initially, novel studies will require feedback from the EAB, but as the project progresses, EAB input will be less and less frequent.
3. In the closing phase of the project, i.e., after the project funding ends, data deletion commitments must be enforced and a final data processing report submitted to the EAB.

Preparatory Phase

During the preparatory phase, the project PIs prepare the background documentation, i.e., an introductory document (“ethics primer”), an “ethics worksheet”, as well as a number of “study process templates/SPTs” (see below) that they can readily identify as being applicable to the human subject studies foreseen in project. The initial documentation is helpful to recruit EAB members prior to the project start, and should thus be the very first step.

Documents

The documents linked below should provide a good starting point. They were created in the context of the EU-funded (FET-Open) project “PD-Net: Towards Future Pervasive Display Networks ”, so all reference to this would need to be replaced by your corresponding project name.

1. Ethics Handbook: Describes basic process and gives background on ethical issues in pervasive display research.
2. Study Process Templates (SPTs): For discussion among – and ultimately acceptance by – EAB members. Once the EAB has acknowledged these templates, subsequent studies do not need to be reviewed by the advisory board. There must be one template for each major kind of study done within the scope of the project.
   • Informed Consent
     
   • Secure Storage
     
   • Volunteer Studies
   • Public Trials
   • Interviews and Surveys
     
3. Ethics Worksheet : To be filled out before every individual human subject study performed within the project. It describes the planned study and links/refers to one or more of the above process templates, as these describe how personal data is collected, processed and stored in the particular experiment/study.
4. About (Project Background): For the use in participant handouts. Concise summary of project and contact information (online presence, phone, email, postal address).

Forming the Ethics Advisory Board (EAB)

The role of the EAB is to provide external ethical oversight. In places with an existing institutional review board (IRB), this looks like a redundant structure. However, in our experience, IRBs at many European Universities only focus on medical/psychological research, if they exist at all. Institutions with an IRB should thus recruit an EAB member from their existing IRB, while organizations without an IRB should seek a local expert in ethical issues, e.g., from the legal team, a legal/philosophical/theological faculty, or the data protection office (a mandatory post in many European organizations). Recruitment should point out the relatively low number of exchanges needed during the research phase – we found that no more than half a dozen explicit interactions were needed.

A sample invitation letter/email may look like this: Invitation Email

Note that no direct coordination among EAB members is needed (though this is possible and adds value to the members) – instead, each EAB member independently provides feedback to the planned studies.

The preparatory phase concludes with the start of the project. EAB member recruitment should thus start as soon as project funding has been secured, while the corresponding documents may already be prepared in time for the project application.

Research Phase

The research phase will see repeated application of the core study approval process, as described below. At the heart of this process are the “Study Process Templates (SPT)”. These templates describe a particular type of study (e.g., lab experiment, field observation), or a particular procedural or technical challenge (e.g., secure storage, informed consent). Each concrete study is mapped onto one or more of those templates, allowing researchers to quickly identify the challenges of a particular study type, as well as using the template to follow proper procedures. Ethical feedback from EAB members is per template, rather than per study. This not only lowers effort on behalf of the EAB members, but also streamlines procedural overhead of the entire process.

The full set of steps described below (i.e., 1-5) is thus only performed for novel studies for which no SPT exists yet. Once a particular study has been performed, subsequent similar studies across all member sites only require the very first step, i.e., filling out a detailed worksheet describing the actual instance of a study type. Similarly, the process is easily extendable and allows for cross-project reuse. The worksheets double as project documentation, process guidance, and as an educational tool. Figure 2 shows the core elements of the process, and how they interact during the preparation of a study.

The five steps are:

1. Fill out Ethical Worksheet prior to planned study
   1. Prepare Consent Form if needed (see “Informed Consent”)
   2. If needed, seek local approval from local Institutional Review Board (IRB) and regulatory bodies
      1. If IRB assessment required, prepare necessary documents and submit
      2. If regulatory approval required, prepare necessary documents and submit
      3. Incorporate any feedback, resubmit if necessary
   3. Identify type of research and consult set of appropriate “Study Process Templates (SPT)”
      1. If no SPT matches, create new SPT for this class of research and submit to project Ethical Advisory Board (EAB) prior to planned beginning of study
         1. Incorporate any feedback from EAB, resubmit to EAB if necessary
         2. Complete Ethical Worksheet with results from EAB, IRB, regulatory assessments
   4. If new IRB approval and/or EAB assessment, submit results to project Coordinator prior to planned begin of study.
   5. Proceed with the planned study only if all relevant SPTs have been approved by the EAB and all local IRB issues (if applicable) have been addressed. The corresponding Ethical Worksheet should be archived on the project wiki and/or website.