Abstract
Successful medical device design necessitates an understanding of stakeholder-driven requirements early in a design process to assure device safety and usability, and support successful and positive patient experiences. Prototypes can be used during stakeholder engagement in the design front end to gather the information that will inform design decisions. However, an understanding of medical device industry practices for front-end stakeholder engagement with prototypes is lacking. Through interviews with medical device design practitioners, this study explored the variety of stakeholder groups engaged by design practitioners, prototype types used during stakeholder engagements, and settings in which engagements occurred during front-end design activities. This study describes the 14 types of stakeholders, 14 types of prototypes, and six types of settings described by practitioners as well as patterns across engagement strategies, stakeholders, prototypes, and/or settings during front-end activities. These outcomes can contribute to broadening designers' stakeholder engagement planning and practices.
1 Introduction and Background
Medical devices are part of the large array of health technologies that help increase access to healthcare [1]. A medical device is an instrument “intended for use in the diagnosis […], cure, mitigation, treatment, or prevention of disease […] and which does not achieve its primary intended purposes through chemical action” [2]. Throughout a design process, medical device designers often engage and seek feedback from diverse stakeholders that are involved in the commercialization and use of devices. Stakeholders include healthcare practitioners, patients, professional and advocacy groups, government officials and legislators, payers [3], risk managers, clinical engineers, maintenance personnel, trainers, and supervisors [4,5]. The beneficiaries—users, payers, and purchasers of medical devices—are often different people [6], potentially leading to conflicting needs [7]. Furthermore, medical devices are subject to a strict regulatory environment that mandates the use of prototypes to test concepts with users [8] during usability testing and fully functional devices during clinical trials [9]. Therefore, diverse stakeholder engagement is an inherent part of medical device design.
1.1 Stakeholder Engagement During Medical Device Design.
Engaging a broad range of stakeholders throughout a medical device design process leads to more successful designs; it is particularly critical for designers to successfully engage stakeholders during the front end of design [10,11], which includes problem and needs finding, identification and definition of design opportunities, articulation of requirements and specifications, and idea generation and development [12]. Stakeholder engagement provides design practitioners with insights into the design context and the values and behaviors of stakeholders [10] and leads to the elicitation of latent priorities [13]. However, barriers exist to stakeholder engagement, such as the intense resources needed to engage medical device users, the limited availability of certain medical professionals and patient populations, and communication gaps between design practitioners and stakeholders [10,11].
1.2 Benefits of Prototype-Based Stakeholder Engagement.
Prototypes have been promoted as tools for engaging stakeholders during design processes [3,14]—to elicit knowledge, needs, and requirements [15,16]. Prototypes are physical or virtual objects that can have many forms, including sketches, digital models, and physical three-dimensional (3D) objects. Prototypes represent design ideas for the end-product as well as subcomponents of the potential end-product, processes for engaging with the product, and experiences with the product [17]. For example, storyboards can be used to represent a user's process of interaction with a medical device interface [5], while virtual reality can be used to simulate a procedure involving a novel medical device [18].
Prototypes provide various ways for stakeholders to participate actively in design activities [19,20], including when stakeholders have trouble articulating knowledge relevant to the design [21]. Prototype-based engagements facilitate designers' abilities to elicit stakeholders' input throughout the various stages of a design process [22] by centering conversations on perceptions of and interactions with the prototypes [14]. Prototypes can support various designer-stakeholder activities, such as communicating a design concept [22], gathering feedback on a design concept, having stakeholders interact with a prototype [23], cocreating with stakeholders [24], helping to establish a common language between designer and stakeholder, exploring the problem space, and eliciting requirements from stakeholders [11]. Lauff et al. [25] described prototypes as intentional tools that facilitate communication. Among the limited studies that have explored the effects of using specific prototype forms with specific stakeholder groups, several studies have found that the prototype form used during user feedback sessions and usability testing affects the feedback received from stakeholders and the results of usability activities [26–28]. Thus, the choices of prototypes to engage various stakeholder groups can influence the outcomes of the engagement.
1.3 Current Use of Prototypes in Medical Device Design.
Prototypes in medical device design have traditionally been leveraged to explore the technical feasibility of a project, to improve a device's functionality and performance [29], and in later design stages, to verify specifications are achieved and validate the fulfillment of clinical needs [8,30]. Some evidence suggests that medical device design practitioners tend to use late-stage prototypes when seeking stakeholder feedback, therefore obtaining user information only during the later stages of a design process [31]. Stakeholder engagement practices are often defined in the context of usability studies meant to identify, quantify, and mitigate use errors [9,13]. Therefore, prototyping for medical device design is often seen as a phase that comes later in a design process [5] rather than as a tool that can also be leveraged at the onset. While in other fields, prototypes are prominently described as being used in front-end activities (e.g., human-computer interaction, where sketches are widely used to mockup interfaces [32], and codesign, where probes are used to explore the problem space [16]), there are limited publications that describe front-end prototyping with stakeholders in the medical device design field.
Human factors, the field within which usability testing emerged, does emphasize the importance of early involvement of users in medical device design, particularly through observations, interviews, and focus groups [5]. Human factors and ergonomics research have shown that the integration of user-specific requirements early in the design processes of medical devices leads to improved safety and usability of devices, improves patient outcomes and satisfaction, and reduces device recalls and the need for modifications later in design processes [13]. Human factors engineering has established methods for early user engagement, consisting of user testing with both early nonfunctional prototypes and downstream functional prototypes, to identify user-device interaction issues as early as possible [5]. However, human factors research focuses on the study of user-interface interaction. Aside from user-interface interaction, the use of prototypes to engage a wider variety of stakeholders during the earliest phases of design—such as for need identification, problem definition, requirements elicitation, and idea generation—is underexplored within the medical device design field.
1.4 Medical Device Design for Low- and Middle-Income Countries.
In general, medical device designers work within strict regulatory environments and navigate changing healthcare reimbursement policies that create barriers to timely and successful commercialization [30]. In addition to these challenges, medical device designers working on solutions for use in low- and middle-income countries (LMICs) face a wide-ranging set of constraints [33–37], including the lack of pathways to commercialization of medical devices; lack of funding; low-profit margins; varied regulatory and intellectual property protection pathways; supply chains deficiencies; lack of supporting infrastructure; harsh use conditions; unique local norms and preferences; maintainability challenges; and other constraints. Many of these challenges are specific to LMIC settings and are seldom at the forefront of design methods for high-income country (HIC) settings. Several authors have reported that medical device designers from HIC contexts engage a broader set of stakeholders more frequently during the early stages of medical device design activities aimed at creating solutions for use in LMIC contexts [38–40], where various constraints and contextual factors may differ considerably from HIC contexts [41]. One early stakeholder engagement activity is to use prototypes, for example, as collaboration tools in codesign approaches, as exemplified in Caldwell et al. [38]. Practitioners who design medical devices for use in LMICs can offer unique insights into early prototyping behaviors for stakeholder engagement.
1.5 Research Focus.
Through interviews with medical device design practitioners working in industry, we investigated the variety of stakeholder groups engaged by design practitioners, the prototypes they used during stakeholder engagements, and the settings in which the engagements occurred during front-end design activities, which included problem identification and needs finding, problem definition, background research, concept generation, early prototyping, and concept selection. We further investigated front-end design patterns across stakeholders, prototypes, and settings. In this study, we leveraged a broad definition of prototypes to include representations of processes (e.g., a clinical procedure), systems, or subparts of a designed product or its use context. Prototype examples included mockups, computer-aided design (CAD) models, drawings, scenarios, and existing products used as prototypes. What distinguished a prototype from an artifact was the intentional way the artifact was used by the designer as a prototype. This study contributes to advancing understanding of stakeholder engagement practices, ultimately supporting the improvement of front-end design activities and design decision making for prototype-based stakeholder engagement, including specific context-related decisions.
2 Methods
2.1 Research Aims.
The following research question guided the study: During front-end medical device design activities, what stakeholders are engaged with what prototypes, and in what settings?
2.2 Participants.
Potential participants were identified through existing contacts, networking at medical device conferences, and online searches. Potential participants were then emailed to determine their interest in participating in the study. Interested participants completed a background questionnaire detailing their prior medical device design experiences, their experiences using prototypes to engage stakeholders during front-end design, as well as their years of industry experience with mechanical or electromechanical medical device design (one or more years of experience required). This approach to recruitment led to the identification of key informants with the expertise and knowledge we aimed to elicit in this study. Participants joined the study voluntarily, provided informed consent, and received US$75 for their participation.
Twenty-two participants were interviewed from sixteen medical device companies. In order to identify practices across different companies working in diverse design contexts on a variety of medical device types, we sought to obtain a balance among participants from multinational companies and companies working in global health settings (in LMICs), as well as among participants from companies that ranged in size. All but one company was headquartered in an HIC. Participant information is provided in Table 1 (individual level) and Table 2 (aggregate level).
Participant code | Product type discussed in the interview | GH/MN | Company size |
---|---|---|---|
A | Treatment (infusion) | GH | Small |
B | Treatment (infusion) | GH | Small |
C | Diagnostics (hypothermia) | GH | Medium |
D | Treatment (phototherapy); diagnostics | GH | Small |
E | Equipment (vaccines) | GH | Medium |
F | Treatment (blood transfusion) | GH | Small |
G | Treatment (infusion) | GH | Large |
H | Treatment (hypothermia) | GH | Small |
I | Training (maternal health) | GH | Medium |
K | Training (maternal health) | GH | Medium |
N | Treatment (intubation) | MN | Small |
O | Treatment (surgical equipment) | MN | Large |
P | Unknown | MN | Large |
Q | Diagnostics (imaging) | MN | Large |
R | Treatment (surgical equipment) | MN | Large |
S | Diagnostics (imaging) | MN | Large |
T | Treatment (catheterization) | MN | Large |
U | Treatment (catheterization) | MN | Large |
V | Unknown | MN | Medium |
W | Treatment (prosthetics) | MN | Medium |
X | Treatment (catheterization) | MN | Small |
Y | Unknown | MN | Medium |
Participant code | Product type discussed in the interview | GH/MN | Company size |
---|---|---|---|
A | Treatment (infusion) | GH | Small |
B | Treatment (infusion) | GH | Small |
C | Diagnostics (hypothermia) | GH | Medium |
D | Treatment (phototherapy); diagnostics | GH | Small |
E | Equipment (vaccines) | GH | Medium |
F | Treatment (blood transfusion) | GH | Small |
G | Treatment (infusion) | GH | Large |
H | Treatment (hypothermia) | GH | Small |
I | Training (maternal health) | GH | Medium |
K | Training (maternal health) | GH | Medium |
N | Treatment (intubation) | MN | Small |
O | Treatment (surgical equipment) | MN | Large |
P | Unknown | MN | Large |
Q | Diagnostics (imaging) | MN | Large |
R | Treatment (surgical equipment) | MN | Large |
S | Diagnostics (imaging) | MN | Large |
T | Treatment (catheterization) | MN | Large |
U | Treatment (catheterization) | MN | Large |
V | Unknown | MN | Medium |
W | Treatment (prosthetics) | MN | Medium |
X | Treatment (catheterization) | MN | Small |
Y | Unknown | MN | Medium |
GH: global health focus; MN: multinational focus; small: 1–10 employees; medium: 10–200 employees; large: over 1000 employees. Participants with an unknown product type did not provide any specific details about a medical device for confidentiality reasons.
Category | Company headquarters | Company type | |||||
---|---|---|---|---|---|---|---|
USA | India | Norway | Sole proprietorship | Public FP | Partnership | Nonprofit | |
Number of companies | 14 | 1 | 1 | 1 | 13 | 1 | 1 |
Category | Age (years) | ||||||
Under 30 | 30–40 | Over 40 | Unknown | ||||
Number of participants | 6 | 9 | 6 | 1 | |||
Category | Job tenure (years) | ||||||
2 years or less | Between 2 and 5 years | More than 5 years | |||||
Number of participants | 5 | 6 | 11 | ||||
Category | Highest degree | ||||||
Bachelor’s | Master’s | Ph.D. | |||||
Number of participants | 7 | 13 | 2 | ||||
Category | Gender | ||||||
Women | Men | ||||||
Number of participants | 9 | 13 |
Category | Company headquarters | Company type | |||||
---|---|---|---|---|---|---|---|
USA | India | Norway | Sole proprietorship | Public FP | Partnership | Nonprofit | |
Number of companies | 14 | 1 | 1 | 1 | 13 | 1 | 1 |
Category | Age (years) | ||||||
Under 30 | 30–40 | Over 40 | Unknown | ||||
Number of participants | 6 | 9 | 6 | 1 | |||
Category | Job tenure (years) | ||||||
2 years or less | Between 2 and 5 years | More than 5 years | |||||
Number of participants | 5 | 6 | 11 | ||||
Category | Highest degree | ||||||
Bachelor’s | Master’s | Ph.D. | |||||
Number of participants | 7 | 13 | 2 | ||||
Category | Gender | ||||||
Women | Men | ||||||
Number of participants | 9 | 13 |
2.3 Data Collection.
Semi-structured interviews were conducted in person with five participants and via videocall with 17 participants. A semi-structured interview approach ensured that a standard set of questions were asked while allowing flexibility to pursue tailored follow-up questions [42]. The interviews lasted 87 minutes on average and ranged from 55 to 152 minutes in length.
The interview protocol was developed following recommended practices for interview development, including beginning the interview with descriptive questions, grounding open-ended questions in the relevant literature and aligning with the research question, and including follow-up questions to gain additional detail [43]. The protocol was revised iteratively as the result of 11 pilot interviews (that were not part of this study) conducted with designers who had industry experience.
The definitions of “front end,” “prototype,” and “stakeholder” were read aloud to the participants at the beginning of the interview to establish a shared language between the interviewer and participants. The definitions of the front end, prototype, stakeholder, and setting are provided in Appendix A. The interviewer then asked participants to focus on a single prior project and describe instances when they engaged stakeholders with prototypes during front-end design activities. Participants were asked about how they engaged stakeholders using prototypes, which stakeholders were engaged, what prototypes were leveraged, and the settings of the engagements. At the end of the interview, participants were asked to compare their experiences of stakeholder engagement with prototypes across projects. Sample interview questions are included in Appendix B. The study was determined to be exempt and was approved by the University of Michigan Institutional Review Board (HUM00137476).
2.4 Data Analysis.
Engagement events served as the unit of analysis for associations among strategies, stakeholders, prototypes, and settings leveraged by practitioners during front-end design activities. We defined an engagement event, based on guidance from Montgomery and Duck's work [44], as a front-end activity where one or more prototyping strategy(ies) was/were used to engage one or more stakeholder(s) with one or more prototype(s) in a particular setting. All instances of engagement events were described using the participants' descriptions of prototyping strategies, stakeholders, prototypes, and settings. Excerpts from a single engagement event could be contiguous or scattered throughout the transcript. An example engagement event is provided in Appendix C.
Two researchers first jointly identified engagement events in one transcript. This process established coding reliability and allowed the researchers to resolve discrepancies through discussion. Then, each researcher read 11 transcripts and identified and described engagement events. Finally, one of the researchers reviewed all engagement events to verify consistency across the dataset. An average of six engagement events per transcript were identified, for a total of 127 engagement events (between one and 11 engagement events per transcript).
After the engagement events were identified, transcripts were coded using two different coding schemes. The first coding scheme identified types of stakeholders, prototypes, and settings using an inductive analysis approach [45], where patterns were recognized across the data through continuous comparison to articulated patterns. Discrepancies in coding were resolved through discussion across two coders. Next, the codes were refined following Urquhart's [45] recommendations for qualitative coding, in this case by using existing classifications of prototype forms [16,46–50] and stakeholder groups [3,4,13,51–57].
The second coding scheme used an existing prototyping strategy codebook developed as part of prior work involving the same dataset [58]; the codebook comprised 17 prototyping strategies used to engage stakeholders during front-end medical device design activities (shown in Table 3).
Strategy | Label |
---|---|
Brief the stakeholder about the project and the prototype(s) shown | Brief |
Encourage the stakeholder to envision use cases while interacting with the prototype(s) | Envision |
Have the stakeholder interact with the prototype(s) in a simulated use case | Simulate |
Introduce the prototype(s) to the stakeholder in the actual use environment | Introduce |
Lessen a prototype's completeness when showing it to the stakeholder | Lessen completeness |
Make prototype extremes to show the stakeholder | Extremes |
Modify the prototype(s) in real-time while engaging the stakeholder | Modify |
Observe the stakeholder interacting with the prototype(s) | Observe |
Polish the prototype(s) shown to the stakeholder | Polish |
Present a deliberate subset of prototypes to the stakeholder | Subset |
Prompt the stakeholder to select prototypes and prototype features | Select |
Reveal only relevant information to the stakeholder specific to the prototype or its use | Reveal |
Show a single prototype to the stakeholder | Single |
Show the stakeholder multiple prototypes concurrently | Multiple |
Standardize the refinement of prototypes shown concurrently to the stakeholder | Standardize |
Supplement a prototype shown to the stakeholder with different prototype types | Supplement |
Task the stakeholder with creating or changing the prototype(s) | Create |
Strategy | Label |
---|---|
Brief the stakeholder about the project and the prototype(s) shown | Brief |
Encourage the stakeholder to envision use cases while interacting with the prototype(s) | Envision |
Have the stakeholder interact with the prototype(s) in a simulated use case | Simulate |
Introduce the prototype(s) to the stakeholder in the actual use environment | Introduce |
Lessen a prototype's completeness when showing it to the stakeholder | Lessen completeness |
Make prototype extremes to show the stakeholder | Extremes |
Modify the prototype(s) in real-time while engaging the stakeholder | Modify |
Observe the stakeholder interacting with the prototype(s) | Observe |
Polish the prototype(s) shown to the stakeholder | Polish |
Present a deliberate subset of prototypes to the stakeholder | Subset |
Prompt the stakeholder to select prototypes and prototype features | Select |
Reveal only relevant information to the stakeholder specific to the prototype or its use | Reveal |
Show a single prototype to the stakeholder | Single |
Show the stakeholder multiple prototypes concurrently | Multiple |
Standardize the refinement of prototypes shown concurrently to the stakeholder | Standardize |
Supplement a prototype shown to the stakeholder with different prototype types | Supplement |
Task the stakeholder with creating or changing the prototype(s) | Create |
To analyze the engagement events, the authors counted the number of times a specific association of strategy, stakeholder, prototype, and/or setting occurred. Therefore, the engagement events revealed trends of associations among strategies, stakeholders, prototypes, and settings and examples of such associations directly taken from designers' project experiences. Because of the discrepancy in the number of engagement events per transcript, the choice was made to keep the counts of associations at the transcript level rather than at the engagement level, so as not to increase the impact of transcripts with larger numbers of engagement events.
3 Findings
3.1 Stakeholder Groups, Prototype Forms, and Engagement Settings of Front-End Prototype-Based Stakeholder Engagement.
Across all prototyping strategies, participants engaged a wide range of stakeholders. These stakeholders were categorized into three groups: (1) users, (2) expert advisors, and (3) implementation stakeholders. Users included active users, passive users, proxy users, and secondary-usage stakeholders. Broadly, participants described active users and proxy users as stakeholders who provided information on the clinical need being fulfilled and on the device design. The next main category of stakeholders—expert advisors—included people with clinical, product, and other knowledge who provided expertise based on their professional experience. Implementation stakeholders, including stakeholders such as manufacturing, marketing, and supply chain stakeholders, provided information on nonclinical aspects of the device. Definitions and examples of each stakeholder group extracted from the interviews are included in Table 4. Interview excerpts are provided in the table, below the definition and examples for each group.
Stakeholder group | Definition | Example(s) within medical device context |
---|---|---|
Implementation stakeholder | Is directly involved in the adoption of the device and influences the success of the device | |
Supply chain stakeholder | Influences the device supply chain; can be an intended actor of the device supply chain | Distributors, integration engineers, suppliers and vendors, quality verification stakeholders |
[We engage] the supply chain people who tell you what kind of [parts] are available. (P) | ||
Community partner | Collaborates with the design team through a community organization partnership | Nongovernmental organizations, abroad offices and organizations, partner universities |
Before going to [a sub-Saharan African country] I emailed several partners who work in family planning and I said, “Listen I'm interested in visiting.” (K) | ||
Manufacturing stakeholder | Provides manufacturing expertise and insights into implementation constraints; can be the intended device manufacturer | Manufacturing stakeholders internal to the company, external manufacturers engaged as individuals or as company representatives |
When we are in the early phases of design and we are still in the concept generation of the product itself, we do include manufacturing there, because we want to make sure that if we design something that the floor cannot currently produce, they tell us. (Q) | ||
Financial decision maker | Contributes money, materials, or goods to the project; are engaged when raising funds or reporting progress | Internal board members, company leadership during a design review, external granters, project managers, donors |
During the concept phase, to go through each phase […] you need to go in front of a [board] and present what you have been doing during these different phases. (P) | ||
Government stakeholder | Works in government agencies affecting the device implementation in a country | Ministry of health officials who purchase medical devices, members of regulatory bodies (e.g., FDA) |
There were a few doctors from the government that we reached out to in the early stages of collecting feedback on the idea. We were […] showing them concepts on paper. (C) | ||
Regulatory stakeholder | Provides expertise on the laws and regulations that govern medical devices | Research councils, regulatory experts employed by the company or a hospital to provide regulatory guidance on the device |
If we were to discuss regulatory risks with our consultants, what we would do, we would show them […] a very detailed description of what the product would do. (F) | ||
Marketing stakeholder | Provides expertise on the market landscape, often working in a marketing or sales role | Stakeholders knowledgeable about the medical device market, stakeholders interfacing with users and customers to conduct market research |
Then you have marketing people coming in to say okay here is the market landscape and this is the trend. What are the popular [products] and here's what people don't like about certain types of things. […] They want to see the [prototype] as it is. (P) | ||
Customer | Purchases the device but is not the intended user or distributor | Hospital purchasing departments, hospital department heads |
Once you have something functional, that was when we started sending stuff to investors and to our customers, [to get] evaluated. (H) | ||
User influencer | Influences the use of the device by the active user | A mother's family whose beliefs impacted what devices could be used on an infant |
[What] was very important was the response of the others in the family. We realized that […] when you put something on a baby, it is not totally the mother's decision. (C) | ||
User | Uses the device and/or benefits from its primary function once the device is commercialized | |
Active user | Operates the device’s primary function; also called “primary user” | Patients who actively use medical devices, healthcare workers (e.g., doctors, nurses), caregivers, and medical trainers and students |
I ran a couple of focus groups with local nurses, based on ideas that our engineers had for upcoming products to see […] what needs the nurses had that weren't being fulfilled. (B) | ||
Passive user | Is impacted by the outcome of the device but has little to no control over the use of it; also referred to as “incidental user” | Patients on whom a procedure was performed with a medical device, (e.g., infants, children, adult patients, and prosthetic users) |
When you are actually putting the prototype on the baby, the baby is not still. (C) | ||
Proxy user | Shares similarities with the active user but is not an intended user of the device; is leveraged when active users are not accessible | Healthcare practitioners who work in a setting that differs from that of the intended users, laypeople (e.g., friends, co-workers), or the designers themselves |
I got to the point where I said: “Who has the largest hands here? Who has the smallest hands here?” […] I'd go around and try [3D printed models] in different people's hands. (R) | ||
Secondary-usage stakeholder | Interacts with the device outside of its primary function, throughout the product use-phase; also called “secondary user” | Technician, immunization manager, maintenance stakeholder involved in service and upkeep of the device (e.g., installation, charging, sterilization) |
We would get [the prototype] out in the hands of some service engineers and we would say, “install this and align this tube […] and tell us what is weird about it.” (S) | ||
Expert advisor | Provides expertise on the device design and usage, and the problem space based on their professional knowledge and experience | Clinical experts, product experts, other medical device company employees, academics, professors, members of partnering organizations |
We can invite people with a special competence within materials or digital solutions that we don't have in our team. (I) |
Stakeholder group | Definition | Example(s) within medical device context |
---|---|---|
Implementation stakeholder | Is directly involved in the adoption of the device and influences the success of the device | |
Supply chain stakeholder | Influences the device supply chain; can be an intended actor of the device supply chain | Distributors, integration engineers, suppliers and vendors, quality verification stakeholders |
[We engage] the supply chain people who tell you what kind of [parts] are available. (P) | ||
Community partner | Collaborates with the design team through a community organization partnership | Nongovernmental organizations, abroad offices and organizations, partner universities |
Before going to [a sub-Saharan African country] I emailed several partners who work in family planning and I said, “Listen I'm interested in visiting.” (K) | ||
Manufacturing stakeholder | Provides manufacturing expertise and insights into implementation constraints; can be the intended device manufacturer | Manufacturing stakeholders internal to the company, external manufacturers engaged as individuals or as company representatives |
When we are in the early phases of design and we are still in the concept generation of the product itself, we do include manufacturing there, because we want to make sure that if we design something that the floor cannot currently produce, they tell us. (Q) | ||
Financial decision maker | Contributes money, materials, or goods to the project; are engaged when raising funds or reporting progress | Internal board members, company leadership during a design review, external granters, project managers, donors |
During the concept phase, to go through each phase […] you need to go in front of a [board] and present what you have been doing during these different phases. (P) | ||
Government stakeholder | Works in government agencies affecting the device implementation in a country | Ministry of health officials who purchase medical devices, members of regulatory bodies (e.g., FDA) |
There were a few doctors from the government that we reached out to in the early stages of collecting feedback on the idea. We were […] showing them concepts on paper. (C) | ||
Regulatory stakeholder | Provides expertise on the laws and regulations that govern medical devices | Research councils, regulatory experts employed by the company or a hospital to provide regulatory guidance on the device |
If we were to discuss regulatory risks with our consultants, what we would do, we would show them […] a very detailed description of what the product would do. (F) | ||
Marketing stakeholder | Provides expertise on the market landscape, often working in a marketing or sales role | Stakeholders knowledgeable about the medical device market, stakeholders interfacing with users and customers to conduct market research |
Then you have marketing people coming in to say okay here is the market landscape and this is the trend. What are the popular [products] and here's what people don't like about certain types of things. […] They want to see the [prototype] as it is. (P) | ||
Customer | Purchases the device but is not the intended user or distributor | Hospital purchasing departments, hospital department heads |
Once you have something functional, that was when we started sending stuff to investors and to our customers, [to get] evaluated. (H) | ||
User influencer | Influences the use of the device by the active user | A mother's family whose beliefs impacted what devices could be used on an infant |
[What] was very important was the response of the others in the family. We realized that […] when you put something on a baby, it is not totally the mother's decision. (C) | ||
User | Uses the device and/or benefits from its primary function once the device is commercialized | |
Active user | Operates the device’s primary function; also called “primary user” | Patients who actively use medical devices, healthcare workers (e.g., doctors, nurses), caregivers, and medical trainers and students |
I ran a couple of focus groups with local nurses, based on ideas that our engineers had for upcoming products to see […] what needs the nurses had that weren't being fulfilled. (B) | ||
Passive user | Is impacted by the outcome of the device but has little to no control over the use of it; also referred to as “incidental user” | Patients on whom a procedure was performed with a medical device, (e.g., infants, children, adult patients, and prosthetic users) |
When you are actually putting the prototype on the baby, the baby is not still. (C) | ||
Proxy user | Shares similarities with the active user but is not an intended user of the device; is leveraged when active users are not accessible | Healthcare practitioners who work in a setting that differs from that of the intended users, laypeople (e.g., friends, co-workers), or the designers themselves |
I got to the point where I said: “Who has the largest hands here? Who has the smallest hands here?” […] I'd go around and try [3D printed models] in different people's hands. (R) | ||
Secondary-usage stakeholder | Interacts with the device outside of its primary function, throughout the product use-phase; also called “secondary user” | Technician, immunization manager, maintenance stakeholder involved in service and upkeep of the device (e.g., installation, charging, sterilization) |
We would get [the prototype] out in the hands of some service engineers and we would say, “install this and align this tube […] and tell us what is weird about it.” (S) | ||
Expert advisor | Provides expertise on the device design and usage, and the problem space based on their professional knowledge and experience | Clinical experts, product experts, other medical device company employees, academics, professors, members of partnering organizations |
We can invite people with a special competence within materials or digital solutions that we don't have in our team. (I) |
A variety of prototype forms were used by participants to engage stakeholders during front-end design activities. Prototypes predominantly represented device ideas or processes. These prototypes were categorized into three groups: (1) physical three-dimensional (3D) prototypes, (2) two-dimensional (2D) prototypes, and (3) digital 3D prototypes. Physical 3D prototypes were typically described as tangible objects made of craft materials, integrated prototypes, existing products used as prototypes, or pilot experiments involving a physical prototype used in a real-world setting. Crafted prototypes, one type of physical 3D prototype, were made quickly by participants, with readily available materials, parts, and rapid prototyping processes. In contrast, integrated prototypes, another type of physical 3D prototype, were made with processes that more closely resembled that of a commercialized product.
2D prototypes were 2D representations of a 3D object, made by hand, with digital tools, or a combination of both methods. For example, participants described using hand drawings, photorealistic renderings, and engineering drawings, and described processes through storyboards.
Digital 3D prototypes, including computer-aided design drawings, video recordings, and interactive renderings, were also leveraged with stakeholders during front-end design, notably with more technical stakeholders or when showcasing the vision of the finished product to stakeholders. Definitions and examples of prototype forms in the medical device design context are included in Table 5.
Prototype form | Definitions | Example(s) within the medical device context |
---|---|---|
Physical 3D | A physical, three-dimensional representation of an idea | |
Crafted prototype | A physical prototype made of materials that were readily available and quick to assemble; these prototypes were often qualified as rough | |
Rapid prototype | A crafted prototype made from a rapid manufacturing method, such as 3D printing, laser cutting, rapid machining, or molding | A 3D printed prototype of a device's outer shell made from stereolithography (ABS); a 3D printed functional prototype of a transportation device for medicine |
3D printing is a more functional evaluation, I would say. Say, for example, [our device has] a space where we keep the [medication], we could organize the [medication], and we use trays to pull in, pull out, and stuff like that. That's more functional. (E) | ||
Constrained prototype | A crafted prototype made from materials with fixed form, such as hardware parts and modified existing products | Plier handles used to mimic functional actuation; scrub brushes and other items with ergonomic gripping handles used to test grip when users wore bloody gloves |
They had ketchup bottles that you squeezed—it was whatever material that was available—and it had the power to communicate that ‘you would put something on your body, and you can control these [ketchup bottles]. But it wasn’t anywhere convincing as a final solution. (I) | ||
Freeform prototype | A crafted prototype made from easy-to-shape materials such as clay, foam, wood, and other craft materials | A versatile clay handle that could be molded into various shapes; a foam model to test the fit of the device concept in the laboratory space |
We use more foam to do esthetic models when we want to do some styling of a product [we ask:] “Does this product relate to the ruggedness of the product that you want?” (E). | ||
Integrated prototype | A physical prototype that had one or more refined aspects of the form or function, built using refined materials and processes | An esthetically accurate but nonfunctional prototype of an injection device; a fully functional prototype of an infant treatment device with no esthetic finish |
You would rather get a looks-like, feels-like prototype model in their hands, and describe how it's going to work. (G) | ||
Existing product | A product on the market used as a prototype to benchmark, trigger memories and reactions, and/or serve as a reference in conversations | Existing body simulators shown to discuss the important anatomy to include in the product; current operating room tools used as stimuli for conversation |
We did use some bigger syringes to actually give an example of what [the device] would look like, sometimes. […] So, usually, that was the replacement image that we would give so people would understand the general operation. (F) | ||
Pilot | A small-scale test where stakeholders used a physical prototype in its intended environment for multiple days | A functional training-device prototype used by teachers and students in a clinical setting for multiple days |
We'll leave a prototype behind in a facility for a month, then we'll go pick it up and we'll see what happened to it? […] Just to try to like see more about the lifetime. (K). | ||
Digital 3D | A dynamic three-dimensional representation of an object or process, created in part with digital tools | |
CAD model | A 3D CAD model, sometimes accompanied by computational tests | Center of gravity analysis of a handheld battery-powered device; finite element analysis of a 3D model |
[For this project], we don't do a lot of hard prototypes. A lot of it is virtual prototypes. […] Very rarely do we build a full system and send it [to the hospital] just because that's like a million-dollar prototype. (Q) | ||
Video recording of a prototype | A video recording of a physical prototype | A video of a heat test of a device |
We make a video of a prototype we're making and have one or two key questions or have Skype calls. (K) | ||
Interactive rendering | A digital model that could be manipulated to move and mimic functionality through digital interfaces | A digital interface flow mockup; a CAD model of a device manipulated on-screen to mimic the function |
We had [stakeholders] program the [operation] on the tablet with the screen mocked up. (V) | ||
2D | A still representation of an object or a process, created by hand and/or with digital tools | |
Drawing | A sketch (rough or refined) used to generate and communicate ideas and/or design concepts to/with stakeholders | Stakeholders' drawings of ideal device features; a sketch of the device functional architecture; industrial drawing of device features; drawing of the overall system |
So, sometimes we just tried kind of pencil and paper to made it look like not even printed out from CAD. Like, just redraw what I had in CAD with pencil and paper because then people would give me more, like, “Oh, she's early on, I can go ahead and give my input.” (N) | ||
Storyboard | Consecutive images detailing a use case of a product to communicate the intended interaction of the product with a person or environment | A series of images depicting how to store, clean, and interact with a device in a clinical setting; a series of images depicting the current workflow of clinicians and how the device integrates into the workflow |
They' repanels, and it's one of the best explanations we have. […] Being able to put that together to show context operation and the situational context around it has been much easier, […] being able to show that visually, versus just trying to explain it. (F) | ||
Photograph | A photography of a physical object, sometimes digitally altered | Photographs of a nonfunctional prototype used to compare with photographs of predicate devices |
The entire first six months, we didn't really send any physical prototypes at all, instead, we would just take pictures, […] have a ruler in the picture, and then send any sort of test data. (H) | ||
Rendering | A virtual image digitally processed using color and shading to make it appear three-dimensional | A rendering of the instruction manual of the device; fast and low-cost renderings of the device with different color variations; device interface mockup |
When it comes to the user interface, […] we've just done on the computer and graphics. We can actually send people a bunch of illustrations and [ask]: “What do you think of this? What does this mean to you?” (A) | ||
Engineering drawing | An image of the internal mechanisms of a device appended with written information about the image | Drawing of the inner mechanisms of a device with a list of components, specifications, and dimensions; labeled pictures of device parts with a description of functions |
We would send them pictures of cross-sections, pictures of various parts involved, a more verbal description of what this part does, and what this component does. (E) |
Prototype form | Definitions | Example(s) within the medical device context |
---|---|---|
Physical 3D | A physical, three-dimensional representation of an idea | |
Crafted prototype | A physical prototype made of materials that were readily available and quick to assemble; these prototypes were often qualified as rough | |
Rapid prototype | A crafted prototype made from a rapid manufacturing method, such as 3D printing, laser cutting, rapid machining, or molding | A 3D printed prototype of a device's outer shell made from stereolithography (ABS); a 3D printed functional prototype of a transportation device for medicine |
3D printing is a more functional evaluation, I would say. Say, for example, [our device has] a space where we keep the [medication], we could organize the [medication], and we use trays to pull in, pull out, and stuff like that. That's more functional. (E) | ||
Constrained prototype | A crafted prototype made from materials with fixed form, such as hardware parts and modified existing products | Plier handles used to mimic functional actuation; scrub brushes and other items with ergonomic gripping handles used to test grip when users wore bloody gloves |
They had ketchup bottles that you squeezed—it was whatever material that was available—and it had the power to communicate that ‘you would put something on your body, and you can control these [ketchup bottles]. But it wasn’t anywhere convincing as a final solution. (I) | ||
Freeform prototype | A crafted prototype made from easy-to-shape materials such as clay, foam, wood, and other craft materials | A versatile clay handle that could be molded into various shapes; a foam model to test the fit of the device concept in the laboratory space |
We use more foam to do esthetic models when we want to do some styling of a product [we ask:] “Does this product relate to the ruggedness of the product that you want?” (E). | ||
Integrated prototype | A physical prototype that had one or more refined aspects of the form or function, built using refined materials and processes | An esthetically accurate but nonfunctional prototype of an injection device; a fully functional prototype of an infant treatment device with no esthetic finish |
You would rather get a looks-like, feels-like prototype model in their hands, and describe how it's going to work. (G) | ||
Existing product | A product on the market used as a prototype to benchmark, trigger memories and reactions, and/or serve as a reference in conversations | Existing body simulators shown to discuss the important anatomy to include in the product; current operating room tools used as stimuli for conversation |
We did use some bigger syringes to actually give an example of what [the device] would look like, sometimes. […] So, usually, that was the replacement image that we would give so people would understand the general operation. (F) | ||
Pilot | A small-scale test where stakeholders used a physical prototype in its intended environment for multiple days | A functional training-device prototype used by teachers and students in a clinical setting for multiple days |
We'll leave a prototype behind in a facility for a month, then we'll go pick it up and we'll see what happened to it? […] Just to try to like see more about the lifetime. (K). | ||
Digital 3D | A dynamic three-dimensional representation of an object or process, created in part with digital tools | |
CAD model | A 3D CAD model, sometimes accompanied by computational tests | Center of gravity analysis of a handheld battery-powered device; finite element analysis of a 3D model |
[For this project], we don't do a lot of hard prototypes. A lot of it is virtual prototypes. […] Very rarely do we build a full system and send it [to the hospital] just because that's like a million-dollar prototype. (Q) | ||
Video recording of a prototype | A video recording of a physical prototype | A video of a heat test of a device |
We make a video of a prototype we're making and have one or two key questions or have Skype calls. (K) | ||
Interactive rendering | A digital model that could be manipulated to move and mimic functionality through digital interfaces | A digital interface flow mockup; a CAD model of a device manipulated on-screen to mimic the function |
We had [stakeholders] program the [operation] on the tablet with the screen mocked up. (V) | ||
2D | A still representation of an object or a process, created by hand and/or with digital tools | |
Drawing | A sketch (rough or refined) used to generate and communicate ideas and/or design concepts to/with stakeholders | Stakeholders' drawings of ideal device features; a sketch of the device functional architecture; industrial drawing of device features; drawing of the overall system |
So, sometimes we just tried kind of pencil and paper to made it look like not even printed out from CAD. Like, just redraw what I had in CAD with pencil and paper because then people would give me more, like, “Oh, she's early on, I can go ahead and give my input.” (N) | ||
Storyboard | Consecutive images detailing a use case of a product to communicate the intended interaction of the product with a person or environment | A series of images depicting how to store, clean, and interact with a device in a clinical setting; a series of images depicting the current workflow of clinicians and how the device integrates into the workflow |
They' repanels, and it's one of the best explanations we have. […] Being able to put that together to show context operation and the situational context around it has been much easier, […] being able to show that visually, versus just trying to explain it. (F) | ||
Photograph | A photography of a physical object, sometimes digitally altered | Photographs of a nonfunctional prototype used to compare with photographs of predicate devices |
The entire first six months, we didn't really send any physical prototypes at all, instead, we would just take pictures, […] have a ruler in the picture, and then send any sort of test data. (H) | ||
Rendering | A virtual image digitally processed using color and shading to make it appear three-dimensional | A rendering of the instruction manual of the device; fast and low-cost renderings of the device with different color variations; device interface mockup |
When it comes to the user interface, […] we've just done on the computer and graphics. We can actually send people a bunch of illustrations and [ask]: “What do you think of this? What does this mean to you?” (A) | ||
Engineering drawing | An image of the internal mechanisms of a device appended with written information about the image | Drawing of the inner mechanisms of a device with a list of components, specifications, and dimensions; labeled pictures of device parts with a description of functions |
We would send them pictures of cross-sections, pictures of various parts involved, a more verbal description of what this part does, and what this component does. (E) |
Participants engaged stakeholders with prototypes in various settings, which were categorized into four groups: (1) meeting spaces, (2) simulation environments, (3) real use environments, and (4) distant settings. Definitions and examples within the medical device context for each setting are included in Table 6.
Setting type | Definition | Example(s) within medical device context |
---|---|---|
Meeting space | A face-to-face meeting environment that did not include elements of the real use environment of the device | |
Designer's workspace | A space familiar to the design team | designer's conference room or office |
When you do the testing, you actually invite nurses, or you have a van you reserve to have nurses come to this venue. (P) | ||
Stakeholder's workspace or living space | A space familiar to the stakeholder | Hospital procedure rooms and hallways when interacting with clinical professionals, user’s home, doctor's office |
We were interacting with […] the head of the departments sitting in their offices. (C) | ||
Neutral location | A space unfamiliar to both designer and stakeholder | A conference or convention, a networking event, a hack-a-thon |
We were at a little symposium conference or something where we had a booth, and we had our demo setup and all. (X) | ||
Simulation environment | An environment made to resemble the user's environment | Cadaver lab, usability lab with anatomical models for demonstration and/or testing purposes |
We used simulation mannequins and the clinical simulation center at the hospital a lot when we would meet with users so that they could try it out. (N) | ||
Real use environment | An environment where the device would be used once commercialized | In the community or private home of the user, a hospital operation room or patient room, a training environment, a manufacturing floor |
So, when we interact with the nurses it was actually in the ward next to the baby. (C) | ||
Distant | A virtual online environment through which communication takes place | Skype call during which prototypes were demonstrated to stakeholders, a physical or virtual prototype was sent to the stakeholder (via mail or email) and stakeholder provided feedback via email or phone call |
With those visuals, we send it to them, and then we get on a teleconference call, and say, “This is our new design. What do you think? Do you have any feedback?” (E) |
Setting type | Definition | Example(s) within medical device context |
---|---|---|
Meeting space | A face-to-face meeting environment that did not include elements of the real use environment of the device | |
Designer's workspace | A space familiar to the design team | designer's conference room or office |
When you do the testing, you actually invite nurses, or you have a van you reserve to have nurses come to this venue. (P) | ||
Stakeholder's workspace or living space | A space familiar to the stakeholder | Hospital procedure rooms and hallways when interacting with clinical professionals, user’s home, doctor's office |
We were interacting with […] the head of the departments sitting in their offices. (C) | ||
Neutral location | A space unfamiliar to both designer and stakeholder | A conference or convention, a networking event, a hack-a-thon |
We were at a little symposium conference or something where we had a booth, and we had our demo setup and all. (X) | ||
Simulation environment | An environment made to resemble the user's environment | Cadaver lab, usability lab with anatomical models for demonstration and/or testing purposes |
We used simulation mannequins and the clinical simulation center at the hospital a lot when we would meet with users so that they could try it out. (N) | ||
Real use environment | An environment where the device would be used once commercialized | In the community or private home of the user, a hospital operation room or patient room, a training environment, a manufacturing floor |
So, when we interact with the nurses it was actually in the ward next to the baby. (C) | ||
Distant | A virtual online environment through which communication takes place | Skype call during which prototypes were demonstrated to stakeholders, a physical or virtual prototype was sent to the stakeholder (via mail or email) and stakeholder provided feedback via email or phone call |
With those visuals, we send it to them, and then we get on a teleconference call, and say, “This is our new design. What do you think? Do you have any feedback?” (E) |
3.2 Associations of Stakeholders, Prototypes, and Settings.
The patterns observed for the stakeholder group engaged, the prototype(s) used for the engagement, and the setting in which the engagement occurred were defined as associations of stakeholders, prototypes, and settings. The summarized frequencies of the associations at the transcript level are depicted in Fig. 1. Details about some of these associations are provided in this section. Italicized words within the text refer to categories of stakeholders, prototypes, settings, and strategies.
All stakeholders, notably users, were most often engaged in meeting spaces where they could interact casually with the prototype(s) presented. Participants described meeting users most often in the user's own meeting space because of availability and time constraints, with various forms of prototypes.
When engaging users in simulation environments, participants described only using physical 3D prototypes. Design practitioners replicated the conditions of use with supporting objects and artifacts used in the actual use environment. Some simulations were unrefined, using readily-available materials to simulate the environment, and some simulations were conducted in cadaver labs, wet labs, or other high-fidelity simulation environments. Participants asked users to perform tasks with the prototype within the simulated setting or demonstrated the prototype to users.
Participants also described engaging users mainly with physical 3D prototypes in the real use environments spanning one or multiple stages of the product's lifecycle, so they could prompt the user to perform tasks with the prototype in the use environment. In two cases, 2D prototypes were used to supplement the physical 3D prototypes, such as a digital interface on a tablet that demonstrated the programing interface of the device.
To engage distant users, although 2D and digital 3D prototypes were easier to send to users, participants also sent physical 3D prototypes home with users to test over multiple days or sent physical 3D prototypes to distant users via mail, to then gather feedback on their experience.
Participants described engaging implementation stakeholders with prototypes most often in meeting spaces. Because many implementation stakeholders were internal to the participants' companies, they were engaged in the designer's space. Participants reported that implementation stakeholders were seldom engaged in a simulation or real use environment. One participant gave a prototype to the customer to perform their own tests in a real use environment and one participant brought a physical 3D prototype to the manufacturing floor to gather feedback from manufacturing stakeholders on the manufacturing process. A subset of implementation stakeholders was engaged remotely, in a distant setting. Community partners in other countries were often engaged remotely, along with international supply chain, manufacturing, government, and regulatory stakeholders, either through sending prototypes via email or mail or by showing prototypes via videocall.
Expert advisors were also cited as being mostly engaged in the designer's space or engaged in a distant setting when meeting in person was not possible, in which case using 2D and digital 3D prototypes were easiest. If the advisors were clinical specialists, then they might have been engaged in a simulation environment to try out the prototype or witness a demonstration. No participant described engaging expert advisors with prototypes in the real use environment.
3.3 Associations of Stakeholders, Prototypes, and Strategy for Prototype-Based Stakeholder Engagements.
In this section, multiple patterns observed for the stakeholder groups engaged, the prototypes used for the engagement, and the strategies leveraged during the engagement are presented. These patterns were defined as associations across stakeholders, prototypes, and strategies, and the summarized frequencies of the associations at the transcript level are presented in Figs. 2–4. First, the associations related to users with prototypes and strategies (Sec. 3.3.1) are presented, then implementation stakeholders (Sec. 3.3.2), and finally expert advisors (Sec. 3.3.3). This section contains excerpts of engagement events during which participants explained their choice of association.
3.3.1 User–Prototype–Strategy Associations.
The patterns observed for prototypes and strategies employed with users are summarized in Fig. 2 (the strategies are ordered alphabetically in all subsequent figures to support comparison across figures). Participants most often described engaging users with physical 3D prototypes during front-end design activities. In a subset of the engagement events, a 2D prototype was chosen to achieve a given engagement strategy, while digital 3D prototypes were used in presentations, to prototype an interface, to supplement other prototypes, or were sent to distant users.
Participants discussed using physical 3D objects to engage users (Fig. 2(a)) with all 17 strategies. For example, Participant N said she felt that users could not envision the idea through other prototype forms:
Having something physical that they could hold and having something that they could move, and use, made the quality of the interaction so much better because some people just can't imagine that next step.
Participant F expressed that a physical 3D prototype generally led to 'better' feedback than other forms:
A lot of those early, early 3D printed and machined prototypes, definitely for end-users over in [a sub-Saharan African country] got the best responses. […] With the physicians, there was a lot of interest around how some of the very specific features of the device and how would apply to specific surgeries. A lot of the nurses were more focused on usability.
Participants leveraged different forms of physical 3D prototypes for different strategies (Fig. 2(b)). To task the stakeholder with creating or modifying prototypes (create), participants used crafted prototypes. For example, Participant N described making a rough handle prototype out of foam and asking users to shape it as they desired:
We did a rough cut of how the handle shape would be and then we just let them shave it off how they think it would be good. […] We used playdough to have them think]: 'How would you want this built out? How big would you want it? Where do you want the thumb to sit?'.
Users tasked by Participant N with manipulating malleable materials and combining the manipulated materials with a base prototype enabled the users to make quick and easy modifications to communicate their preferences.
Participants expressed using 2D prototypes to engage users with the create strategy (Fig. 2(c)). However, using drawings for active stakeholder engagement was perceived as ineffective for Participant B, who described users' discomfort when asked to draw:
We said, 'Here is a card, you can draw what you think the [device] would be, or you can write down characteristics that you would have in something that you would make. […] Only two [users] drew.'
Participants described leveraging the strategy to polish the prototypes shown to the stakeholder (polish) with physical 3D prototypes and users (Fig. 2(d)). For example, Participant A described removing less esthetically pleasing and unfinished elements of a prototype to avoid distracting users:
[Users] can't help but focus on the unfinished aspects even though you know it's not really a concern at this point. So when I'm trying to put something out in the field, I'm trying to get it as finished as possible, even just esthetically. I need to spray paint it or something because people will look at a 3D print and be like, why is it this color?
For a subset of strategies, physical 3D prototypes were seen as detrimental during early engagements with users. For example, Participant N discussed using 2D prototypes, such as drawings, to not bias users with a more advanced prototype and to encourage them to provide input, following the strategy to lessen a prototype's refinement when showing it to the stakeholder (lessen completeness) (Fig. 2(e)):
Sometimes we just tried kind of pencil and paper, […] just redraw what I had in CAD with pencil and paper because then people would give me more, like, ‘Oh, she’s early on, I can go ahead and give my input.'
Participants also described using renderings, another form of 2D prototypes, to show multiple prototypes to the stakeholder concurrently (multiple) (Fig. 2(f)). Participant A described how renderings allowed different design concepts to be compared without creating multiple different physical 3D prototypes, hence saving resources:
Because you can do shading and stuff and make it look pretty good and it saves you from having to go through an actual production of a 3D print or something like that which is not cheap.
Another example was the use of 2D prototypes to encourage the stakeholder to envision use cases while interacting with the prototype(s) (envision) (Fig. 2(g)). 2D prototypes provided Participant D with additional opportunities to evoke use cases:
Showing this abstract device that's floating on a white background, a lot of times people can mistake even understanding what the device does. […] We also did a version where we a little bit clumsily photoshopped it into a photo of a real person […] and tried to show where the device would go.
3.3.2 Implementation Stakeholder–Prototype–Strategy Associations.
A wide variety of implementation stakeholders, such as manufacturing, marketing, and government stakeholders, were engaged during the front end. The association frequencies of implementation stakeholders with the prototypes and strategies used are summarized in Fig. 3.
Physical 3D prototypes and 2D prototypes were both used with implementation stakeholders. Digital 3D prototypes were sent to distant implementation stakeholders or were used during design reviews with financial decision-makers.
Some participants showed polished prototypes to financial decision-makers (Fig. 3(h)). Participant A described polishing 3D printed prototypes when engaging financial decision-makers to impress and lend legitimacy to the project:
For funding purposes, it would be the nicest looking, most functional device you had at any given time because you want to impress. You do not want to show them a bunch of junk.
Some participants described using digital 3D prototypes during design reviews with the company's internal financial decision-makers (Fig. 3(i)), as exemplified by participant Q:
Another stakeholder is like the leadership team, right? The people who are our leaders guide the direction. With them, we would use a combination of the 3D models and finite element analysis to show them that the design is solid and fair.
However, when engaging external financial decision-makers or customers, some participants cited using physical 3D prototypes (Fig. 3(j)). Participant C, for example, chose physical 3D prototypes because they perceived them as more convincing than other prototype forms:
We were pitching our concept [to external financial decision-makers]. If we were showing things to them which were not real, if for example, if I'm showing a presentation or showing a booklet […], that was less convincing as opposed to if I had this thing that I would actually demonstrate in front of them.
Participant E described engaging government and regulatory stakeholders with 2D prototypes during front-end design to discuss device features and regulatory and manufacturing risks (Fig. 3(k)). Participant E described how these specific prototypes, including drawings and storyboards, were relevant to the concerns of this stakeholder group:
We would send them pictures of cross-sections, pictures of various parts involved, and a more verbal description of what [each component did], and a very detailed description of what the product would do. That is […] enough for regulatory people to comment, and come back and tell, or, “You seem to have a reusable component. You seem to have a sterilizable product.” […] [For the ministry of health officials] it does not make sense to take a huge foam mockup to them. They are more interested in what does it cost and where are you manufacturing it, and what is the battery life […]. You make really quick sketches or renders to just convey the idea. […] They're not going to be fixated on the visuals [and] would just look at the bullet points […] I think PowerPoint presentations with visuals of sketches, […] storyboards would be good enough.
3.3.3 Expert Advisor–Prototype–Strategy Associations.
Participants described engaging expert advisors with a variety of prototypes during front-end design, but described leveraging fewer of the 17 strategies with experts than with other stakeholder groups. Associations of expert advisors with prototypes and strategies are summarized in Fig. 4.
Expert advisors generally provided technical feedback, such as feasibility, based on their domain-specific knowledge. Hence, participants discussed showing expert advisors more technical prototypes, such as functional physical 3D prototypes, 2D prototypes of various concepts and device architectures for down-selection, and digital 3D prototypes. Some clinical advisors also provided feedback on the ergonomics of physical 3D prototypes.
One strategy most cited to gather feedback from expert advisors during front-end design was to supplement the prototype shown to stakeholders with additional representations (supplement), with 2D and physical 3D prototypes (Fig. 4(l)). Participant W described bringing drawings and a physical mockup to an engagement with an expert advisor:
In between user tests, we'd go to an [expert advisor] with a new idea or concept in mind, usually accompanied by a drawing or a really crude physical mock-up that shows how it's supposed to work, and consult the [expert advisor] and get their feedback, opinions about whether or not they thought that idea would work from a patient standpoint, make sure it would work from an anatomy standpoint.
4 Discussion
Our findings revealed that medical device design practitioners engaged a diverse set of stakeholders with prototypes during their front-end design processes. Although the stakeholder groups engaged by participants in this study have been reported in the literature (broadly, not specifically with respect to front-end design engagement supported by prototypes), only a subset of the stakeholder groups are currently represented in design frameworks. The stakeholder group users, including active and passive users, appear in multiple stakeholder frameworks [3,57,59]. The prominent presence of users in stakeholder frameworks aligns with literature tying user engagement to project success, notably during its earliest stages [5,60]. Other stakeholder groups reported in this study have been less frequently incorporated into published stakeholder frameworks. For example, proxy users, secondary-usage stakeholders, and expert advisors, which were identified in this study, have only been described in individual medical device design studies [4,13,61], but are absent from many frameworks (e.g., Refs. [3,57], and [59]).
Yock et al. [3] and USAID ready, set, launch [57] mentioned trade groups and healthcare facilities as two important stakeholder groups to engage during a design process. Although healthcare facility stakeholders were mentioned several times by participants as the gatekeepers to healthcare practitioners (active users), healthcare facility stakeholders were not engaged with prototypes by the participants in this study. The lack of healthcare facility stakeholders mentioned in this study might have resulted from the types of medical devices discussed and/or because of the contexts in which the participants worked.
A variety of prototypes were leveraged by the medical device design practitioners in this study to engage stakeholders during the design front end. Multiple classifications of prototype forms exist, but no single classification matched the breadth and depth of prototype forms described by the participants. The list in this study most resembles taxonomies that describe the materials and fabrication approaches for creating prototypes [62–65]. These taxonomies were used to help define the codes.
Simple physical 3D prototypes were typically described by participants by the manufacturing methods used to fabricate them and/or the materials used to develop the particular form factors (e.g., 3D printed). However, when describing more complex physical 3D prototypes, created with multiple types of materials and/or fabrication methods, participants tended to instead describe their functionality and/or esthetic properties. Hence, the integrated prototype category emerged based on the work by Jensen et al. [47]. Houde and Hill [32] stated that describing prototypes by the tools used to create them and their level of refinement can be distracting, and they proposed that prototypes should be described by their goals rather than their form. While some participants did use “goal-oriented” language to describe early prototypes (e.g., “works like”), most did not. One can hypothesize that the materials of simple prototypes and the refinement of more complex prototypes may be salient characteristics that were easier to recall and thus used as descriptors, while the goals of the prototypes might not have been as easy to articulate or were not readily recalled by design practitioners' during the interviews (i.e., might have required specific interview prompts to elicit this information).
Furthermore, when making 2D prototypes, participants commonly described drawings of concepts or photographs of physical prototypes that were then enhanced through digital alterations. Hence, the distinction between paper and digital prototypes was blurred. Similarly, some CAD models (digital 3D prototypes) were used as a basis for renderings, and the actual CAD model was seldom shown to stakeholders. The advent of virtual and augmented reality prototyping technologies may increase the use of digital 3D prototypes in the future [66] and might further blend the lines between 2D, digital 3D, and physical 3D. Hence, a material-focused description of prototypes might be increasingly difficult to articulate as prototypes are created through mixed media to a greater extent.
Several settings were identified in this study for engaging stakeholders with prototypes during participants' front-end design activities. Most front-end stakeholder engagements with prototypes occurred in meeting spaces. In addition, early in their design processes, participants engaged users in simulation and real use environments, which aligns with regulatory guidelines for medical device development that mandate designers to seek to understand the actual use environment of a device, through user feedback and observations [67]. The use of simulation environments is well reported in medical device design literature [9]. The advent of virtual reality may enhance the opportunities for designers to engage stakeholders in simulation environments, a resource-intensive endeavor [68] and one not emphasized in this study sample.
In addition to users, a few participants also engaged implementation stakeholders in real use environments, such as on the manufacturing floor, to explore other parts of the lifecycle of the device. The high proportion in the sample of engagements conducted in real use environments may have stemmed from the fact that half of the study sample designed medical devices for use in LMICs, and hence traveled to their users, with potentially greater access to the real use environment. Testing a prototype in its use environment has been shown to be essential to uncovering previously unknown requirements [69]. Mattson and Wood, 2013, suggested integrating testing of the artifact in the real use environment throughout the whole design process rather than as a “final step” [39].
Participants also leveraged distant environments to avoid the financial expenditures and time associated with in-person visits. The use of distant environments was sometimes coupled with longer periods of prototype testing performed in the real use environment when participants sent physical 3D prototypes to users to evaluate in the real use environment.
The findings from this study illustrate the broad combinations of strategy, stakeholder, prototype, and/or setting choices made by medical device design practitioners for stakeholder engagements with prototypes during front-end design activities. Some associations appeared more frequently in the dataset, for example, participants demonstrated a preference for polishing prototypes as opposed to lessening the completeness of the prototype when engaging implementation stakeholders. This tendency might have been due to a high number of engagement events where financial decision-makers were shown polished prototypes to gain their support, where the commonly accepted practice of showing users low-fidelity prototypes constructed quickly [providing] limited or no functionality to encourage preliminary feedback [70, p.78] did not apply. Furthermore, the strategy to supplement was common across all stakeholder groups and prototype forms, which might indicate that for many stakeholder engagement activities, a single prototype form does not adequately support the full range of stakeholder engagement activities.
In our data set, expert advisors were not associated with a wide variety of strategies nor engaged at high frequencies. This finding may have resulted from the existence of common disciplinary “language” shared between designers and advisors and/or the nature of the relationship between advisors and medical device companies where advisors may have been perceived to be extended members of the design team and therefore the engagements might have been less formal and resulted in less strategic pre-engagement planning work.
Participants highlighted associations of 2D and digital 3D prototypes with specific stakeholders, based on the technical background of stakeholders. For example, nontechnical nonuser stakeholders were often shown 2D prototypes (particularly government and regulatory stakeholders), while technical stakeholders (e.g., expert advisors, internal financial decision-makers), were shown CAD models. CAD models can communicate functional and technical aspects of the prototype and might be harder to understand when one is not familiar with cad software, which could explain their limited use with stakeholders other than those interested in the project's technical feasibility. Prior research in the automotive industry has shown that to convince stakeholders of the potential of a project, such as financial decision-makers, strategies comparable to supplement are leveraged, and physical 3D and 2D prototypes such as PowerPoint slides, and diagrams have been used in conjunction with video recordings of mockup scenarios [71]. In contrast to internal financial decision-makers, external financial decision-makers were presented with physical 3D prototypes that were polished. Changing the engagement parameters based on the stakeholders' technical backgrounds has been recommended by authors in the software design space [72,73] and one can see such changes described in the study data. Future research could include the technical background of stakeholders in their categorization as well as their internal/external categorization.
The many associations found in this study can form the basis of a toolkit for stakeholder engagement with prototypes during front-end medical device design. While more research is needed to understand specific associations, a reassuring subset of the findings aligned with associations that have previously been reported in the literature across various design fields. For instance, strategies leveraged primarily with users, such as to simulate, observe, subset, and reveal, were strategies typically found in guidelines for usability testing and medical device design [3,9]. Participants described applying such best practices during very early informal testing scenarios to better understand the requirements around usability and user preferences. Physical 3D prototypes were emphasized by participants as the most effective prototypes to engage users, an existing recommendation in engineering design texts [74].
4.1 Limitations.
Limitations of the study included participants' open interpretations of what constituted front-end design activities. Although a definition was provided at the start of each interview, participants had varying perceptions of what constituted front-end design activities. Further, participants had different job roles and worked on different types of medical devices, which may have affected their front-end design experiences. To partially mitigate such effects, the pool of prospective participants was intentionally limited to those individuals that had prior experience designing mechanical and electromechanical medical devices. Although narrowing the participant pool controlled for some factors, it limited the diversity of the sample with respect to the broader medical device industry. Participants were mostly from U.S.-based companies, which further limited the generalizability of practices across geography and contexts.
The stakeholder groups emerged based on participants' descriptions of their roles and the type of feedback stakeholders provided. However, some stakeholders could have belonged to multiple groups. For example, a clinician expert advisor or a community partner could have sometimes acted as a proxy user or active user. Hence, frequencies of stakeholder groups, along with prototype forms, setting types, and associations, require further study to determine a more specific prevalence of behaviors.
4.2 Implications.
Practitioners, both novice and professional, can use the lists developed in this study to evaluate their stakeholder engagement plans and strive to consider more diverse approaches to front-end design stakeholder engagements with prototypes. By developing general definitions of stakeholders, prototypes, and settings, the results may be applicable across industries and contexts. The domain-specific examples provided illustrated different stakeholders, prototypes, and settings with nuanced explanations, applicable to medical device design. The associations of strategy, stakeholder, prototype, and setting exemplify the various intentional choices of design practitioners when engaging stakeholders with prototypes during the design front end. High-frequency associations could be used as guidelines for promoting novice designers' awareness of ways of engaging stakeholders with prototypes. Lower frequency associations could inspire potentially novel stakeholder engagement approaches for seasoned practitioners.
5 Conclusion
This study provided a comprehensive description of stakeholders (users, implementation stakeholders, and expert advisors), prototypes (physical 3D, 2D, and digital 3D), and settings (meeting space, simulation environment, real use environment, and distant) leveraged by practitioners during front-end medical device design activities. The breadth of stakeholders, prototypes, and settings illustrates the many ways practitioners conduct front-end activities (e.g., engaging proxy users and government stakeholders with prototypes, using constrained and free form physical 3D prototypes or photographs and video recordings of prototypes). The descriptions and categorizations of stakeholders, prototypes, and settings, as well as the rationales provided for using specific forms of prototypes for engaging specific groups of stakeholders in certain settings, have the potential to enhance existing design frameworks and inform design practitioners' front-end prototyping practices with stakeholders. The results of this study were based on practitioners' perceptions and recollections of prototyping strategies used; additional research could explore which of these strategies are most effective in various contexts. Future work should also explore the transferability of these findings across industries.
Acknowledgment
We thank all participants of the study for their contribution to the study.
Funding Data
National Science Foundation Early Concept Grants for Exploratory Research (NSF EAGER) (Grant No. 1745866; Funder ID: 10.13039/100000001).
National Science Foundation Graduate Research Fellowship Program (NSF GRFP) (Grant No. 2017248628; Funder ID: 10.13039/100000001).
University of Michigan Rackham Merit Fellowship (RMF) (Funder ID: 10.13039/100007270).
University of Michigan Rackham Graduate School Summer Research Opportunity Program ESROP) (Funder ID: 10.13039/100006801).
Appendix A: Definitions Framing the Research Questions
Appendix B: Sample Interview Questions
Example Engagement Event From Ref. [75]
Interview data excerpt:
I had to work on ways how to attach [the device]. We got a collection of nurses, both U.S. based nurses1 but also nurses here in the U.S. who had experience or were from other countries2. (…) What we were putting in front of users was a little more polished3. It was stereolithography print in ABS4 and it sort of had titer tolerance dimensioning and it contained a battery and everything like that. Then I had my own overlays made that would put on the front, so they were pretty good-looking prototypes5 by the time we were getting the really detailed user feedback at that point.
Engagement event: Participant conducts an engagement activity with 1proxy user (stakeholder group) and 2active users (stakeholder group), where the 43D-printed prototype (prototype form) used in the engagement is 3,5polished (strategy type).
Any additional interview excerpts pertaining to this stakeholder engagement event were associated with this engagement event. For example, the participant described the composition of the engagement room later in the interview, which was then associated with this engagement event.