ISERN Program
| Sunday | 6th October | |
| 18.30-20.30 | Welcome reception | |
| DAY 1: | Monday 7 October | |
| 8.30-9.00 | Registration | |
| 9.00-10.30 | Welcome and new introductions | |
| Chair: Dieter Rombach | ||
| ISERN is open to academic and industrial groups world-wide that are active in empirical software engineering research and willing to adopt the empirical research framework. ISERN members are pairs of organization and contact person. If the contact person leaves the organization, the organization must reapply for membership. Interested organizations may apply by sending an electronic proposal to "isern at informatik.uni-kl.de" describing their past experience in empirical software engineering research as well as their expectations from a future ISERN membership. Candidates will be invited to observe the ISERN Meeting following their application. | ||
| The goal of the session is to facilitate the membership application process by giving an opportunity for candidates to present their research and for observers to introduce themselves. Membership is granted according to a 3-step procedure: | ||
| 1) Attending as invited observer at an annual ISERN meeting. | ||
| 2) Attending as invited candidate at the following ISERN meeting giving a presentation. Membership is granted if a two-thirds majority of current members approve the application in an email voting after the meeting. | ||
| 3) Attending as a full ISERN member at following meetings. | ||
| Current members present contact/affiliation changes (give a 2 min presentation each): | ||
| Further information regarding changes (newly accepted members, leaving members) | ||
| 10.30-11.00 | Coffee break | |
| 11.00-12.00 | Plenary workshop: Top Ten Unsolved Problems in ESE ‐ Results from last year's session | |
| Chair: Andreas Jedlitschka / Natalia Juristo | ||
| Abstract: The goal of the
workshop is to continue the process of creating a list of the 10 most
important problems the ESE community should focus on for the next years. The
list will be elaborated following the Delphi approach. Last year,
participants were asked to propose a list of at least 5 problems they think
ESE should try to solve in the near future. The purpose of the session is to present the results from last year's session on the open questions and to initiate the next step, which is the voting. There are three large groups of problems: (1) those concerning empiricism at its heart, (2) those concerning SE topics which are of interest and for which people would like create/have evidence, and (3) how to disseminate/exploit what we have. During the session (first round), we would like to discuss for each group what we found, aiming at confirming that we did the summary correctly and perhaps some complementary items (you could name it consolidation). To initiate the discussion, we will give a short explanation of what the item means. In the second round, we would ask the people to vote on the items with regard to the items' importance (this term needs to be defined). |
||
| 12.00-13.30 | Lunch | |
| 13.30-15.00 | Ongoing collaborations | |
| Chair: Nachi Nagappan | ||
| Short presentations describing collaboration among ISERN members. | ||
| 15.00-15.30 | Coffee break | |
| 15.30-16.30 | Searching for collaboration | |
| Chair: Guenther Ruhe | ||
| Participants are given 2 minutes each to present their poster (where they look for collaboration in a specific topic). The rest of the time people are left free to talk to the presenters. | ||
| 16.30-17.15 | Open Space: This is a space for informal meetings. A list of announced meetings is avalable here. | |
| 17.15-17.30 | Wrap-up and plan for Tuesday | |
| 17.30-18.30 | ISERN SC Meeting (by invitation) | |
| 19.00-22.00 | ISERN Dinner | |
| DAY 2: | Tuesday 8 October | |
| 9:00-10:00 | Plenary workshop: ISERN recommended core reading list | |
| Chair: Per Runeson | ||
| Abstract: The ISERN Recommended Core Reading List is intended to give newcomers to ISERN a knowledge platform of both historical and current papers. A proposed list of papers was presented and discussed at ISERN 2012, and now we present an updated version, as well as some experiences of the use of the reading list. The session will also discuss how to make sure the reading list is evolving over time. To that end, we invite all ISERN members to select one of their recent papers and be prepared to present it very briefly in breakout groups for possible later inclusion in the reading list. | ||
| 10.00-10.30 | Coffee break | |
| 10.30-12.00 | Session A: GQM+Strategies Working Group: Collaboration Mode and Future Work | Session B: Naming the Pain in Requirements Engineering – NaPiRE Globally distributed, yearly replicated family of surveys |
| Co-chairs: Victor Basili / Jens Heidrich | Co-chairs: Daniel Méndez/ Stefan Wagner | |
| Abstract: The GQM+Strategies approach is an evolved version of GQM that aims at aligning organizational goals and strategies through measurement. Goals are future states the organization wants to achieve (e.g., in terms of its business). Strategies are any actions defined for obtaining these goals. Its main goal is to set up a strategic measurement programs allowing for data-based decision making in the organization. Goals and strategies across all levels are linked to each other and measurement data is collected for systematically evaluating goal attainment and the success/failure of strategies. The session should bring together collaborators from the current GQM+Strategies working group and attract new collaborators having an interest in actively participating in the development of GQM+Strategies. The session will deal with (1) defining a new collaboration mode among the workgroup participants based on current issues and (2) setting up a rough roadmap for the future development of the approach. | Abstract: We published the design of a global family of surveys on Requirements Engineering and first results from Germany. The first replication is currently taking place in the Netherlands. The idea for the survey was born at last years ISERN meeting and the design was strongly supported by members of ISERN. Our aim at the workshop is to briefly present the results of the first survey round, the infrastructure used and, in particular, to initiate a new long‐term research collaboration where different researchers conduct an independent replication of the survey whereas the design of the instrument, the synthesis of the results, and the publication of the results shall be performed on a yearly basis using the ISERN forum. As part of the thematic workshop, we plan to re‐adjust the used instrument and to agree on a detailed process for future replications. | |
| Session goals: To coordinate what is being done at the various organizations, define a collaboration mode, and lay out a research and application plan for future work based upon what has been learned so far. | Session
goals: * present the current results from the survey performed in Germany and the Netherlands * re‐adjust the instrument used in dependency to research interests of the participants * agree on a process for a yearly replication, and jointly define a time schedule where * the design shall be formally accepted * the surveys shall be initiated in different countries under independent coordination of corresponding researchers (using the same provided infrastructure) * the results of all replications shall be synthesised, presented and discussed at the next ISERN meeting, before publishing the results as a community effort |
|
| Development of the session: The session will be organized as an interactive discussion on the following topics: (1) Collaboration Mode and (2) Roadmap and Future Work. | Development
of the session:
1. Short presentation: Introduction & Overview of survey design and first results from Germany and the Netherlands. (15 min) 2. Re‐adjustment of the questionnaire a. Short presentation of survey design and hand out of underlying questionnaire. (5 min) b. Working session: Each participant revises the questionnaire according to his area of research interests. (20 min) c. Joint discussion on the synthesis and first sketch of the categories of questions for the next replication. (10 min) 3. Definition of the process a. Short presentation of the proposed design for the yearly presentation. (5 min) b. Open discussion on the process for yearly design including (20 min) i. the re-‐design of the questionnaire at ISERN ii. the independent replication of the survey at the participants country using their research contacts iii. the collaborating synthesis of the results for a joint publication and presentation at the upcoming ISERN meeting 4. Summary and joint discussion on potential research co‐operations (15 min) |
|
| Background and recommended reading list: Participants should either be part of the actual GQM+Strategies working group or have sufficient knowledge about the GQM+Strategies approach from articles and/or tutorials. | Background and recommended
reading list: For many years, we have observed industry
struggling in defining a high quality requirements engineering (RE) and
researchers trying to understand industrial expectations and problems.
Although we are investigating the discipline with a plethora of empirical
studies, those studies either concentrate on validating specific methods or
on single companies or countries. Therefore, they allow only for limited
empirical generalisations.
To lay an empirical and generalisable foundation about the state of
the practice in RE, we have designed a family of surveys on the industrial
expectations, the status quo, and contemporary problems in RE and its
improvement. This family of surveys shall bring together different research
communities and follow the principles of openness, e.g., by making the
instruments and the results available via the open PROMISE database. As a
long‐term goal, the family of surveys aims at supporting us to steer
future research in a problem-driven manner. At this years EASE conference, we presented the design of the family of surveys and presented our results from Germany. The first replication is currently taking place in the Netherlands. The idea for the survey was born at last years ISERN meeting, its design and validation (e.g. of the questionnaire) was strongly supported by members of ISERN, and we strongly believe that ISERN is the ideal forum to coordinate the family of surveys in the future. Our aim therefore is perform a thematic workshop to openly discuss the survey design and its future independent within the community. Recommended Reading: D. Méndez Fernández, S. Wagner. Naming the Pain in Requirements Engineering: Design of a Global Family of Surveys and First Results from Germany, In: Proc. 17th International Conference on Evaluation and Assessment in Software Engineering (EASE 2013), ACM Press, 2013 |
|
| Expected Outcome: (1) Rules for partners participating in the GQM+Strategies working, (2) Rough roadmap and work assignments | Expected Outcome: The outcome of the workshop shall be an informal agreement on the collaboration and its operationalisation. | |
| 12.00-14.00 | Lunch | |
| 14.00-15.30 | Session A: Empirical Specification and Evaluation of the ilities | Session B: Replications |
| Co-chairs: Barry Boehm/Ray Madachy | Co-chairs: Maria Teresa Baldassarre / Oscar Dieste | |
| Abstract: A topic previously researched by FC-MD and USC on the NASA High Dependability Computing research project, and currently being researched by USC, NPS, MIT and others. The ilities, or nonfunctional requirements, specify how well a system is performing (reliability, evolvability, agility, scalability, usability, affordability, etc.), where the functional requirements specify what a system is performing. Getting the ilities defined and measured incompletely or inaccurately has been the source of numerous software project overruns and shortfalls. For example, a stakeholder may specify reliability in terms of "The system shall have a mean time between failures (MTBF) of at least 10,000 hours," where "failure"" means loss of liveness. However, other stakeholders may be relying on the system's ability to deliver all messages accurately, and will be seriously dissatisfied with a system with a liveness MTBF of 10,000 hours that loses or delivers garbled messages several times a day. Various standards such as ISO/IEC 9126 and the 25000 series have tried to usefully define the ilities and their relationships, but are generally incomplete in ility coverage and recognition of the variation of ility values by stakeholder need, by operational scenario, or by conflicts with other-ility requirements. | Abstract:the importance of replications is known to be a relevant issue
for the ISERN community. Most members are active in this field as they not
only carry out single empirical investigations but also promote replications
to enforce, support research findings and build families of experiments.
Previous ISERN sessions have focused on aspects such as guidelines for reporting controlled experiments, case studies and replications. In this year’s session our intention is to focus the discussion on replication types and collect information/lessons learnt that will allow to identify a shared classification of replications in SE. |
|
| Session goals: To assess the state of the art and practice of ility specification and evaluation, and assess the viability of a community of interest in the area | Session
goals: some questions that the session will try to
answer are: how do ISERN members consider replications? What replication
types are used? How much collaboration goes on between replicators? Our intention is to come out with an agreed list of replication types for the software engineering field, along with an explanation of what each type of replication is good for and what the potential drawbacks are. |
|
| Development of the session: Request participants to concisely summarize their related research and challenge areas; identify primary challenges and promising approaches for addressing them; assess interest in participating in a community of interest in the area. | Development of the session: a short presentation will illustrate the current classifications of replications that are present in literature. From there, participants will discuss characteristics of each type of replication pointing out positive and negative points, feelings, ideas regarding replications. | |
| Background and recommended reading list: | Background and recommended reading list: Participants should be knowledgeable in the field of replications and have experience in conducting types of replications. | |
| Expected Outcome: Continuing the work beyond the ISERN session | Expected Outcome: elicit ideas that the community has regarding replications, based on experiences, lessons learned, and reported literature. Identify a classification of replication studies that the community can use as reference for replication studies. | |
| 15.30-16.00 | Coffee | |
| 16.00-17.00 | Workshops summary session | |
| Chairs: Jeff Carver and Barbara Russo | ||
| Each group describes conclusions, outcomes and future actions in 10 minutes | ||
| 17.00-17.30 | ISERN business | |
| Chair: Dieter Rombach | ||
