Piloting the FLOOD-serv System

Since mid 2018 the FLOOD-serv Project has entered in its last phase called Verification, Piloting, Evaluation and Validation. In brief, we refer to it as Piloting. In the piloting phase a software system, in our case the FLOOD-serv System, is deployed on a limited scale in order to be tested, verified, validated and evaluated by a limited number of users. In our case piloting the FLOOD-serv System means deploying and testing it at the level of the five pilot cities, i.e. five partner organizations at city or otherwise local level:

  • Municipality of Bilbao (Spain);
  • Self-Governing Region of Bratislava (Slovakia);
  • Municipality of Genova (Italy);
  • Prefecture of Tulcea (Romania);
  • Municipal Council of Vilanova de Famlicao (Portugal).

Piloting is scheduled to take place in three cycles each containing 7 phases as shown in the figure below:

 

 

Each cycle contains several preparatory phases: 1-4 (planning, training participants, etc), the actual testing (phase 5), the analysis of results (phase 6) and the execution of improvements to the FLOOD-serv System and its Components (phase 7).

Who is involved in piloting?

Piloting within FLOOD-serv involves orchestrating the participation of a variety of people and organizations who may be internal to the FLOOD-serv Consortium or external to it:

  • Professional testers, employees of the Technology Companies responsible for the technical components of the FLOOD-serv System. These professionals perform testing for verification that various functionalities of the system work according to specifications. This testing is done in a test environment before the components are deployed.
  • Employees of the pilot cities, persons involved in the management of emergency situations generated by floods. They validate the system for conformity to their expectations and may provide additional inputs for improvements.
  • Representatives of stakeholders, persons from organizations others than pilot cities but who may be involved in some way in emergency management. These may be specialized agencies like firefighters or lower or higher level organizations than the official pilot city (e.g. lower level municipalities when the city is represented in the project by a regional organization).
  • Last but not least, citizens. The entire FLOOD-serv Project and its creation – the FLOOD-serv System, are ultimately designed to be citizen centric, respond to citizen needs and allow for citizen involvement.

What participants in piloting do?

Participants in piloting execute tests of what the FLOOD-serv System does. What various functionalities of the System do is compared to the specifications as they were defined in the user requirements elicitation phase of the Project. Moreover, as part of the validation process, users/testers may assess to what extent functionalities of the system are useful as they are or may require modifications or additions. In addition to functional testing, participants answer a series of questionnaires referring to the usability of the system.
Ultimately participants test the System and offer their opinions in both quantitative and qualitative judgments. These are used by us to analyze the results of the testing and to plan changes and improvements to the System.

Results of the first cycle of testing

The first cycle of testing was finalized in October 2018. It was probably the most difficult as it was the first time many of the partners were involved in this process. Technical training provided by technical partners to pilot cities was crucial. The main focus was on the involvement of employees of pilot cities but important steps were also taken for the involvement of citizens and stakeholders.
Overall the target of the first cycle of testing was to test 70% of the functionalities and to have 70% of them passing the tests. Overall these targets were well surpassed. Some issues and bugs were identified and the technical teams worked on correcting them before the second cycle.

The second cycle of testing: status and expectations

Our aim compared to the previous cycle is to extend the number of functionalities tested (up to 95% of elicited requirements) as well as to expand on the feedback obtained, especially qualitative feedback. We are also looking forward to a wider involvement of citizens and stakeholders in the testing piloting process.
Actual system testing is planned to be finalized by the end of February 2019.