Many innovations in the transport sector (such as technologies, transport concepts and policy measures) are tested and demonstrated at pilot scale prior to decision-making on their large-scale introduction. This reduces the investment risk and permits fine-tuning of the innovation. However, many pilot and demonstration projects have been conducted in an ad-hoc way, without reference to previous experience. This has led to avoidable problems and sub-optimal results, including the failure to provide answers to the underlying policy questions. A significant factor has been the absence of a reference text or guidelines on standard procedures and good practice for such projects.
The aim of MAESTRO was to establish as a reference text, a common framework for the selection, design and evaluation of transport-related pilot and demonstration projects. This would provide practical advice and strengthen the link between project results and policy decisions.
MAESTRO developed, tested and documented a set of guidelines for transport-related pilot and demonstration projects. These guidelines cover the various decisions and evaluation phases through the entire life-cycle of a project, from the definition of the transport problem, through project design and initial evaluation, to implementation, final evaluation and exploitation of the results. The guidelines can be found on the project web site (see contact details).
The expected groups of users are: decision-makers; project managers; experts in the project team; and other stakeholders. The guidelines provide an overall framework as well as checklists for particular stages in the project. It is anticipated that they will be used in conjunction with other texts dealing in more detail with particular aspects of project management and evaluation.
The guidelines identify three main stages of evaluation:
- an initial evaluation of the selected site and preliminary design for the project, looking at the expected impacts of the project in mainly qualitative terms;
- an 'ex-ante' estimate of impacts, based on the detailed design in comparison with a 'do-nothing' option;
- an 'ex-post' evaluation of the actual impacts and the extent to which the project has realised its objectives. The outcome of this stage will influence the decision on whether or not to proceed to full-scale implementation.
A vital feature of the MAESTRO methodology is the clear definition of the project objectives. These objectives need to be framed to address the policy goals and questions of interest, and they must be explicit enough to guide all the subsequent stages of design and evaluation.
The MAESTRO guidelines are intended to act like a quality assurance framework, ensuring that pilot and demonstration projects are run:
- efficiently, utilising funding resources in a way that is closely related to objectives and project plans;
- effectively, such that the project results give clear-cut answers to the policy-related questions/issues that called for (the piloting of) a new transport solution.
The guidelines advise projects how they can add value at a European level, for example by making findings more transferable between different city and national contexts. Advice is also given on taking the decision whether or not to proceed to full-scale implementation of tested policies.