Overview
The TRANS-TALK Thematic Network was set up in January 2000 under the Fifth Framework Programme with the objective to provide a networking platform for those involved in the field of transport evaluation; explore the conceptual and empirical problems in contemporary transport evaluation; and develop guidelines that help improve transport evaluation.
The launch of the TRANS-TALK thematic network reflects the renewed strategic interest in transport evaluation. This has different sources - conceptual, empirical and policy-relevant.
At the conceptual level, it is related to the realisation that the conventional methods for evaluation that apply to infrastructure appraisal, like cost-benefit analysis, are not adequate to address contemporary challenges. At the empirical level, there are the problems of choice among a multitude of methodologies and of measurement and comparison. Finally at the policy-relevant level, there is an intensified demand for evaluation that looks at the (long-term) outcomes of policy rather than alone its (short-term) direct outputs.
The TRANS-TALK Thematic Network had the following objectives:
- To provide a networking platform for those involved in the field of transport evaluation from either the demand or supply sides;
- To explore the conceptual and empirical problems in contemporary transport evaluation;
- To develop guidelines that help improve transport evaluation in providing a preliminary framework for the integration of different policy and project evaluation methodologies, thus setting standards in the field.
As a thematic network, the work programme of TRANS-TALK was implemented through three workshops and supporting activities that included literature reviews as input to the setting of the workshop agendas, the publication of the conference proceedings, the writing up of synthesis reports to summarise the conclusions and main recommendations deriving from each workshop, and a survey among transport RTD projects to obtain feedback on the themes of the thematic network and for setting up a Virtual Library.
The three workshops organised in the framework of TRANS-TALK were:
- WS1: Policy and Project Evaluation: Context, Theory and Methods (29-31 May 2000)
- WS2: Projects, Programmes, Policies: Evaluation Needs and Capabilities (6-8 November 2000)
- WS3: Improving Evaluation Practices in Transport: Towards a Better Integration of Technical and Political Perspectives (30 May to 1 June 2001)
All workshops were organised in Brussels.
A selection of the papers presented at the series of TRANS-TALK workshops were published in the two TRANS-TALK book publications with the Ashgate Publishing House (Giorgi et al., eds., 2002; Pearman et al., eds., 2003).
Funding
Results
Current state and main trends:
Transport-specific evaluation frameworks exist in most Member States of the European Union, albeit mainly for infrastructure assessment at project level. These are used for ex-ante evaluation or appraisal, and for the prioritising and phasing of projects. The appraisal of transport infrastructure relies mostly on cost-benefit (CBA) and multi-criteria analysis (MCA). Road and rail projects are the ones most commonly subjected to CBA. Analytically, CBA is widely used for the so-called direct transport impacts. Among environmental impacts, noise and local air pollution are included in appraisal across Member States but they are valued in monetary terms only in half. The treatment of indirect socio-economic impacts is uneven. At the programme level, and with few exceptions, the drawing of transport master plans is common practice in EU Member States at least for one mode - most frequently again road. Policy or multi-modal master plans, on the other hand, are less common, in part because of the remaining co-ordination problems between relevant agencies or policy institutions. At the European level we find no integrated or harmonised evaluation framework to apply to projects, programmes or policies that are of common interest or added value. This is in part the result of the lack of harmonisation with regard to transport data, forecasts, models or scenarios. The other reason for the absence of a harmonised evaluation framework is largely political, and related to the demand for flexibility by Member States in view of the subsidiarity principle.
Pressures for change:
The gradual consolidation of the European Union as a polity, with own institutions, an own redistribution budget, own policy networks and agendas and own decision processes, represents a challenge to evaluation in the European context. This is not least because of the subsidiarity principle which prescribes that the Union may only intervene where there would be an added value from this intervention. Only there does not always exist either clarity or agreement as to how to define and measure this added value. At the more technical level of information, being able to evaluate the European-added value also means being able to rely on at least a minimum set of harmonised data inputs, and, in part, modelling outputs, including projections and forecasts. Data availability and comparable information remains a problem. Furthermore, our knowledge on the underlyin
Policy implications
- At the policy level there are two types of evaluation that are relevant, namely, output evaluation at regular intervals to examine or check whether the action points or measures announced have also been implemented; and outcome evaluation at the short-, medium- and long-term for the purpose of monitoring with reference to broad impact indicators. A combined monitoring system combining output- and outcome-specific indicators is a support tool that no decision-making level can do without, if policy evaluation is to be taken seriously.
- The level of detail of the evaluation of a programme's results or outcomes depends on the character of the programme. Some programmes are so multifarious that evaluation can only be carried out at the aggregate level, as with policy. Other programmes are more clearly delineated, thus allowing the application of more detailed evaluation techniques.
- At the project level, a certain degree of harmonisation is called for with regard to the background assumptions to guide forecasting exercises. The same applies to transport data and documentation.
- In view of the conflicting interests whether there should be (or not) a harmonised evaluation framework for transport, the open co-ordination method should here be used as prescribed by the recent White Paper on European Governance.
- The evaluator must ask and document whether a policy, programme or project was implemented and/or how successful it was, yet must also inquire into those factors, specific to the decision context, that have facilitated or obstructed its implementation.
- Complex decision-making procedures call for a new culture of evaluation that places more emphasis on deliberation, especially with regard to appraisal. At the minimum level this involves making conceptual and methodological choices transparent to allow reflective discussion - making evaluations open for critical review is a major step towards establishing a deliberation culture in evaluation. At the maximum level, the evaluation should be thought of as simulating a decision process and use methods for gathering different opinions and bringing different actors together to discuss issues of common concern.
- The multiplicity of evaluation methods can be mistaken to imply that everything goes, more specifically that either any (desirable) result is possible to deliver if the choice of methods is carefully thought through, or that differences in evaluation results are methodologically grounded. Absolute relat