Executive summary
D91: Evaluation of project results
This deliverable describes the work undertaken in WP09 of the SmartGov Project, and concerns the evaluation of the SmartGov Platform. The overall aim of SmartGov is to specify, develop, deploy and evaluate a knowledge-based platform to assist public sector employees to generate online transaction services by simplifying their development, maintenance and integration with already installed IT systems. The methods used to conduct this evaluation are; detailed questionnaires, field observation, analysis of operational or managerial data and technical-log data. The structure of the evaluation was based upon the framework developed in deliverable D8.1. With respect to this model, the evaluation was categorised into three main components:
Technical acceptability: The Platform software was assessed against the user requirements as outlined in deliverable D4.1. Following this we evaluated specifically, the response, reliability, compatibility and installation of the platform. In the conformance to user requirements analysis we find that the overall total of user requirements met either fully or partially is 78%, with more than 80% of the compulsory requirements being met. Evaluation of the installation procedure for the SmartGov software noted that as it stands it works well enough, though for some components specialist knowledge is required that may be beyond that which a public authority, IT engineer might reasonably be expected to have.
Usefulness: This was concerned with utility and usability issues relating to how the Platform performed and was perceived by the real users in their native environment. In this we took a holistic view of the Platform in-situ at the two pilot sites detailing the use of the development environment by the Public Authority Staff to create the online pilot services. The results of this evaluation of platform usefulness were generally positive in that the majority of the success criteria were met, taken across the two pilot sites as a whole. However there was a disparity between the two sites, which can be seen in the separate results for each site. This disparity was due to the fact that the service developers in CEC had limited IT experience, whereas in GSIS, the individual service developers had a high level of IT expertise. However, some useful amendments to the platform were drawn from this evaluation.
Social/organisational acceptability: This focussed on a cost benefit analysis of the Platform in order to estimate the overall added value of Smartgov. The cost benefit analysis carried out by consortium partner Archetypon, suggests that the platform could provide added value.
Proceed to download D91 or return to the results page
Last updated: October 2003
