Dec 202011
 

Its a bit early to be making predictions about how IGIBS might evolve, but a recent presentation to the EDINA geoteam followed by some discussion indicated some of the possibilities.

  • The WMS Factory Tool.  With the simple but effective styling capability that Michael Koutroumpas engineered, I think we have a prototype thats not too far off a production strength tool.  There are loads of scenarios where its valuable to have access to a tool that makes it easy to see your “non-interoperable” data alongside the growing number of INSPIRE View Services (read WMS) from public authorities across Europe going online.  So top of my list is improving this tools styling capability.
  • Associated with this would be better understanding of necessary data publication infrastructure, eg, making it easy to use the other OGC Web Services.  Something like the GEOSS Service Factory ideas emerging from the EuroGEOSS project.  I think there is a real demand for tools to make it easy to use the OGC standards.
  • In the immediate future, I think its likely that the IGIBS team will do some promotion of the project outputs, eg:
    • presenting the project at relevant events, eg, Association GI Laboratories Europe conference, OGC Technical Committee meetings.  This might cost as little as £500 depending on where the event is.
    • use of social media to promote both the WMS Factory Tool and the report on “Best Practice Interaction with the UK Academic Spatial Data Infrastructure”.  This too could cost as little as an additional £500.
  • The latter report is worthy of a lot more investment.  A major output from this project, possibly the single most important output, is the increase in use of UK academic SDI services within the Institute of Geography and Earth Science (IGES) at Aberystwyth University.  IGES is acting as an exemplar for best practice research data management around geospatial data, the department is actively building on the IGIBS work and it will be interesting to see how it develops and if other departments in other institutions see the benefit and start to emulate what Aberystwyth is doing.  More work promoting Steve Walsh’s report would help.
 Posted by at 21:19 Project Management Tagged with: , , , , , , , ,  Comments Off on IGIBS Followon and use of Underspend
Aug 082011
 

My apologies to the Duke of Wellington for mutating his often quoted call to a jilted mistress about his intimate letters, but the sentiments in the original statement do suggest the power of the publication process to give information a life of its own. 

If geospatial data were published in a similar way to research findings (or even to letters from the rich and famous to their mistresses) then data management and the academic spatial data infrastructure (SDI) would be an even more  rapidly developing entity, that had the commitment of every academic researcher that generates such data.  OK so a sweeping unrealistic statement but this is how I came to the thought…. 

One of the pillars of modern day science is the peer review process. It takes a piece of original research and, through the  publication process, many stages of refinement are applied to it until the researcher is satisfied that it stands a chance of acceptance by an appropriate journal. During this process it will be proof read, checked for errors and formatted in the appropriate way. Then after input from independent referees it will be further improved and finally, if judged acceptable, the research is published where it can be accessed, seen and discussed by the wider scientific community (or any community that wishes). 

After publication it is archived and catalogued so that it can be found on-line or in hard copy and can be used and quoted by anybody who wishes. There may be a network of people and libraries that will have subscriptions to the journals and they will see the newly published articles appear on their shelves, desks or screens every few months. Finally if the research is worthy it may be used as a component of more research and go on to help develop the knowledge base. 

Concomitant with this process is the recognition given to the researcher and to the associated Institution that can result in promotion for the former and extra funding for the latter. This rather idealised description of the peer review process is something that the majority of academic staff and postgraduate students are fully engaged with and committed to. You might have to forgive my simplistic view but all I wish to establish is the principle that the publication process and the recognition it attracts drives the quality, accessibility and reuse of research findings. 

Now let’s consider how geospatial data is managed. It’s not so easy to simplify the process as there will be much greater variation. Some important data will be lodged with data centres where it may have a guaranteed 10 year life span (if its lucky) or maybe the metadata will be put in a discoverable place with a series of hurdles to cross before anybody can get access to the data itself. Quite often the data will never leave the IT systems of the Institution that it was created in; rather it will take second place to the research publications and may not be made accessible at all. This has been shown to be through worries over intellectual property, through a lack of awareness of a suitable data management and publication methods and a lack of recognition for such activities.  One thing is for sure, that most data will not have the same exposure to the science community as the research findings it supported. 

The long term life expectancy of such data is also likely to be shorter than its wordy cousin the research paper.  In fact it may not even exist after its collector has moved posts or retired or suffered a serious IT problem. It is very unlikely to be archived as well as the printed word and its creator is much less likely to have received credit for collecting it and the institution she/he works for is unlikely to receive improved research ratings or extra funding for generating it. 

Now imagine a word where data is King (or at least Queen alongside the research paper King) and research funding and University Chairs are partially reliant on the proper publication of peer reviewed data sets. I think that a fully functional academic SDI with all the bells and whistles that you could want would be a reality within the next decade. In the same way that JANET has, and continues to develop in the UK with its fast speeds and links to other country networks so would the SDI. 

Obviously this isn’t an original idea, there are a few journals dedicated only to data publication and there are strong policy statements all the way from Government through the publicly funded research councils and even to a few Universities that make the publication and accessibility of data a priority. What is missing are real incentives for researchers to treat data in the same way as research findings and until this gulf is filled then data will be the poor relation of the academic publication World. If the translation of the INSPIRE (see my previous post)   directive into European Governments’ actions includes Universities then maybe it will provide some significant infill for this gulf and move the discussion from “why should I?” to “how do I?”

Background

 Posted by at 12:29 asides Tagged with: ,  Comments Off on Publish Data or be Dammed
Aug 012011
 

I have been reading and  thinking about the relationship between long term spatial data preservation and the short term needs of day-to-day data security during the life of a research project. With research data being generated at faster and faster rates and the life cycles of supporting technologies getting shorter data preservation is destined to be a continual problem requiring new and smarter solutions every few years. Just dealing with new data that can be produced by Earth observation satellites at the volume of terabytes per day and may exist in several formats as is passes through complex processing stages is enough to take the issue  well into the scope of being a serious problem.  The sentiment expressed by Moss points to the aspiration of researchers for their hard won data.

“Scientists now want to keep everything, which they assume is digitally possible, in the belief that everything has value and can be retrieved and repurposed.”  Michael Moss 2008

The question is; will the technology and the resources exist to meet this aspiration?

It is very easy for a researcher in a Higher Educational institution to secure data in the short term via either their own arrangements or by using the services of a central IT department. In that way data can be backed-up to multiple locations and held on hardware that is up-to-date and covered by manufacturer’s warranties. I am sure this professional approach is found in most (or all) HE institutions. It’s still up to individuals to avail themselves of these services but there are few obstacles standing in the way. Even storage costs are falling and for a few £s per gigabyte a university department can store data in professionally managed institutional servers.  

In the hierarchy of data preservation the next levels up become harder for a researcher to arrange.  Consider keeping spatial data for third parties to discover and use for the next 5 years. Immediately there is the need for precise and comprehensive metadata. This has been addressed in several ways and the development of specific standards via UK AGMAP has given, anybody who looks, an easy lead into useful metadata creation. For this longer term data storage it may also be necessary to look outside of your home institution to ensure suitable data curation and discoverability.  But where do you put the data? It needs an accessible location that links metadata and the data object and makes them discoverable by future researchers.  The tools provided by GoGeo provide a solution. Data can be described and even lodged within this service so it becomes searchable and accessible to other researchers. 

So there is an infrastructure for this stage of the data management process but it now needs the data producer to step outside of their daily routine and to work on tasks not always considered core for a busy academic looking to their next paper.  This 5 year time horizon is also significant in that the European INSPIRE Directive will be in force   for its Annex III type data by2013. This means that many university generated geospatial data sets will need to comply with the INSPIRE standards promoting interoperability across boundaries. Possibly more difficult to achieve will be dealing with older data which will also have to meet INSPIRE standards by the next decade.

Once we look beyond the next few years and start to focus on spatial data of high quality or significance things get really interesting and much more challenging.   It’s very easy to talk of data archiving and curation as if there are standard easily accessed facilities in every library. The more I have read the more I realised that it’s a far more fluid and developing science than I appreciated. 

Who decides which data are in need of professional curation, or which data can we afford to curate?  These kinds of questions move the process beyond the researcher into the realms of professional librarians or data curators and government departments working to budgets and polices.  All this leads to a further stream of questions: Can data be given to one institution to look after? Can we guarantee that any institution will be a permanent fixture?  Will the metadata that was created during data collection still have sufficient context to be useful in 10, 50 or 100 years time? How will the increasing number of data objects be kept searchable and accessible?  With hardware life cycles only being a few years, who will ensure the passing on of data to the next technology and will that technology still support the data format? These questions just start to scratch the surface of the issues involved in designing future data curation methods and policies.

Let’s hope that the situation described in the quote below won’t be applied to the early 21st century when looking back in 20 years time.

“In terms of preserving our digital cartographic heritage, the last quarter of the 20th century has some similarities to the dark ages. In many cases, only fragments or written descriptions of the digital maps exist. In other cases, the original data have disappeared or can no longer be accessed due to changes in technical procedures and tools.”Markus Jobst 2010

It’s possible to take this timeline one stage further and start to consider which spatial data sets, that are so important to major scientific discoveries or advancements, should be considered for preservation in  the equivalent of a scientific museum that holds the essential heritage of our scientific community.

Now where did I save that first human genome I was given for safe keeping in 2003? Are well not to worry it wasent geospatial data anyway, well not unless the DNA doner had an adress?! Oh and it was a mapping project so I better fnd it…….

 Posted by at 15:18 asides Tagged with: , , , ,  Comments Off on Don’t get your Back-Up over Data Preservation
Jul 202011
 

Alisdair is a PhD case study for the IGIBS project. He is 8 months into a PhD that is developing a tool to interpret almost ant kind of imagery into part of a time line for mapping habitat change.  You can view his university profile here.  He says that the key descriptors for this work are  “universal” and “precise”. So he is aiming create a tool that can be applied to images from Landsat , IRS  and SPOT for example and from these data  he is trying to draw out both changes in extent of habitats since 1975 and in timings of changes within a season. So he might be able to look at loss of semi-natural woodland through felling  and to identify changes in the timings of tree bud burst that might relate to climate change. 

In order to make  data created at different times precisely comparable he needs to perform some complex processing of imagery and to  filter out variable factors that distort the images, such as atmospheric quality.  This means that his data requirements for the PhD are very wide ranging and have covered satellite and aerial data from the past 35 years detailing ground cover, atmospheric quality  and many things in between. 

There are also issues around the accessibility of data that will be worth exploring for the IGIBS project as some of his source data is commercially protected but much of his output and analysis might be  more freely available. So some data will only be  for University use while other parts of his work will be more accessible.  

While the ultimate aim is to provide change maps for the whole of Wales, the pilot study is covering Borth Bog (Cors Fochno in Welsh) in the Dyfi Biosphere and so will provide a suitable case study for IGIBS. 

In particular this case study will highlight some major areas of the IGIBS project.

  1. His need for such a wide range of geospatial data  from differing  sources could  feed into recommendations for metadata and data discovery aspects as well as security issues and intellectual property rights.
  2. He will be producing maps of habitat change that could be useful to landscape researchers, WAG agencies, and many other categories of data users and so he is attempting to   establish a link with the National Library of Wales so that his research can establish a  resource for future use.
 Posted by at 11:59 phd, User Reqs Tagged with: , , , , , , ,  Comments Off on Alisdair Cunningham – IGES PhD student
Jul 132011
 

A quick hello from your newest member (Steve Walsh) I now have my feet under the desk at Aberystwyth and look forward to working with you all. I will try and keep up regular blogs on developments….

I had a meeting with Jonathan Brownett yesterday and got some very useful insights into the use of spatial data from the coal face …….Below is a brief summary of Jonathan’s section of the project’s case study.

Jonathan Brownett – IGES MSc student 

Project 

 Jonathan is looking at land cover change in the Dyfi Catchment over an 8 year period using a mixture of data from LANDSAT 7 and field parcel data (LPIS) from the Welsh Government. He has also made use of the Digimap service to access OS MasterMap data and elevation and shape file data accessed from within from IGES. 

Jonathan is then relating this data to the relatively new Land Cover Classification System (LCCS) which uses pre defined classifiers that can be applied to a level of detail suitable for a particular project. With this method he should be able to track changes in 20 or so categories of vegetation, bare land and water covered areas. 

Combined with some confirmation of actual ground cover species Jonathan should then be able to describe some of the major land cover changes  that have happened in the Dyfi catchment (very similar in extent to the Dyfi Biosphere area). 

He is well underway with his work and should have his data analysed buy the end of July and should receive some expert help in using LCCS from Lucile who will be visiting the Department from France and will be bringing her expertise in carrying out LCCS ground survey to the Dyfi Biosphere area. 

Relevant experience for the IGIBS case study 

In several ways Jonathan is a very useful case study as he has recently come from a Conservation BSc and has experienced GIS and Remote Sensing courses and the functioning of an SDI in two academic departments. With his MSc thesis he is now   discovering the intricacies of accessing data from external agencies and discovering the wealth of data held by individual researchers with their own approaches to metadata creation. He is also being forward thinking and has suggested that some of his data might be suitable to use for undergraduate teaching so suggested that he will discuss the best way to make available the research data he has generated with his supervisors.

Jun 042011
 

One of the many goals of IGIBS is to generate Web Map Services that will be used in conjunction with INSPIRE type View Services which themselves are compliant with the INSPIRE Technical Guidance for View Services version 3.0. To that end, it made sense to take the following basic INSPIRE criteria into consideration when making our choice of tools:

  1. Support for the LANGUAGE request parameter in a GetCapabilities Request.
  2. Support for “extended attributes” including elements extending the
    _ExtendedCapabilities substitution group of the WMS 1.3.0 schema with a custom
    xsd.
  3. Support for the optional WMS 1.3.0 parameters wms:identifier, wms:AuthorityUrl and wms:LayerLimit

Up till ~3 weeks ago (May 12th) no stable release of either geoserver or mapserver satisfied any of the above criteria.

Mapserver Customisation

In order to make IGIBS services INSPIRE compliant we are using a customised version of mapserver 5.6.6. The customizations involve backporting selected features from the development tree of version 6.0 plus our own additions to add support for the LANGUAGE parameter and the extended attributes in the GetCapabilites response. The code is available for perusal here for any interested parties. It comprises a patch against mapserver 5.6.6 plus a sample mapscript wrapper that can be run as a cgi to provide an INSPIRE compliant View Service. Since Mapserver 6.0 the patch should no longer be necessary, but the mapscript wrapper is still required.

Latest Developments

On May 12, 2011 mapserver released version 6.0 and geoserver released version 2.1.0. As part of that release, Geoserver got funding from the Ordnance Survey to add support for the aforementioned INSPIRE spec as a plugin and can now satisfy all of the above criteria, while mapserver only got support for the wms:Layerlimit attribute.

Conclusion

The choice of software depends on one’s requirements. For a national mapping agency seeking INSPIRE compliance it seems that geoserver 2.1.0 is currently the best route. For the purpose of IGIBS, we will stick to the modified mapserver 5.6.6 for the following reasons:

  • Speed. Mapserver has performed considerably faster in our tests involving rendering and reprojection of geospatial data, which is crucial for the dynamically generated services of IGIBS.
  • Flexibility. Mapserver can be very easily scripted in a high level language for prototyping and experimentation.
  • Tried and trusted modifications to ensure compatibility while still being flexible enough to follow the fluid INSPIRE specs.
  • Geoserver does not yet fully support all parts of the INSPIRE TG e.g. the  “scenario 2” mentioned in the standard.

Please feel free to submit any comments.

 

Jun 032011
 

The overall aim of the IGIBS project is to try and improve the relationship between the UK’s National Spatial Data Infrastructure (SDI) as manifested through the UK Location Programme (UKLP) and the UK’s academic SDI.

Our main objective is to focus on use cases emerging from research and education related to a particular area – the UNESCO designated Dyfi Biosphere Reserve.  Once articulated, these user requirements will drive the creation of two pieces of software of wider applicability and assist Aberystwyth University in developing resources for use by local students.

We are building on much prior art, especially in the area of Access Control.  EDINA runs the UK Access Management Federation (UKAMF) and, while it might not be fashionable, the reality is that many SDI resources, eg, data and web services, are going to stay protected.  This is true both of INSPIRE at the European scale and the UKLP nationally.  We aim to show how Shibboleth (the open source software that underpins the UKAMF) can be used to enable a wider range of use cases, so that UK students can get access to both open and protected resources, eg, from UK public authorities like Welsh Government.

We expect that the main four products resulting from this project will be:

  1. Working prototype of a “WMS factory” tool
  2. Simple mapping application
  3. Best Practice model for using UK academic SDI at the departmental level
  4. Demonstration of UK access management technology being used to secure public sector services in combination with academic sector services

SDI is underpinned by open geospatial standards like the OGC’s Web Map Service (WMS).  The “WMS factory” tool will allow users to upload their data and instantiate a WMS so that their data can then be viewed online, via a simple mapping application, in conjunction with reference data from Welsh Government.

Shibboleth is already used in academia, we extend its use here to demonstrate how public sector data can be made securely available to authenticated and authorised users within academia.

The Institute of Geography and Earth Sciences (IGES) has ambitions to improve the way it educates students in the use of open geospatial interoperability standards and intends using the Dyfi Biosphere Reserve area as an exemplar.  To this end we are conducting an inventory of data for the area and creating a repository for educational use.  The “Best Practice model for using UK academic SDI at the departmental level” will feed into this activity as well as provide guidance for the wider university sector.

May 232011
 

Last Wednesday (18th May) I attended a meeting of the Dyfi Biosphere Research Forum at Aberystwyth University.  Chaired by Mike Woods of the Institute of Geography & Earth Sciences, the meeting showcased a selection of the wide range of different research activities that have taken place (or are underway) in the Dyfi Biosphere area.

  • Chris Lea (Welsh Assembly Government) gave the keynote stressing WAGs support for R&D in the area, particularly as it relates to the work of the Sustainability and Environmental Evidence Division
  • Palma Blonda (CNR, Italy) and Richard Lucas (Aberystwyth University) gave an overview of the EU funded BIOdiversity multi-source monitoring system: from Space TO Species (BIO-SOS) project
  • Mike Bailey (CCW): dipped into what is obviously a deep mine of personal knowledge of the area and presented on recent research in the core conservation zone of the Dyfi Biosphere
  • Mike Hambrey (Aberystwyth University): gave a fascinating presentation on the last glaciation in Wales with a focus on the Dyfi catchment
  • Paul Brewer (Aberystwyth University) outlined some surprising results in a presentation entitled “Altered morphodynamics in the tidally-influenced lower Dyfi: re-thinking catchment management, flood risk & material fluxes”
  • Yours truly (Chris Higgins) presented on IGIBS (IGIBS_BDB_Research_Forum_May11) emphaising that the project needs input, eg, via the questionairre from this group if it is to benefit Dyfi related R&D and bring further resources into the area
  • Mike Christie (Aberystwyth University) gave a very erudite presentation on a hot subject that I think we will be hearing a lot more about: Valuing ecosystem services
  • Ambra Burls (Independent researcher) and her guest Zena Willmot from Coed Lleol talked about very pertinent related work – Environment and health
  • Man (apologies, didnt catch the name and not on the programme) from Centre for Alternative Technology talked about the groundbreaking work CAT is doing in relation to “Building adaptation for climate change”
  • Finally, Michael Woods (Aberystwyth University) spoke on “The Dyfi biosphere in context: research from the Wales Rural Observatory.”  Again, very apposite, some very surprising insights and all the more interesting as coming at the question of Biosphere related research from a different angle

We had 20-30 minute at the end for a brief plenary discussion.  Lots of good ideas here but no time to go into any of them in any real depth. At the risk of pre-empting the minutes, IMHO, a couple of themes started to emerge:

  • Need to broaden the scope of the Research Forum to include greater representation from the social sciences
  • Must bear in mind the need to engage with the wider community
  • Terms of reference need nailed down
  • Lack of resourcing a problem

Of course, the IGIBS project has some resources (at least up until November 2011) and, as project manager, I volunteered that the project team would do what we can to help as long as it aligns with the project objectives.

The one thing we can help with is laying firm foundations for getting the most out of online Geographic Information and managing Dyfi Biosphere research related data.  Our hope is that the work we do in this short project helps you and others like you inititiate and execute future projects in the area.  To lay this foundation well, we need input from stakeholders in the Dyfi Biosphere and from the research forum in particular.  Please fill in the questionnaire, even if its only a partial response.

 Posted by at 20:13 Project Management, User Reqs Tagged with: , , , , , , , , , , ,  Comments Off on Biosffer Dyfi Biosphere Research Forum meeting
Apr 162011
 

This is a big topic that is often neglected and often proves fatal for SDI initiatives meeting their ambitions.  The reality is that much valuable data is restricted; this is true for the UK National SDI (UK Location Programme), the UK academic SDI and INSPIRE.  A genuinely interoperable means of allowing OGC Web Service clients (like the one we are developing in IGIBS) to consume WMS (open and restricted) from multiple distributed organisations without having to provide multiple user credentials is difficult.  Building on much prior work, we are going to try to demonstrate in IGIBS how Shibboleth – the open source SAML implementation that powers the UK Access Management Federation – can be used to allow protected public sector WMS can be made securely available to the academic sector.  We will also demonstrate the converse, how users in the academic sector can securely publish their data and control who can see it if they need to.  We will use this page to solicit comment and hopefully gather recommendations for further work.

 Posted by at 15:33 Security Tagged with: , , , , , , ,  Comments Off on Security Category