This document details the software architecture of the Bernstein Atlas service and the elements required to make it function. The homepage of the Bernstien GIS services is here and the prototype Atlas here in ArcGIS 9.3 JavaScript API version and here in ArcGIS WebServer 9.2 version.
Some information on this page might be outdated. Please refere to this page for the latest version.
The following figure provides a graphical summary of the architecture of Bernstein geographical service.
Extended view:
Compact view:
Databases - The databases contain georeference information to the papers and watermarks records. They are configured to interact with the Workspace.
Workspace - In the Workspace users construct requests on the data available in the databases. The Workspace software forwards the requests to the databases and displays the output. It also access georeference information, prepares and manage it for use by the Atlas.
Atlas data - There are three types of data that the Atlas consumes: dynamic data originating in databases as provided by the Workspace search engine, static data generated by project Bernstein but not part of the integrated databases, data from the Web.
Atlas interface - The Atlas interface displays maps of the Atlas data and gives users tools to interact with it. It is driven by webbrowsers and consists in html layouts, css stylesheets and JavaScript scripts.
GIS server - The GIS server generates maps to display in the webbrowsers based on the Atals data and users settings.
Third parties - The Bernstein Atlas has the ability to use geographical information generated by entities outside the project itself. Also, third parties can in turn use in their own mapping services datasets originating in the project.
The Atlas service can be accessed - from the search interface of the Workspace - from the Atlas page itself - from outside the Bernstein project
In the first scenario - search interface to Atlas - the user is initially on the Bernstein search interface of the Workspace, constructs a search request, the Workspace software generates the reply and provides a link to the Atlas. The user clicks on the link and a new webpage is generated where he can see a geographical mapping of the results. There he also finds tools to modify the map or add new maps.
In the second scenario - Atlas to search interface - the user can select datasets for mapping among those provided by the integrated databases, those outside them or those of third parties on the Web. If he chooses the integrated databases datasets, a link will bring him on the search interface, where he constructs a query after which a link will bring him back to the Atlas page where the results are mapped.
In the third scenario - third party to Bernstein - the user can either map Bernstein static datasets by simply selecting them in the list provided by the service he uses, or be linked to the Bernstein Workspace search interface to construct a query, then prompted to go back to the initial service to map the results.
For each request the search interface software generates a unique set of files containing the results. A subset of these provides the data processed for consumption by the Atlas. The location of the geographical file is made aware to the Atlas software by embedding it in the link on which the user will click to access the Atlas.
Dynamic link encoding
Lets take a look at the link to be encoded in the search interface webpage once the response to the user's request is ready.
<a name="BernsteinAtlasLink" href="http://www.viskom.oeaw.ac.at/~vlad/atlas/online/ atlas/atlas.html?url=http://www.viskom.oeaw.ac.at/~vlad/atlas/online/atlas/data/static/ geo_default.jsonp" target="_blank">Show Map</a> |
You see that the link contains the URL http://www.viskom.oeaw.ac.at/~vlad/atlas/online/atlas/atlas.html - this is the Bernstein Atlas webpage (the URL given here is not the final one, just that of the development phase of the project). After a question mark follows the URL of the geographical file produced by the Workspace search interface. Several files can be referenced separated with 'and' symbols &. These locations have to be accessible on the Web. More about the file itself in the Datasets section further down.
Note that target="_blank" will make the Atlas appear in a new page. It enhances the 'real estate' where the map will be deployed and separates the Atlas software from that of the Workspace for easiness of development and portability.
Because the link is referenced by a name - name="BernsteinAtlasLink" - it makes it possible to third parties to consume in their own environments the geographical results of a search done with the Bernstein interface.
A similar link can be embeded in a button.
<form name="BernsteinAtlasLink" action="http://www.viskom.oeaw.ac.at/~vlad/atlas/online/ atlas/atlas.html?url=http://www.viskom.oeaw.ac.at/~vlad/atlas/online/atlas/data/static/ geo_default.jsonp" method="post"> <input type="submit" value=" Show results "> </form> |
File naming
There is no requirement set on the naming of geographical files. The following format is however suggested.
Format: geo_<unique file id>.json
Example: geo_wxcvqsdf.jsonp
The prefix geo indicates a geographical content; the id keeps the file distinct from others at the same location; the extension jsonp specifies the content as JSONP formatted, which is the standard used by the Atlas software for information interchange.
Storing location
No requirement other than the location should be readable form the Web. A convenient location to store all datasets intended for consumption by the Atlas is the directory <atlas software location>/data/dynamic/, where <atlas software location> is the URL of wherever the Bernstein Atlas will be stored. Presently this is http://www.viskom.oeaw.ac.at/~vlad/atlas/online/atlas/.
Cache cleaning
With every request made to the search interface resulting in maps, the geographical files produced by the Workspace accumulate. A mechanism has to be put in place to clean the files that are deemed not to be used by users anymore. Time lapses seem appropriate to the kind of website that is Bernstein. Files could deleted 24 hours after their creation.
When the Atlas webpage is uploaded in a webbrowser the URL content of the link on which the user has clicked to arrive at the Atlas is forwarded to the webbrowser. The Atlas software reads it and extracts from it the location of the file containing the georeferences data to map. This information exchange is sufficient to the Atlas software to start provide Bernstein users with GIS services. URLs to datasets of third parties are hard coded in the JavaScripts of the Atlas software. The files themselves are read as JavaScript files, who's URL is dynamically written in the HTML after reading it from the URL of the HTML. (Note: Local files - but not cross-server files - could be read via XMLHttpRequest.)
Mapping service by Bernstein is based on ArcGIS products ArcGIS Desktop, Server and JavaScript API. While waiting to upgrade from version 9.2 to 9.3, released during summer 2008, the ESRI company's free server services are used.
Prior to the release of the JavaScript API we experimented with the WebSever 9.2. The results presented here show that:
If a dynamically file generated by the Bernstein search interface wants to be used by third parties its location can be automatically found with a JavaScript code that scans the search webpage for the name of the link - BernsteinAtlasLink - and extracts the URL from the content of the link. A demonstration page shows the code in action.
The URLs and description of the static datasets served by Bernstein are given on the Atlas page.
In short communication between elements of the geographical service is happening by having the Workspace passing to the Atlas the locations of geographical files through links embedded in the search webpage and the URLs they contain. Third parties get the same information by parsing the search webpage for these links tagged with know names.
Follow the requirements set on the dataset files.
padding - Needed to make the content readable to the JavaScript program. The padding enclosing the JSON content must be "jsondata( [... JSON data ...] )".
header - Gives information about the file content and media. The terminology follows the recommendation of the Dublin Core Metadata Initiative (v2008.01.14). This header is reusable for of the search results file that can be downloaded by Workspace users.
items - Contains information about each location. With the exception of the 'records' field these are extracted from the data providers. 'Records' is computed by the Workspace.
Example - source: http://www.viskom.oeaw.ac.at/~vlad/atlas/online/atlas/data/static/geo_default.jsonp
jsondata( {"header": { "content": { "title": "The members of Project Bernstein: The Memory of Paper", "abstract": "The list of Project Bernstein members with their respective contributions.", "subject": "history, paper studies, watermarks, georeferences", "source": "Project Bernstein: The Memory of Paper", "creator": "Project Bernstein: The Memory of Paper", "publisher": "Project Bernstein: The Memory of Paper, http://www.memoryofpaper.eu", "created": "2008.11.11 12:59:59", "rights": "Project Bernstein: The Memory of Paper, http://www.memoryofpaper.eu" }, "criteria": { "databases": "", "hits": "", "term": "", "motif": "", "dateStart": "", "dateEnd": "", "depository": "", "height": "", "heightTolerance": "", "measurementUnit": "", "chainLines": "", "chainLinesTolerance": "", "laidLines": "", "laidLinesTolerance": "", "laidLinesUnit": "", "useplace": "", "useplaceStatus": "" }, "spatial": { "coordinates": "decimal", "nomenclature": "NUTS 2003" }, "media": { "encoding": "UTF-8", "format": "JSON" }, "extra": "" }, "items": [ { "status": "locality", "placename": "Stuttgart", "latitude": "48.77", "longitude": "9.18", "regionId": "DE111", "country": "Deutschland", "region1": "Baden-Württemberg", "region2": "Stuttgart", "region3": "Stuttgart, Stadtkreis", "region4": "", "records": "90000", "extra": "<a href='http://www.piccard-online.de/'>Archives of the State of Baden-Wuerttemberg</a>: data provider" }, { "status": "locality", "placename": "Den Haag", "latitude": "52.08", "longitude": "4.30", "regionId": "NL332", "country": "Nederland", "region1": "West-Nederland", "region2": "Zuid-Holland", "region3": "Agglomeratie ’s-Gravenhage", "region4": "", "records": "16000", "extra": "<a href='http://www.kb.nl/'>Koninklijke Bibliotheek</a>: data provider" }, { "status": "locality", "placename": "Wien", "latitude": "48.21", "longitude": "16.37", "regionId": "AT130", "country": "Österreich", "region1": "Ostösterreich", "region2": "Wien", "region3": "Wien", "region4": "", "records": "8000", "extra": "<a href='http://www.oeaw.ac.at/'>Austrian Academy of Sciences</a> (<a href='http://www.viskom.oeaw.ac.at/'>VISKOM</a>, <a href='http://www.ksbm.oeaw.ac.at/'>KSBM</a>): coordinator, data provider, GIS, image processing" }, { "status": "locality", "placename": "Firenze", "latitude": "43.77", "longitude": "11.25", "regionId": "ITE14", "country": "Italia", "region1": "Centro (I)", "region2": "Toscana", "region3": "Firenze", "region4": "", "records": "4000", "extra": "<a href='http://www.iuoart.org/'>Dutch University Institute for Art History Florence</a>: data provider" }, { "status": "locality", "placename": "Leipzig", "latitude": "51.30", "longitude": "12.33", "regionId": "DED31", "country": "Deutschland", "region1": "Sachsen", "region2": "Leipzig", "region3": "Leipzig, Kreisfreie Stadt", "region4": "", "records": "", "extra": "<a href='http://www.ddb.de/'>German National Library</a>: bibliographical data provider" }, { "status": "locality", "placename": "Paris", "latitude": "48.87", "longitude": "2.33", "regionId": "FR101", "country": "France", "region1": "Île-de-France", "region2": "Île-de-France", "region3": "Paris", "region4": "", "records": "", "extra": "<a href='http://lamop.univ-paris1.fr/W3/'>National Center for Scientific Research / University Paris 1</a>: GIS, contextual data provider" }, { "status": "locality", "placename": "Graz", "latitude": "47.07", "longitude": "15.45", "regionId": "AT221", "country": "Österreich", "region1": "Südösterreich", "region2": "Steiermark", "region3": "Graz", "region4": "", "records": "", "extra": "<a href='http://www.tugraz.at/'>Graz University of Technology</a>: integration" }, { "status": "locality", "placename": "Liverpool", "latitude": "53.42", "longitude": "-3.00", "regionId": "UKD52", "country": "United Kingdom", "region1": "North West", "region2": "Merseyside", "region3": "Liverpool", "region4": "", "records": "", "extra": "<a href='http://www.liv.ac.uk/'>University of Liverpool</a>: integration" }, { "status": "locality", "placename": "Delft", "latitude": "52.00", "longitude": "4.37", "regionId": "NL333", "country": "Nederland", "region1": "West-Nederland", "region2": "Zuid-Holland", "region3": "Delft en Westland", "region4": "", "records": "", "extra": "<a href='http://ict.ewi.tudelft.nl/'>Delft University of Technology</a>: image processing" } ]}) |
The following lists the requirements set by the Bernstein geographical service on the databases, Workspace, Atlas and third parties.
The databases owners need to:
The Workspace developers need to:
The next steps in developing the Atlas service and the known bugs are detailed here.
To consume static datasets thrid parties have to:
To consume dynamic data generated by the Workspace as a result of a search on the integrated databases:
Relying on the region codes instead of region names for making request to the databases, searching teh databases and ultimately transferring the results back to the workspace is a good idea with the potential to get the results quicker to the user. However it has a fundamental impediment. Lets see first the details of the scenario, the analyze the flaw.
Idea - The idea is to look up the codes of the region names selected by the user and query records from the databases with the region codes instead of the region names. In turn the databases will send back only the region codes and the Workspace will reconstruct the region names from the codes using the lookup table. The NUTS nomenclature table is found here.
Drawback - The use of region codes relies on the assumption that the length of the code is proportional to the degree of detail specified by the NUTS (see next paragraph). However it was not possible to find the codes for all location, if they exist at all. While Europe is fairly well described by NUTS, most non-European states are referenced in the georeferences of Bernstein by only the code of the country, even if the names for lower region level is provided. This leads to such a case as a code 'US' from which it is impossible to know if the location 'Washington' is in the Washington state or the District of Columbia. Coverage of non-European states is essential to Bernstein: many records on paper use reference locations in Eastern Europe, an important amount of repositories are outside Europe (especially in WILC and the additional datasets of ISTC and GW), and a substantial proportion of the bibliography locations are scattered worldwide.
Solution - A solution would be that if the code is only two-digits long, referencing only the country, then information is passed from the Workspace to the databases with the full name of the region instead of just the code. The databases have to implement a software procedure to deal with this modality. Is this software complication worth the trouble and does it speed data processing?
Code structure - Note the formation of the codes: two-digits country code + one digit nuts1 name + one digit nuts2 name + one digit nuts3 name + one digit nuts4 name. For example DEA21 stands for 'Deutschland > Nordrhein-Westfalen > Köln > Aachen, Kreisfreie Stadt'. Since the region is defined only down to the NUTS3 level, the code is also only five digits long. An alternative variant to note the level of detail is to use zeros for not defined NUTS levels. 'Deutschland' can be thus rendered as 'DE' or 'DE000'. Software developers should be aware of this variation.
NUTS evolution - The NUTS is an evolving entity, changing every couple of years according to changes in the territorial organization of countries or demographic modification (since NUTS are essentially means for uniform demographic statistics). For Bernstein we choose the NUTS nomenclature of year 2003, the lasts for which polygon coordinates defining the regions contours can be provided (they are included in ArcGIS). Polygons allow to select data without the need to know the names of the regions that are to be selected. Data selection geoprocessing are easier, more flexible and more powerful.
Bernstein Atlas
Bernstein georeferences
NUTS
JSON, JSONP, XMLHttpRequest, Dublin Core
ArcGIS
-- VladAtanasiu - 15 Nov 2008
I | Attachment | Action | Size | Date | Who | Comment |
---|---|---|---|---|---|---|
![]() | BernsteinGisArchitecture.gif | manage | 31.5 K | 16 Nov 2008 - 02:12 | VladAtanasiu | gis architecture - picture v1 (handdrawn) |
![]() | atlas-architecture.png | manage | 35.1 K | 28 Jul 2009 - 16:59 | VladAtanasiu | gis architecture - picture v2 (digital draw) |
COMMONS
WORKPACKAGES
*
TOOLS SITE INFO |
Copyright © by the contributing authors. Bernstein - The Memory of Paper http://www.bernstein.oeaw.ac.at Ideas, requests, problems regarding Bernstein? Send feedback | ![]() |