Who's Online

We have 4 guests online

Login Form






Lost Password?
Search & Index

At the conception of the ENAWU project, how to search across multiple websites, with different types of data - either online, and in some cases with secure user/membership constraints was a challenge that was likely to require extremely heavy programming or considerable expenditure. However, the solution to the first challenge - a text or context oriented cross-platform search - was close at hand.


Search

Our IT development team carried out considerable research into free, open source and proprietary solutions (running into thousands of dollars per year) that would be capable of:

  • Simultaneously searching 10, 20, 50 or 100 specified websites.

  • Crawling deeply into online databases with thousands of articles and texts, each categorised, indexed or identified in different ways.

  • Allow us to easily add or remove websites and servers.

  • Have low maintenance and administration requirements (we are a small team!)

Eventually, we decided on the Google Custom Search application - despite the inherent risks and cynical criticisms of Google's overall impact on our internet experience these days.  

 

We appreciate and respect that - for some of our partners - the Google solution appears too simple.  However, while we continue to attempt to address that - there were several reasons for our choice:

  • The flexibility of Google to handle / crawl / return so many different data types, servers, repositories (provided they are online and the search has the required user names and passwords programmed in to give it access).

  • The simplicity of integrating the search engine into the ENAWU website (or indeed any other website)

  • For many of the users that we want to attract to ENAWU - in particular the generations of web-savvy students, researchers, journalists - the Google type search capability is well known, simple yet powerful and effective.

And this leads us to the more complicated and challenging requirement - both for our own organisations (Arab West Report, CIDT, CAWU) and for several others - building not only an indexing system, but then using that indexing system to give ENAWU users an advanced, index (topic) based search capability.

Indexing & Classification

As we have mentioned in the section about the Arab West Report index , there are many ways that data is and can be classified and indexed these days. Indeed, in addition to the traditional systems such as the Library of Congress Classification (LCC) and Dewey Decimal Classification (DDC) (which have been adapted over the years from well established bibliographical systems for use in computer applications) other systems have been designed with a different focus - like making data information (such as meta data) more easily accessible to a variety of data reading / mining / searching / managing applications - the MARC Standards are a good example of this.

At the moment - thanks to the funding of ENAWU Phase 1 - finding and implementing a suitable index for ENAWU is being researched and developed around our own custom index (which has been developed over 10 years in order to make our own unique archive of more than 20,000 texts accessible) and at the same time mapping this index to both the LCC and DDC classification systems.  Below is a snapshot of the AWR index as it is today (click on the image to see it in the context of the website).  This is not yet complete, but we are very happy with the progress being made.

AWR index

The other challenge associated with developing the index is to ensure that all of the data is properly indexed, mapped, cross referenced, uses the same names, translations, etc.  So in addition to our own work we are also exploring specialist service providers who may be able to convert, cross-map and 'clean up' our existing data archive - itself a massive task. 

In ENAWU Phase 2 we are looking to do two things with respect to the index:

  • First, see how this index can be expanded to encompass the data archives of other partners - not as a requirement but as a collaborative venture to establish an 'ENAWU Index' (thats not the official name by the way!).  We envisage an indexing system that will allow participating partners to add their specialist categories into the index.

  • And second, find a way to use the index as the core of an advanced search capability - search across the ENAWU partner sites, but a system that is constantly crawling and updating the index with new categories and content uploaded by our partners.

 

 

 

 
< Prev   Next >