Usability Survey 2010 – Results
In September, ecoi.net conducted a usability survey. Click on “read more” to see how our users responded.
ecoi.net is used around the world, users in 50 countries filled out the questionnaire (greetings to New Zealand!). The largest share of respondents (19%) work in Austria, followed by Germany, Belgium, The Netherlands & Switzerland. The Top 10 are completed by Ireland, UK, Canada, and (ranking equally) Nigeria, Spain & Sweden.
Since when do you use ecoi.net?
33% of the respondents use ecoi.net since before 2006, 17% startet using it in 2006 or 2007, and 37% from 2008-2010. 11% visited ecoi.net for the first time when they filled out the questionnaire.
Frequency of use
13% visit ecoi.net less frequently than once a month, not including the 11% of first-time users mentioned above. 39% use it once or a couple of times a month, and 37% use ecoi.net from once a week to daily.
Profession
44% of respondents are working for a government authority dealing with refugee status determination. Another 2% are working for other government departments (not involved in asylum procedures) and 2,5% are judges or other staff working for appeal courts.
24% are working in refugee assisting NGOs, 12% are refugee lawyers or legal counsels, and 7% work for an international organization.
User satisfaction
The vast majority of respondents (about 90%) is satisfied with usability, navigation, the search function, the currency and the sources covered – variations between these aspects can be seen by comparing those users who were „very satisfied“ to those who were „rather satisfied“. See the following figure for details.
Usage frequency of various features
We asked our users how often they use various features of an ecoi.net country page, namely:
- the list of newly uploaded documents,
- the list of important documents,
- links to country profiles,
- maps,
- national laws,
- country-specific links,
- the search form,
- and the topics & issues files.
Of these features, the list of new documents and the search form are the most frequently used, national laws and topics&issues files are the least frequently used. Considering the updating of topics&issues files has been stopped in November 2008, it is not surprising they are not accessed often anymore. Regarding the lesser usage of maps and laws, we will try to improve the visibility of these features.
Ease of use
We asked our users how easy it is for them to:
- access a country page
- identify the publication date of an entry
- identify the source/publisher of an entry
- identify the original title of an entry
More than 95% of respondents stated it is „very“ or „rather“ easy for all of the processes mentioned above. No respondent stated any of the processes is „very difficult“.
Differentiating between those who selected „very easy“ and those for whom it was „rather easy“ shows that accessing a country page is the easiest, recognising the original titles the least easy process.
95% of respondents stated the brief summaries ecoi.net offers for every entry are “very” or “rather” helpful for assessing the relevance of an entry.
Source descriptions
While 21% of respondents rated ecoi.net’s source descriptions with “very good” and 43% with “good”, a large share (22%) stated they did not know these descriptions existed. We will try to improve the visibility of our source descriptions.
Search function
Respondents rated their satisfaction with the search results:
- 23% very satisfied
- 72% rather satisfied
- 4% rather not satisfied
- 1% not satisfied
Please see the following figures on what functions of the search engine respondents use:
We asked if the assistance provided by the COI Thesaurus (automatic inclusions of synonyms and near-synonyms, as well as the moderated search) is regarded as helpful.
42% of the respondents “fully agreed”, 30% “mildly agreed”, while 12% “tended to disagree” and 16% did not know about this feature.
Personalisation
56% of respondents registered an account on ecoi.net. Of those respondents who did NOT register, 88% stated they didn’t do so because they did not know registration offered additional features and 12% do not find the features useful for them.
The following figure shows what features registered users stated to use:
As ecoi.net is also the basis for the Austrian Federal Asylum Agency’s country documentation database, providing additional documents to users registered at the Agency, at least a share of the 16% of registered users who never used personalisation features can be explained.
As a result of this survey and the usability study conducted at the same time, we want to make the advantages of registration easier to understand. Therefore, we will consider the option of a “stay signed-in” function, and we will remove the least use feature, “my documents”. The usability study showed this feature was difficult to understand and that it was difficult to explain the difference between “my documents” and the “research baskets”.
Consequences
When the survey was online, we conducted a usability study in parallel. On the basis of the feedback we received through both the survey and the study, we will envisage the following changes:
- add a “cite as” feature for every entry. This function was requested often in comments in the survey and it is so far only indirectly available through our “research baskets”
- consider a change of ecoi.net’s start page, making more room for content
- consider a change of the country pages, making features like “maps” & “laws” more visible
- remove the feature “my documents”
- improve the search help, especially regarding the Thesaurus-based “moderated search”
- remove the out-dated topics&issues files, considering a replacement
Thank you very much for your time & feedback!
If you have further questions or comments, please do not hesitate to contact us: info@ecoi.net
