Analytics and evaluation

Intranet satisfaction survey results

Yesterday I blogged about our intranet benchmarking results which compared the intranet against other intranets and gave us an expert evaluation.  Today I’m going to look at survey results from our staff intranet satisfaction survey (run by Customer Carewords) which we ran alongside the benchmarking tests.

Survey format

We used Gerry McGovern’s “Customer Centric Index” method which is designed to get instant, gut-level responses about staff experience of using the intranet.  The survey is designed to report on 3 key areas:

  1. Content
  2. Social
  3. Visual/architectural


We evaluated the intranet from both an expert viewpoint and a staff perspective.  Interestingly, both exercises show similar results.  Which means that we’ve got a good picture of the current “state of the nation.”

So how did we do?  Staff all agree that the new intranet has a great layout, is simple to read and visually appealing. They also said that the information is kept up to date and that the menus are clear and easy to use.  Good to learn that staff actually trust the content, stating it to be open and transparent.  But, (and no surprises here,) our people finder function is pants.

Curiously and annoyingly, it doesn’t look like staff are calling out for “social” functions, like being able to comment or vote on content or to collaborate with each other.  I guess I was hoping that they would.


One conflicting metric remains.  Staff say they have problems finding content on the intranet.  Conversely, our expert review scored us highly on “findability”, and I unbiasedly agree.  We spend loads of time checking metadata and page titles and formatting documents correctly so that Google can index our intranet content.  To the point of being obsessed.  I regularly tweak the Google Search Appliance based on the analysis I get from reports on users’ search patterns.  So what’s gone wrong?  Why are staff telling us that they can’t find anything?  I just don’t believe it!  Who can’t find stuff with Google when the content is well labelled?

Next steps

So the next project is to find out how staff use search and what their perception of it is.  I suspect the problem is having one HQ intranet and a handful of individual intranets.

Ending on a positive note, the survey results compliment the benchmarking results and confirm that the intranet team  have done a great job on the overhaul in terms of design, content and architecture.  Staff opinion, expert evaluations and comparison against other intranets worldwide show that there is a remarkable overall improvement and that we are making big steps in intranet design.

Analytics and evaluation

Intranet benchmark results

Our core intranet team spent the last 2 years working on a project to overhaul and improve the intranet (in addition to publishing daily content and building other sites.)

Project outline (2008-2010)

  1. Research
  2. Information architecture and cardsorting
  3. Wireframe designs and user testing
  4. HTML and CMS build
  5. Content audit and migration
  6. Comms & launch
  7. Evaluation

Benchmark results

We launched in the new year and last month we had our annual benchmarking review. Results are in.

We use IBF as our benchmarking company and we’re members of the Europe group. The MoJ intranet 2010 was benchmarked on “design and usability”.  Our expert evaluation score elevated us into the top 20% of  Europe group members. We also managed to produce the highest score of all IBF members on the expert evaluation of our “design” metric (so, so proud of that one!)

We got some great advice at our feedback session with IBF earlier this week. There are several areas to improve. Some areas we can get to grips with immediately by making quick incremental tweaks. One particular page layout problem brought our usability testing score down – but at least we know what is wrong now and can fix it. Others are going to take time and careful planning and overcoming of obstacles.

Moving forward, we need to tackle intranet strategy and governance and look at ways of introducing collaboration, staff engagement, knowledge sharing, peer to peer and staff to management communication and customisation/personalisation on the intranet. Also a call from stakeholders for intranet stats and analytics.

I’m over the moon that the efforts of the intranet team have paid off and that we’ve been recognised as achieving major steps in improving overall design, findability and accessibility.

The project involved users from the start in research and testing and a lot of the groundwork, in terms of information architecture and site wireframe designs, was in place before we started building HTML prototypes and got the paintbrush out. (I hate working on projects when, at the first meeting, someone says “So, what colour do we want?”) Our coders managed to make the best use of our limited *flat* HTML platform with jQuery and javascript. And our in-house CMS experts developed the news delivery section of the intranet (which was a miracle based on my quirky design specifications and the limited functionality of the CMS platform).

Looking forward to the next phase of evolution…


End of purdah and election 2010

It feels like I’ve had my online mouth clamped for the past month or so due to “purdah”, the rule that prevents civil servants from expressing opinions which might influence election results.  So, to continue…

April and May have been eventful in the intranet team.  Since the announcement of the election date, we have tried to plan for every eventuality, including a name change and a hung parliament.  Now that the new government is in place, we’ve updated details of new ministers and archived news stories from the previous administration. We’ve got new Twitter, Flickr and YouTube accounts.  And a brand new website (but still with no ability to share news stories.)

We have completed our annual intranet usability benchmarking exercise with IBF and also an intranet satisfaction survey using Gerry McGovern’s “Customer Centric Index”.  Should get results in next week.  Can’t wait.  The research will serve as a good benchmark against industry best practice (where I hope we’ve done well) and will give us valuable insight from our users and key stakeholders (where I fear we have a lot more work to do).  I’d like to use this research as a basis to draw up a plan for the next six months using Step Two’s 6×2 methodology.

Meanwhile, back at Headquarters, our IT security department has shut down access to Google Analytics.  An action that will impact on our ability to evaluate and gain insight from staff intranet activity and communications campaigns.  They claim that it’s possible for people outside the firewall to rewrite the intranet.  This is from a department that still installs Internet Explorer version 6 as the corporate browser and who employ third party companies who leave sensitive data on buses.

There are rumours that another government department has shut down access to Yammer, the internal micro-blogging tool.

Despite having a vision of how the intranet could slowly start to engage staff more, allowing them to interact with each other and influence the direction of intranet development, my hopes of ever introducing a collaborative/digital workplace are dwindling in this environment where you just can’t make any progress with IT behind the firewall.  The whole of the civil service is being asked to do “more for less” yet we are forbidden from using modern digital tools that help to increase productivity and staff morale.  Why is it that government is promoting social media and digital innovation but won’t let their own staff use it in the workplace?

Ok, rant over. On a positive note, June should kick off with a great start on the 2nd and 3rd with IBF 24 where I’m looking forward to seeing some great examples of progressive companies who are introducing social media into the workplace and turning over power to the people.


Intranet redesign or drip-feed improvements?

I’m noticing a lot of talk around whether to go for a major redesign or implement small improvements little by little.  I thought I’d give my view on the subject.

I’ve been in my current job for two and half years. In that time the intranet has had a rebrand due to major changes in the organisation back in 2007, a homepage redesign in 2008 and a complete site redesign released at the start of 2010.

We are warned against the dangers of “big-bang” redesigns because staff can’t handle too much change all at the same time. Because of this, when we released our new intranet in January, I was expecting either a great big fanfare and applause or a load of boos and hissing. I got neither. It was business as usual, almost as if nothing had changed. The intranet had a complete new look and feel, a totally reorganised structure and navigation system (based on intense user testing) and still there was very little reaction from the user base.

Having spoken to a few intranet professionals, apparently I shouldn’t be alarmed and this is a good sign that staff are just using the intranet as usual. Unsurprisingly, we haven’t had bags of positive feedback, but neither have we had lots of complaints.

I feel that we are now at a point where we can start to make incremental changes. The intranet is now in a state where we have consistency across the board, all the content is organised. It is not perfect, but I don’t see the need for any further “big-bang” redesigns. Now it’s time to tackle the smaller issues as well as plan for future enhancements such as an improved staff directory and integrating feedback and collaboration features.

I don’t see any harm in a big redesign project in itself where it is appropriate. Sometimes it’s just best to start again rather than try to do lots of smaller fixes. So long as it’s not repeated again and again.


Perfect Human Resources intranet page

Welcome again to the Human Resources section of the intranet

Human Resources reports to the Corporate Services business group, just so you know who you’re dealing with.

These pages contain reference material and information about your employment with the company, plus help and advice and useful forms. We know that may come as a surprise.

We  have designed these pages to be easy to use, and feel the need to tell you about it. In fact, they’re so easy that we even give you detailed instructions on how to use them.

We aim to review this site on a regular basis and to keep these pages up to date.

You’ll find all the options that you actually came here for, hidden way down below the bottom of the screen because to tell you about us and what we do and how fabulous we are and how to use this site is more important than you actually finding what you wanted.

Please bookmark this page so that when we restructure our pages, you’ll get a broken link if you try to come back.

We hope you enjoy your visit to Human Resources.

Thank you for flying air intranet. Good bye now. Good bye. Thank you. Goodbye.

Page updated: 3 May 2008