Intranet satisfaction survey results

Yesterday I blogged about our intranet benchmarking results which compared the intranet against other intranets and gave us an expert evaluation.  Today I’m going to look at survey results from our staff intranet satisfaction survey (run by Customer Carewords) which we ran alongside the benchmarking tests.


Survey format

We used Gerry McGovern’s “Customer Centric Index” method which is designed to get instant, gut-level responses about staff experience of using the intranet.  The survey is designed to report on 3 key areas:

  1. Content
  2. Social
  3. Visual/architectural

Results

We evaluated the intranet from both an expert viewpoint and a staff perspective.  Interestingly, both exercises show similar results.  Which means that we’ve got a good picture of the current “state of the nation.”

So how did we do?  Staff all agree that the new intranet has a great layout, is simple to read and visually appealing. They also said that the information is kept up to date and that the menus are clear and easy to use.  Good to learn that staff actually trust the content, stating it to be open and transparent.  But, (and no surprises here,) our people finder function is pants.

Curiously and annoyingly, it doesn’t look like staff are calling out for “social” functions, like being able to comment or vote on content or to collaborate with each other.  I guess I was hoping that they would.

Problems

One conflicting metric remains.  Staff say they have problems finding content on the intranet.  Conversely, our expert review scored us highly on “findability”, and I unbiasedly agree.  We spend loads of time checking metadata and page titles and formatting documents correctly so that Google can index our intranet content.  To the point of being obsessed.  I regularly tweak the Google Search Appliance based on the analysis I get from reports on users’ search patterns.  So what’s gone wrong?  Why are staff telling us that they can’t find anything?  I just don’t believe it!  Who can’t find stuff with Google when the content is well labelled?

Next steps

So the next project is to find out how staff use search and what their perception of it is.  I suspect the problem is having one HQ intranet and a handful of individual intranets.

Ending on a positive note, the survey results compliment the benchmarking results and confirm that the intranet team  have done a great job on the overhaul in terms of design, content and architecture.  Staff opinion, expert evaluations and comparison against other intranets worldwide show that there is a remarkable overall improvement and that we are making big steps in intranet design.

Intranet benchmark results

Our core intranet team spent the last 2 years working on a project to overhaul and improve the intranet (in addition to publishing daily content and building other sites.)

Project outline (2008-2010)

  1. Research
  2. Information architecture and cardsorting
  3. Wireframe designs and user testing
  4. HTML and CMS build
  5. Content audit and migration
  6. Comms & launch
  7. Evaluation

Benchmark results

We launched in the new year and last month we had our annual benchmarking review. Results are in.

We use IBF as our benchmarking company and we’re members of the Europe group. The MoJ intranet 2010 was benchmarked on “design and usability”.  Our expert evaluation score elevated us into the top 20% of  Europe group members. We also managed to produce the highest score of all IBF members on the expert evaluation of our “design” metric (so, so proud of that one!)

We got some great advice at our feedback session with IBF earlier this week. There are several areas to improve. Some areas we can get to grips with immediately by making quick incremental tweaks. One particular page layout problem brought our usability testing score down – but at least we know what is wrong now and can fix it. Others are going to take time and careful planning and overcoming of obstacles.

Moving forward, we need to tackle intranet strategy and governance and look at ways of introducing collaboration, staff engagement, knowledge sharing, peer to peer and staff to management communication and customisation/personalisation on the intranet. Also a call from stakeholders for intranet stats and analytics.

I’m over the moon that the efforts of the intranet team have paid off and that we’ve been recognised as achieving major steps in improving overall design, findability and accessibility.

The project involved users from the start in research and testing and a lot of the groundwork, in terms of information architecture and site wireframe designs, was in place before we started building HTML prototypes and got the paintbrush out. (I hate working on projects when, at the first meeting, someone says “So, what colour do we want?”) Our coders managed to make the best use of our limited *flat* HTML platform with jQuery and javascript. And our in-house CMS experts developed the news delivery section of the intranet (which was a miracle based on my quirky design specifications and the limited functionality of the CMS platform).

Looking forward to the next phase of evolution…

Bounce rate on the corporate intranet

We’re starting to look at “bounce rate” as a metric for staff engagement on the intranet.

I work with two definitions of bounce rate:

1) Google’s official definition: one page view in one visit
2) Google’s unofficial definition: “I came, I puked, I left”

On the web, analytics packages work on the concept that people visit a website from a search engine or from another campaign medium.  Websites require something to “bring in” the visitors.

On the intranet, we have a captive audience. But if the intranet homepage is given to staff by default when they open the browser, then I think we have a reason not to include it in bounce rate metrics.  Measuring staff interaction should start when they make the first click on the homepage, not when the homepage loads.

The intranet homepage essentially signposts people to other content pages lower down in the intranet structure. I don’t care too much about how many people viewed the homepage.  What I care about is whether they got to the sub-layers of the intranet and consumed content.

For most of my analytics, I exclude the homepage as part of the user journey.  The journey starts when someone “lands” on a lower-level page.  That’s when I start counting bounce rate. If these pages bounce then I know that I have a problem.  If I were to count the homepage, then by definition, these secondary pages would never bounce and would always appear to be working well.

How to setup Google Analytics on an intranet

This post highlights what you need to think about when setting up Google Analytics on an intranet.  Where detailed instructions are available elsewhere I have included links.

Your intranet and web access

Check that staff can access specific Google pages on the web and that your intranet URL is ok.  For example http://intranet won’t work. See specific Google pages.

If your company imposes restrictions on web access you may have to ask to add these addresses to the whitelist of allowed URLs.

In addition, your browser must run javascript and allow cookies.  See first-party cookies.

Get yourself some Google accounts

You need a Google account.  You can register your work email address; this makes it easier when setting up reports later.

You need a Google Analytics account.  Sign up for a Google Analytics (GA) account using the email address registered against the Google account.  You’ll need to create a profile in GA in order to get a profile ID, something like UA-1234567-1. This is the magic profile number that you need to include in your intranet pages.  See examples of the tracking code.

Add the google code to every page

In the process of creating your profile, you’ll see a piece of javascript code that you need to embed into your intranet pages.  This code needs to appear in your published HTML pages.  You’ll need to add the code into your HTML templates or includes or manually into every page.  Google only tracks pages that have the code.  See how to install the tracking code.

Decide how to tackle downloads

The Google tracking code only tracks HTML pages.  It will not automatically track Word documents or PDF files etc.  There are two methods to use in order to track downloads, depending on whether you publish via CMS or manually.

Tracking downloads with a content management system

Inside every anchor link for documents that you want to track, you’ll need to add an extra piece of javascript that will be triggered when the user clicks the link.  See how do I track files?
To track documents, I’d recommend using the URL as it would appear in the natural intranet folder structure:
onClick=”javascript: pageTracker._trackPageview(‘/pensions/docs/membership-form.pdf’); ”

Tracking downloads with manually published HTML pages

Tracking documents for manually published HTML pages is a little trickier, since you won’t want to add bits of javascript to your pages every time you add a link.  There is a javascript trick that we use which runs when the HTML page is loaded and will then seek out all links to documents and add the necessary Google tracking code.
The code will vary depending on your browser and intranet configuration.  I’m not a javascript programmer so please check these websites for ideas (we had to tweak the code to make it work for IE6 – no surprises!)

Tracking downloads from your search engine

Similarly to links to documents within your HTML pages, you’ll need to add code to your search engine results page to track links to documents.

What else do you want to track?

Campaigns

  • Track links from an email promoting an intranet page
  • Track links from a PDF newsletter

Add a bit of code to the end of your URL links from any medium (email, PDF, Word doc etc.)
See the Google campaign URL builder and guidance on using the buider.

Events or user interactions

  • Track “clicks” on a voting button
  • Track “plays” of a video

See the Google event tracking guide.

Limitations

There are a few loopholes and some situations where you can miss tracking.  For example, if you have embedded URLs in a document that link to another document then Google Analytics won’t know about it.  If other intranets or websites link to a document on your intranet then this won’t be tracked. Where possible, encourage external sites to link to your intranet page that contains your download document, rather than the download document itself.