Sometimes users don’t act as you’d expect. My recent analysis of search terms that staff use to find the online learning area of the intranet showed some unexpected results.
A while back, our Human Resources department introduced an online learning area to the intranet. They wanted to label it as “JusticeAcademy.” I argued that we shouldn’t use “marketing” language and to try something like “learning and development” or “online training” which would more accurately describe the content within.
After a fair bit of debate, and because all the communications and marketing material was already in process, we went with the “JusticeAcademy” label, although I managed to slip in some keywords to help staff searching for terms like learning and development, e-learning, CBT and online training.
I checked out our intranet analytics reports for the past year, splitting staff searches into 2 groups; “JusticeAcademy” and “learning and development and everything else”. There were 26,139 individual searches for this area of the intranet. And to my surprise, 91% of these searches were for “JusticeAcademy.”
Usability best practice tells us not to use flowery marketing names and instead to use plain language. So why are staff searching for a branded name instead of using regular plain language terminology as expected? The answer is that the HR department have marketed the online learning area of the intranet very effectively. All literature and communications have managed to get the new name across and staff know where to go for their training needs.
Ironically, we are soon launching the newly rewritten HR area of the intranet because content ownership has moved to a different part of the organisation. This area will still incorporate the online learning section and the content owners want to call it “Learning and development”. So now I have to go back to them to say “You know, a few years back I wanted to call it learning and development, but, actually…”
In an attempt to get more out of our analytics data, I’m going to experiment with tagging users as they visit the intranet.
It’s best practice to look at more than the whole picture when analysing web traffic data; segmentation provides greater insight into what is happening out there in the real world. A blanket report of page views for the complete intranet will not provide many insights.
Another good motto that I use is “no lonely metrics” (coined from Avinash Kaushik.) In other words, don’t just give me a number. Always compare one metric against another, for example, a number for this month compared to last month or page views compared to time on page.
On an intranet, based in one country, it’s hard to get the same amount of demographic data that you’d normally find on analytics from a web site. On our public site I can find out where visitors are located and how they found our website. But staff on the intranet all start with the same homepage when they open their browser. Unless they land on the intranet from a tagged email or newsletter, we don’t have much idea where they came from, geographically or organisationally. In fact, the most that we seem to be able to offer in terms of visitor statistics is the number of visitors. Because we are all on the same network, few visits to the intranet are from referrals and there is no geographical diversity.
Consequently, most of our intranet analytics reports are based around intranet pages and documents. There is very little detail about the visitors. We know what people are looking at but we don’t know much about the people. Which means that we can’t segment and chop up our reports from the user perspective. Which means that we are not getting a clear picture.
By using cookies to tag visitors based on the content that they view or by offering a self-service page of check-boxes where staff can tag themselves, I hope to be able to integrate this information into our analytics and produce more informative insights where the emphasis moves from pages to people.
Analytics, evaluation, data and statistics all have the gas turned up in the workplace. Rightly so. We need to make business decisions, evaluate performance, and design usability improvements based on hard evidence.
Facts and figures should generate action. Not sit in a pile of paper on a desk or add to a growing repository of electronic reports, ultimately clogging up our servers and recycle bins.
The problem with generating lists of numbers and handing them to someone is that they don’t mean anything. As Avinash Kaushik says, if you can say ‘so what?’ to a statistic then it was pointless in generating it. You got 10,000 visitors this month. So what? Is that good or bad? Better than last month? How many did you expect to get? What will you do now because you know this?
I really believe we should ban statistics reports unless they are supported by a specific question. What do you want to know? It may be the same thing every month. Fine. Great. Then we can start comparing month on month. Then, reports will start to have meaning. Then we can take action. But first we need questions.
Did my email newsletter produce more traffic than the graphically designed intranet advert? Did the recent page redesign have the desired outcome? Did the news story result in more people signing up to the company initiative? Have more people been reading my pages over the last six months?
On the intranet we have devolved analytics. Our publishers and stakeholders are free to peruse the stats and graphs. We also do a quick training session on how to get the most out of the intranet analytics. This approach reduces wasted paper reports because people only check out the analytics when they have a question. If they are really interested in their content and want to monitor ongoing stats then they can create regular automated reports. It cuts down meaningless requests to the central intranet team, allowing them to concentrate on the bigger picture, and it encourages publishers to become more familiar with the life of their content after they have published it.
Do you spew out endless statistics reports or do you answer questions?
To help staff become acquainted with the new look and feel and functionality before the launch, we published a set of *intranet familiarisation* videos, using the new intranet template, and linked them from the existing intranet via a feature story.
In the month before launch we scheduled several intranet feature articles detailing the forthcoming launch. Our in-house design team created a flyer insert for our monthly staff magazine and we let our stakeholders know through email bulletins and newsletters. We published a final intranet article as we broke up for Christmas holidays saying bye bye to the old intranet.
I was very proud to press the button on New Year’s Day which would set the CMS in motion, publishing the new homepage, the news section and linking to all newly migrated content. It was then time to sit back and wait for the response from staff.
We had managed to migrate all of the content except for some older news stories and we were busy pedalling away in the background as the new intranet went live to staff.
There was an eerie silence after the launch. Business as usual. No cries of joy and no rotten tomatoes. Staff were just getting on with it. Hopefully our comms and familiarisation helped to bed them in a little.
The average time spent in a visit to the intranet has fallen from an average 2 mins 30 down to 1 min 20 since the launch. We’ve almost halved the time that staff take in a typical visit to the intranet. I’m optimistically reading this as evidence to show that they are finding and digesting information faster. Hopefully the improved navigation, legibility and layout are having an effect.
Uptake for feature news has almost doubled since the launch and is steadily rising. All the time spent in planning the news delivery was worth it.
Around 3 months after launch, we evaluated the *usability and design* benchmark as part of our annual IBF review. We also used Customer Carewords again to run the intranet staff satisfaction survey. See my previous blog posts for the results:
Since launch, the intranet has changed and adapted, due to requests for new content and campaigns, issues highlighted by analytics reports and recommended actions from benchmarking evaluation reports.
One glaring problem that came out of our benchmarking report this year was that although I had done a lot of testing with the navigation to ensure staff could find their way around the intranet, I had not paid so much attention to the task of downloading a document. In the new intranet, downloads such as PDF and DOC files were listed in the right hand column. However, staff (rightly so) saw this column as links that would take them elsewhere – *related information*. As a result, they often missed the links to the documents. So we’ve been working to remedy this, moving download links to the body section of the page, in context.
I feel now that the intranet has reached a level where further enhancements can dripfeed through, rolling out small improvements and developments one at a time. Although the intranet is alive and constantly changing, this redesign project gave it a much needed kick in the butt to bring it into a state where no further big bang launches are necessary.
Other intranets within the organisational group have adopted the new design template and this is a good first step in providing a unified experience across our family of intranets.The intranet has come a long way, but work doesn’t stop. There is always room for improvement, as well as trying to keep up with what’s happening out on the web and constant changes and restructures within the organisation. I blogged previously that we are currently working to introduce more engagement functions on news stories, including *Like*, *Share this* and *Comment*.
I’m aware that we don’t have a true task-based navigation system and that several sections still hang off the departmental structure. We have no proper social functions on the intranet because we don’t have the technical architecture or platform. We have *strap-on* sites such as Huddle and Civil Pages but they don’t talk to each other and they don’t integrate with the intranet. We still have the most useless employee directory, again not integrated into the intranet or any other systems such as Outlook. And I’ve been waiting over a year (or is it two?) for our Google Search Appliance upgrade.
So plenty to get on with.
For now, that’s the end of this budget intranet redesign series.
Thanks for all the mentions and RTs over the past weeks.
I have been meaning to write up the phases of our intranet redesign project for a long time. I’m going to blog the 6 phases of the project leading up to launch on New Year’s Day 2010. The project took around a year and a half to complete, with a core team of 4 people, working in addition to our day to day jobs of intranet publishing, website builds and the usual internal comms intranet panics. This was a major overhaul project, aiming at a big-bang relaunch.
This initial phase of the project produced a brief containing the aims, scope and deliverables for the project plus a detailed research analysis.
Alongside my own usability review, I looked at our existing IBF usability benchmarking reports to feed the expert evaluations into the brief.
There was very little imagery throughout the site and pages appeared bland and cumbersome. Design wise, there was little trace of the corporate brand.
Behind the scenes, the folder structure was haphazard, as was filenaming.
Analytics reports and search logs
I analysed the analytics and search logs to find out which content was popular, unpopular, surplus or missing. I looked at page views, time on pages and search phrases. This analysis gave me a good idea of what the staff were browsing and searching for. No prizes for guessing the top pages: job vacancies, forms, people search, pay information. At that time we didn’t have a restaurant menu; missing content!
Gerry McGovern, Customer Carewords and contextual navigation
We used Gerry McGovern’s Customer Carewords methodology to survey staff with the objective of finding out what they wanted to do on the intranet in terms of top tasks.
Around the same time, I tuned into a webinar that Gerry was hosting in which he talked about contextual navigation. The practice of only showing menu options relative to the page content. He talked about how the BBC website used this technique and demonstrated how menus adapt, for example, from sports, to football, to first division football without still showing cricket or tennis. I made a note to experiment with this.
We spoke with stakeholders from across the organisation with particular emphasis on internal communications. There were lots of detailed discussions about news and how we were going to present news articles. The current 4 categories of news needed simplifying.
Having looked at all the evaluations, research, stats and surveys, we developed a basic brief, summarised below:
Improve the overall site design and layout (modern look and feel, improved readability and ease of use)
Use the internal brand
Redesign the information architecture by auditing and recategorising content
Implement a simplified navigation system (test contextual navigation)
Simplify the news categories and improve delivery
Rectify the flow of stories from the homepage to news archives
Redesign the search results interface
Overhaul intranet content metadata to give a solid search experience
Standardise folder structure and filenames
No end date! Project Manager nightmare! But we knew that we would have to work on the project alongside our day jobs (where anything can happen!) and only decided on a launch date as we prepared for content migration (phase 5).