Even though this book was published about 10 years ago, Web Analytics 2.0 by Avinash Kaushik is still rich with information and provides a great overview of applying analytics analytically (lol). It doesn’t focus on using a specific tool, but more on the principles of effective digital data analysis which I find much more valuable.
Something that has been becoming more apparent to me over time is that you get whatever you choose to take from a book. What I mean by this, is you can learn as much as you want from a book that many others believe to be shallow, if you take the time to think critically while reading.
I have been trying to actually transcribe my highlights/notes a bit more while reading, even writing alongside my reading when I have an interesting thought/application of the material. I believe I am getting the most value of actually capturing my thoughts as opposed to purely typing up quotes or excerpts.
Book Highlights & Notes
A brief note on formatting: text in italics is taken verbatim from the book. Other text represents my own notes and annotations.
Chapter 1: Introduction
- The paradox of data: A lack of it means you cannot make complete decisions, but even with a lot of data, you still get an infinitesimally small number of insights.
- Clickstream data: what is happening on your website (Identify)
- Multiple outcomes analysis: how much X is happening?
- Experimentation and Testing/Voice of Customer: why is this happening? (Diagnose)
- Competitive Intelligence: what is happening elsewhere?
- Insights
- Web analytics 2.0 is:
- the analysis of qualitative and quantitative data from your website and the competition, to drive a continual improvement of the online experience that your customers, and potential customers have, which translates into your desired outcomes (online and offline).
- Focusing deeply and specifically on measuring Outcomes means connecting customer behaviour to the bottom line of the company.
- Knowing how you are performing is good. Knowing how you are performing against your competition is priceless.
Performance-based competitive analysis may be valuable from a benchmarking perspective?
- Multiplicity: multiple constituencies, tools, and types of data sources make it much easier to do effective analysis.
- On multiple tools: Each tool provides insights that, taken together, give you the data you need to succeed.
- Typically most companies only need Clickstream + Outcomes + VoC/Testing
- Using tools such as Coradiant for more technical analytics, is it because suddenly your cart and checkout pages were slow and not making it to your customers? Or is it because of 404 errors on your important pages?
Chapter 2: Choosing a Vendor
- Don’t be data rich and information poor.
- Analysis vs. reporting depends on business needs. Versett should focus on analysis, not reporting
- Introspective Questions
- Do I want reporting or analysis?
- If you really want to analyze data, you need to know the context to make sense of the numbers.
- Do I have the IT strength, business strength, or both?
- Am I solving just for Clickstream of Web Analytics 2.0?
- This is a mindset shift. Are you simply trying to understand clicks or to understand the customer experience explicitly.
- Do I want reporting or analysis?
- Vendor Questions
- What is the dfifference between your tool/solution and free tools?
- Are you 100% ASP (SaaS) or do you offer a software version?
- What data capture mechanisms do you use? (JS snippet, web logs, other)
- Can you calculate the total cost of ownership for your tool?
- What kind of support do you offer? What do you include for free, and what costs more? Is it free 24/7?
- What features in your tool allow me to segment the data? Segmentation is the key to finding insights. You segment, or you die.
- What options do I have for exporting data from your system into our company’s system?
- What features do you provide for me to integrate data from other sources into your tool?
- Can you name two new features/tools/acquisitions your company is cooking up to stay ahead of your competition for the next three years?
- Why did the last two clients you lost cancel their contracts? Who are they using now? May we call one of these former clients?
- Make sure you run a pilot and review the SLA for any missing things you expect.
Chapter 3: Analysis I – Metrics
- Life is about taking action; if you are not driving action with your data, stop and re-assess.
- Old Metrics
- Hits = How Idiots Track Success
- Measured in aggregate, page views mean nothing.
- We are moving to outcome-based metrics.
- Evolution: Hits -> Page Views -> Visits -> Outcomes
- We are still in outcome phase
- A KPI is a metric that helps you understand how you are doing against your objectives.
- Think of this like the KR of your O. (OKRs)
- Visits == Sessions, a collection of requests.
- Unique visitors = number of people who come to your website
- This is done via cookies so if more people are blocking these, numbers may get inflated over time
- Daily unique visitors is useless if you look at a timespan longer than a day. People will get double counted.
- This same rule applies to Weekly:
:Multiple Weeks. - Ex: For daily, each individual daily for days 1-7 are good, but the aggregate/sum for those days is bad
because they double count.
- This same rule applies to Weekly:
- If you have access to Absolute Unique Visitors, use this number. It represents unique individuals without any duplication.
- Session time requires 2+ pages to be seen – calculated as the difference between loading pages
- You are always missing the time on the last/exit pages
- Bounce rate = the percentage of sessions on your site with only one pageview
- Some sites use: exit <5s to calculate bounce rate
- Bounce rate represents zero engagement, not a single follow on click!
- Blogs can be an exception to bounce rates since many times people will only come to read the one page
- “Entry Page” can help reveal your SEO landing/home pages – home pages according to google and referrers
- Where are people exiting? Is this what you expect/want?
- Pair bounce rate + exit page to see which pages people come to and leave without ever engaging
- Conversion Rate = Outcomes/Visits (or unique visitors)
- Using visits assumes a conversion should happen each visit
- Using visitors assumes people browse over multiple sessions (more likely)
- Engagement:
- degree = amount of engagement (X pages/visit)
- Kind = hard to measure. Is this good or bad engagement?
- How do you plan to measure engagement for a product/site? Depends on the value you intend to provide.
- Consider getting qualitative data to learn more: exit surveys or NPS for example
- Good alternative: customer retention over time.
- Four attributes of great metrics
- Uncomplex
- Relevant
- Timely
- Instantly Useful
- A simple report of “what’s changed” this month is very useful for identifying symptoms
- If you can’t take action on a metric, perhaps eliminate it.
- An educated mistake is better than no action at all.
- Pick a critical few metrics to measure your business. Don’t overdo it.
- Root Cause Diagnosis
- Identify the influencing levers of your target metric.
- Which ones should you focus on?
- This helps you better understand key metrics and your KPIs
- Macro Insights
- How many Visitors are coming to my site?
- Where are Visitors coming from?
Referring URLs & Search Keywords- Are there any surprises?
- How do these keywords reflect intent?
- What do I want visitors to do on the website?
- Why does this website exist?
- What are your top 3 acquisition strategies?
- What do you think should be happening on your site?
- What are visitors actually doing?
- Top entry pages—home pages are dead.
- Top viewed pages
- Site overlay/click density analysis
- Understand Visitor intent
- Understand navigational challenges
- Abandonment Analysis
- Where are people abandoning your funnels?
Chapter 4: Analysis II – Solutions
- Traffic Report
- What are the high-level trends? How is your new vs. Return ratio?
- Pages/Visit and Avg. Time on Site are typically not correlated:
usually people see lots of pages, can’t find what they want, and spent little time.
- Visitor Report
- Sometimes ad campaigns show up as direct traffic if not coded correctly
- Low direct traffic may imply poor retention
- If your referring sites % is low, perhaps you are not link-worthy.
- SEO usually accounts for 33-50% of traffic, though this depends on the business.
- It’s important to understand customer intent—what are they searching to find you?
- Sometimes ad campaigns show up as direct traffic if not coded correctly
- Which of your pages are key points of failure?
- Compare Top Entry Pages vs. Bounce Rate
- Compare Top 25 Keywords vs. Bounce Rate
- Use A/B testing or interviews to find the cause or improve the outcome
- Click Density Analysis can help you get into the customer mindset (visualize stats on landing page design)
- How many visits does it take to purchase?
- Some analytics tools will have this metric.
- Compare Visits to Purchase with Days to Purchase to better understand customer behaviour
- Most data is better understood through trends/changes over time, not static numbers lacking context
- Setting up goals is essential for any site—if you’re unable to set up goals, are you even ready to launch?
- Segmenting by customers who do your ideal behaviour helps you better understand them: what are they doing differently? What are your worse customers doing?
- How can you determine customer intent?
- Site search usage
- What are people searching for?
- Site search quality
- Bounce Rate vs. Search Term (sometimes called Search Exit)
- Are people refining their searches after? (This is bad)
- Segmenting
- Conversion rate by searchers vs
non -searchers
- Conversion rate by searchers vs
- Site search usage
- Make sure all campaigns are tagged appropriately based on your tracking tool
- Can you answer how PPC/SEO/other paid campaigns affect the bottom line?
- It’s important to not just look at click performance from campaigns, but conversions that are happening as a result—more clicks isn’t always good!
- Be sure to measure not just email metrics from email campaigns, but on-site behaviour too like bounce rates and conversion.
- Try to be efficient with your data collection; some service providers are pay per metric/collection/view, but also adds to your time required for sifting through it all
- Historical data is not that important, trends are though.
- Keep a single spreadsheet of data you want to keep for the future
- On Video Tracking:
- It’s hard to manage scale and get lucky with the session you’re watching
- Probably not worth pursuing – other ways to get the data more efficiently
- Sticking to one tool eliminates the need to spend time on data reconciliation
Chapter 5: KPIs
- Focus on the critical few, not the insignificant many.(3-4 tops). All other metrics should provide visibility into these core targets.
- Why does your product exist? This helps inform the metrics you should target
- If one metric could identify if your business is going up in flames or not, which one would it be?
- For ecommerce, pair Cart Abandonment with Checkout Abandonment
- Cart: 1 – (Start Checkout/Add To Cart Clicks)
- Checkout: 1 – (Complete checkout/Start checkout)
- Pair Conversion Rate with Average Order Value to ensure you are still converting high value transactions.
- Understand why people visit your site. A 2% conversion rate, for example, could be great if only 20% of visitors intend to buy. You can learn this from:
- Exit Surveys
- Viewing what content people are visiting
- Macro & Micro Conversions: Macro is your main goal, but there are likely a few more sub-goals for your site.
- Are we effectively converting people going to the careers page?
- Are we providing valuable support?
- Are people creating new accounts?
- Tie some of your calculations to offline metrics such as LTV of a customer when calculating the vale of conversions
Chapter 6: Qualitative Testing
- When making usability recommendations, categorize them as Urgent, Important, and Nice to Have
- Consider using on-page surveys to collect more qualitative “Why?” Data
- This could work like a “have something to add?” prompt you may use on iOS for rating
- Three crucial survey questions:
- What is the purpose of your visit to our website today?
- Were you able to complete your task?
- If no, why not?
- These questions are typically in an ‘exit survey’
- What is the Versett standard analytics toolkit?
- GA
- Amplitude
- Survey Tool
- NPS tool
Chapter 7: Testing & Experimentation
- A/B tests are best for testing big changes to layouts
- Your experiments are only as good as the ideas you put into them
Biggest impact is likely your low performing landing pages and funnels, not CTAs- To test
physical impact of online advertising, run targeted ad campaigns by geography and seeimpact in those specific regions compared to rest ofmarket - To build a culture of testing, share the times you were super wrong and testing showed you that
- Always start with a hypothesis, it needs to have a built-in success metric and a threshold to be marked a success
- Solving only for a specific target (conversion rate) may have a negative impact on sub-goals. Pay attention to this
Chapter 9: Solutions for Tricky Analytics
- On the
web there are 2 approaches: Accuracy or Precision.- Prefer Precision. Precision is predictable.
- The slogan for analysis ninjas is: move fast, think smart.
- Dashboards leave interpretation to the executive, and most people who build them often lack organizational context. Use this context to provide insights and recommendations.
- Creating Awesome Dashboards
- Report the critical few metrics
- Create an action dashboard (insights, impact, and actions, not numbers)
- You add value by interpreting trends & supplying context
- Identify the root causes of trends and recommend action
- As a result of this trend, what was the impact on the company and its customers?
- Benchmark and Segment: no metric exists without context. You want insights to jump out, not questions.
- If your dashboard is over a page it’s a report, not a dashboard.
Chapter 11: Guiding Principles I
- Easy ways to add context:
- Compare key metrics by time period (monthly overlay, for example)
- Segment the data
- Compare against the average
- Include a partner metric
- Compare against benchmarks/competitors
- Add institutional context
- When comparing over time, ask “what is different about this year and last?”
- Annotate data with institutional knowledge (label the spikes/dips)
- In aggregate, trends can hide insights and hence dirty the data.
- Compare related segments instead
- The top 10 anything rarely changes, instead look at “The biggest movers” for more interesting data
- Don’t forget to measure behaviour over the long-term, understanding latent behaviour – not everyone buys the first time.
- Don’t forget that bounce rate measures the effectiveness of that page for what someone is searching for.
- Leverage the “long-tail” to find monetize-able opportunities
- Stop spending on SEM for terms already linked with your company, spend it on category key phrases (long-tail words)
- Make sure you aren’t compensating for bad SEO with SEM
- Use Google’s Search-based keyword tool to understand relevant keywords
- Generally, upper funnel keywords have a 0% conversion
rate, but are essential for bringing people to your site - Map your keyword portfolio to each of the life cycle stages – use different metrics for each stage and don’t assume each level of the funnel needs to convert
Chapter 12: Guiding Principles II
- It’s common that people have multiple touchpoints to convert, it’s unclear what to do about this.
- If most people convert on the same day, you don’t have a multitouch attribution problem.
- Path analysis turns out to be a terrible waste of time because the Web is not structured; it’s chaotic. There isn’t one path to success; there are thousands.
Chapter 13: Hiring
- Ask “so what?” three times about your metrics. If you don’t get a recommendation for action, you are using the wrong metric.
- Always ask “and how is this important?”
- Key Attributes of Good Hires
- They get the web [product]
- Mental flexibility and
open to change - Curious: always trying new approaches. Prefer curiosity over intelligence; if you must choose.
- Critical Thinking Skills
- You cannot teach people these traits.
- To test critical thinking, provide the candidate with a real business problem that requires critical and analytical thinking and
ask them to solve it.- No matter what they provide, push back a bit, regardless of
solution . - See how they defend their answer or if they can be swayed.
- See how fast they can think in a corner
- No matter what they provide, push back a bit, regardless of
Chapter 14: Building a data-driven culture
- Give stakeholders the gift of an answer. “What’s one question you wish you could get answered about our website/product?”
- You can’t convince people by puking data out. You can’t expect they’ll figure it out.
- “What’s your point?” – give value not data. Based on that…what should I do?
- Data should exist to serve the needs of the business
- You create understanding by doing rather than by talking.
- Too much data is a self-imposed problem.