Spark Trajectory Research Centre

The value of qualitative research

We do a lot of interviews. Bang for buck, they are the best way of finding out about a topic from either employees (colloquially known as users for ill or good) or stakeholders.

Surveys are great, but people hate them now from overuse. We had one client have an intern roam around the office with a laptop coercing employees to participate. Focus groups and workshops are delightful and we love them, but they are time-consuming, expensive, difficult to schedule and suffer from all the well-documented group biases we are now much more aware of. I love a user diary but people can't spare the time. Online workshops are good for project teams, but the dynamics don't work well for regular Joes and Joannes to share their thoughts, let alone Joaquins, Jacintas, Jamilas, and Jians.

In a 30-minute call, the researcher can consistently discuss many topics and get a feel of someone's working life. In a one-hour call with a stakeholder, we can explore the topic from many different angles. It is one of our favourite work activities. It's hard work, but a privilege and, from the point of view of the interviewee, just another call on their calendar. I've had days with early morning calls to New Zealand and late evening calls to Alaska. We've spoken to ship's captains in the middle of the ocean and engineers down mines. We've had research interviews on every continent except Antarctica.

People don't need to prepare, and we get raw and unvarnished opinions, not groupthink. We always offer anonymity to interviewees so that they can speak freely and will also offer the option to take certain comments off-the-record so they won't even appear as anonymised quotes in the outputs.

In the old days, qualitative research was a big old hot mess. We'd all try our best to navigate huge amounts of hand-taken notes from perhaps dozens of interviews. If the interviewee spoke really fast, you'd probably miss the odd golden nugget. The truth is, as soon as you get beyond about five interviews, the relationships between the concepts involved start to get very complex. Who said what, and in what context? The world used to be a tangle of Excel spreadsheets. But modern technology has moved on. Even the sailors and the mine engineers had laptops and fast Wi-Fi, and transcription is universal.

We wanted something better. Something professional and consistent that would allow us to push further and faster.

Fibery and the Research Centre template

Fortunately we were already using Fibery. Fibery is a sort of open relational database that you can make any sort of application. It is often used by product start ups and small companies wanting to resolve everything into one structured application. Try doing that on SharePoint.

We'd seen its potential in 2020 over the first months of Covid, started to use it to run the company and we started using it with clients as soon as we could. We build all of our consulting models in Fibery and they get heavily adapted to the clients needs as we innovate our way through projects.

We can also use it directly for data collection with clients, and then leave the structured data and drafts as well as all other deliverables able to be accessed by clients when we are done. (Fibery is a secure well-managed platform that is both SOC2 and GDPR compliant as I am constantly pointing out to IT security people. It is also hosted in the EU.) The key thing is to stay away from files: things only ever hit a spreadsheet or a document as a client deliverable.

We brought qualitative and quantitative research into Fibery right away (it is particularly good at analysing survey data). In 2024, Fibery released a new highlights system that allowed tagging of specific content from multiple different "Sources" to specific "Targets". For each of these Sources that we set (such as interviews notes, documentary sources or qualitative survey data), we can link them to Targets that exist in our models. Think of it as having a series of classifications that we have already set up that we want to link to. We go through the research source looking for useful and interesting nuggets of information matching them to the right classification.

Fibery then supports us to do that with some AI functionality by suggesting the best Target for the Source. We create some example content in the targets that allows the AI suggestion to match up likely hits in the source. If we have a Target for "Information architecture of the intranet", we will generate some content that would trigger the discovery of that tag with natural language content that people would use to describe structure, sections, links, categories, etc. This mostly works, but it is easily overridden if we have a better idea of how it should be coded. Subsequently, it learns from other sources that we have already coded as the research proceeds.

For example, we might be looking for specific user stories or user journeys (as part of Task Trajectory), or specific management topics like "governance" or "organisational culture" that may have a specific research tag or hypothesis attached, or likely requirements for a future platform (for an Intranet Product Evaluation).

Now it all gets consistently funnelled towards our models and other parts of the research centre. The set up for this looks like this.

The outcome is that we have tagged content from our original sources, say an interview transcript, and none of that person's input is wasted. We can see the cross section of the data that we need, and then drill into the highlight and then read the context around the quote that we chose.

This of course has been a revelation. It is not outsourcing our interview data to an AI and letting it come up with our themes automatically. It is an assistant that massively improves the quality of what we are able to provide.

We can capture semantic relationships between the quote and the target. Does this quote validate someone's hypothesis, or challenge a position. Is what the person is describing blocking or preventing the strategic change of state that the project is seeking. Again, this would have been very difficult without this technology.

Hypotheses

This allows much better control of the research process, freeing up time to consider the insights. What Steve and I now do throughout the project is create and update hypotheses. What do we think is going on? What do we think different stakeholders want as outcomes? These get linked to the data so we can see if they are supported or disproved.

Each has an assumption, a rationale, an expected outcome and a counterargument and we review them, add to them, dispense with them, bring them up in regular meetings with the project team and some may grow into the strategy itself.

This prevents bias, fosters an attitude of inquiry and new ideas throughout the process and helps keep us objective.

Lots of different angles

The reason for all of this is because qualitative research is so very rich. In the course of one sentence, someone might give you some factual information about the current state, describe how frustrating it is and then effortlessly describe the future that would suit them better. The analysis that such data needs depends very much on context and piping this information through a methodology to ensure that it lands where its relevance can be realised.

AI and LLMs, bless their non-deterministic cotton socks, will not be able to do this because they lack the context, structure and experience and no amount of prompting will bring them to heel. They'll give you a confidently incorrect answer, every time. There is no objective or scientific proof that can be wrought from this sort of research. Only sensibly stewarded opinions and questions to take the process forward.

So the bottom line

At Spark Trajectory we have innovative ways of collecting and managing the data that we collect. It gets structured and used more effectively because everything else we do is also structured and the structure brings meaning, consistency and discipline. It's behind the scenes and because we always do anonymised research, the customer won't see the inner workings. But we are proud of it because we know that it ensures that the voice of the employees and the stakeholders that we speak with doesn't get lost in the mêlée.