Logo Clymbol


Methods tested in CLYMBOL

Methods for evaluating consumer understanding

To measure consumer understanding, CLYMBOL’s researchers tested CUT (“Consumer Understanding Test”) as well as the soft and the hard laddering method.

The CUT (“Consumer Understanding Test”) is a method used to objectively analyse the understanding of health claims. Study participants are shown the health claim in context (e.g. in its packaging and/or a (TV) commercial. The statements from the respondents, about the health claims, are categorised as following:
•    Safe: the answer is in accordance with the EU Register of nutrition and health claims made on foods
•    Risky: the answer is not in accordance with the scientific dossiers behind accepted claims
•    Vague: the answer is vague (e.g. “it’s a healthy product”) or irrelevant to the health claim (e.g. “the product is easy to eat”)

Results of the CLYMBOL studies showed that the most common answers were vague or concerned the content of the product, e.g. that the product contained the mentioned nutrient. Answers falling into the Safe category were less common, but still more often than the ones in the Risky category. Increasing the information in the claim didn’t lead to more safe answers, rather in most cases it seemed to result in less safe answers (meaning they were less in accordance with the scientific dossiers that underpin the approved claims).

The laddering method uncovers associations people have about a product. For this, during in-depth interviews, a product’s attributes (e.g. the health claim), perceived consequences of consuming the product and ultimately the personal values people hold are being elicited. Answers from those in-depth interviews can be structured hierarchically, resembling a ladder of importance from the product to one’s own values and beliefs. In soft laddering interviews, participants can talk freely and respond more openly to questions (‘e.g., why is this product attribute important to you?’) than in hard laddering interviews, where a limited number of open-ended questions, in a structured sequence, is presented to the consumers to elicit attributes, consequences and values.

Results from the soft laddering method showed that the most commonly mentioned attributes referred to the health claim itself, e.g. its usefulness and length. The most commonly mentioned consequence of a health claim concerned health and the most noted (personal) value was the quality of life and well-being. Participants also mentioned the health claim’s relevance to themselves and in general. The hard laddering method lead to similar results. Both studies revealed different results for different wordings of a claim. The absence of a health claim was seen as negative by some participants as they wanted to have more information, whereas others preferred the product without a health claims as it was perceived as the regular, therefore familiar, product. However, it was also more likely that participants found the claim too long and had no interest in reading the claim and were irritated or frustrated.


Methods for evaluating purchase behaviour

Olive oil bottleSales/scanner data offer real-time purchase data for a large sample of customers. Food choice and purchase behaviour are not influenced as this type of methodology does not interfere but merely observes and collects data. In cooperation with the German retailer Globus, CLYMBOL researchers compared the sales of an olive oil with the same brand and packaging but with three different claims, during different sales periods. Sales were highest for the olive oil carrying the nutrition claim compared to the versions that carried a general claim (referring to taste) or a health claim. However, as sales/scanner data do not allow for an analysis of why products were purchased, the reason behind these food choices could not be explained. 

Purchase data for the same product can vary for different reasons, including the choice of retailer, as observed effects may be specific to a specific region. Also, a short-term data collection period, here six weeks, may be influenced by seasonal fluctuations such as bank and school holidays. Long-term studies, however, are expensive and it takes a long time for results to be available. Lastly, using sales/scanner data may be difficult to impossible as they are considered sensitive data and retailers may not want to share them.

As such, the sales/scanner data were used as a starting point for measuring purchase behaviour. The served as the reference for additional measures, with the goal of explaining these purchases through other methods. First, an eye-tracking experiment was carried out in an experimental supermarket setting.
Eye-tracking is used to observe consumers’ attention and can be used in laboratory or field experiments. While eye-tracking shows whether the claim was looked at, it does not tell you if and how the claim was understood.
Almost half of the participants fixed their gaze on the general or the nutrition claim on the olive oil, but when a health claim was displayed, most participants didn’t look at it. Results showed a connection between fixation and the results from the sales/scanner data. An olive oil product with a health claim was less likely to be chosen and purchased than one with a nutrition claim.  
Based on these results, additional experiments were conducted using electrodermal reactivity measures to evaluate the arousal of a person.

Arousal is the basis for emotions, motivations and behavioural reactions and is a major factor in consumers’ interaction in a store. This method can be used in field experiments in retail stores and it is almost impossible to influence the test results consciously.
Results from the electrodermal measures showed that an olive oil carrying a nutrition claim led to more arousal than a health claim and much more reactions than a general claim. However, arousal data only shows first (un)conscious approach behaviour towards the different claims, in order to explain the purchases from the sales/scanner data additional variables have to be regarded.

Additional surveys revealed that olive oils with a nutrition or a health claim triggered ‘surprise’ in the respondents. Olive oils carrying a nutrition claim also received higher ratings of uniqueness by the respondents. However, a conclusive way of explaining the sales/scanner data through these surveys for testing purchase behaviour could not be reached and will be subject to further research.


Methods for evaluating consumption behaviour

Two main consumption methods were compared: i) repeated real-world food diary; and ii) one-off laboratory recording of food weight before and after consumption. The validity of food diaries were measured using repeated urine samples (urinary nitrogen is a bio-chemical marker of protein intake), as well as changes in weight or energy balance (via the combination of consumption data with activity watch and physical activity diary data).

The study protocol followed a single-blinded, parallel, randomised, experimental design. Participants (N=48) completed three visits to the University of Surrey, as well as at-home completion of food and physical activity diaries and urine samples.

  • At the first visit, participants completed a welcome questionnaire, anthropometric measurements (height, weight, and hip-to-waist ratio) and were provided with materials to wear an activity watch (to record sleep and activity over the duration of the study), complete a four day food and physical activity diary, and (males only) to collect two 24hr urine samples.
  • CerealAt the second visit, participants were randomised into a health claims or control group. This visit was introduced as a fasted breakfast taste test. A container containing cereal carrying either a health claim (“Oat beta-glucan has been shown to lower blood cholesterol…“) or control label was displayed throughout the session. Participants were asked to complete a hunger questionnaire and then taste and rate the cereal, followed by ad libitum cereal consumption (as they wished for their breakfast). Participants were then provided with a bag of the cereal (carrying either a health claim or a control label) and asked to continue to taste the cereal at home (to increase exposure to the product, either with or without the health claim) as well as complete a second four day food and physical activity diary, and (males only) to collect two further 24hr urine samples.
  • The third visit included completion of an exit questionnaire, repeated anthropometric measurements, compliance interview and un-blinding the study.

In this study, health claims were not seen to influence consumption, meaning researchers could find no statistical significant difference in the consumption, neither in the repeated real-world food diary not in the one-off laboratory recording of food weight before and after consumption. The manipulation check (“Please mark on the line whether you think the cereal might have been bad-good for the heart”) suggests this may have been due to participants not perceiving any difference in the healthiness of the cereal if it did or did not carry a health claim. Nevertheless, the ability to make this conclusion is predicated on the ability to be confident a consumption outcome measure is accurate, valid, and reliable.

The Breakfast Research Study has shown it is possible to carry out a study using more than one measure of consumption. It was a resource intensive process to measure both laboratory consumption (following acute exposure to a heart health claim on a breakfast cereal) as well as field consumption (following repeated exposure to a heart health claim on a breakfast cereal). However, the pros and cons of various measures were exposed, such as the more accurately measured but less ecologically valid laboratory consumption compared to the more real-world, yet reliant on self-report and dietary assessment methods of the food diary consumption measure. Analysis is on-going to assess the validity of food diaries measured via the repeated urine sample (urinary nitrogen is a bio-chemical marker of protein intake), change in weight and energy balance data (via the combination of consumption data with activity watch and physical activity diary).

The limitations of consumption methods are widely known and reported. Therefore, researchers are required to consider various factors to select the most accurate and reliable method that is also feasible and appropriate for the research question under study and to help inform the study design. Recommendations from this piece of work for the future design of studies into health claims include considering whether consumption is the outcome measure of choice, i.e. is it possible or necessary to have a consumption measure that can identify health claim-relevant consumption or are purchase methods adequate proxies of consumption for these purposes? If consumption is to be considered then it this need to be defined, i.e. does consumption refer to the amount consumed or the frequency of consumption or both? Will the outcome measures be food choice, intention to consume or intake measures at the dietary, food or nutrient level? Whatever the answers to these questions are, it appears the single item questionnaire survey measures of consumption frequently employed in the available health claims literature are not likely to not be sufficient measures of consumption. Researchers should be aware of their limitations and consider the possibility of incorporating more than one measure of consumption in their study design.


Back to Second Newsletter